paths_allowed

data.frame generated by <code>rt_get_permissions()</code> with
three columns: 'useragent' (name of the bot), 'permission' (permission
granted: Disallow/Allow) and 'value' (path for which permission applies)

permissions

paths for which to check bot's permission, defaults to "/"

paths

wrapper to <code><a href="/link/path_allowed?package=robotstxt&version=0.1.2" rd-options="" data-mini-rdoc="robotstxt::path_allowed">path_allowed</a></code>

internal

Provides a 'Robotstxt' ('R6') class and accompanying methods to
    parse and check 'robots.txt' files. Data fields are provided as
    data frames and vectors. Permissions can be checked by providing
    path character vectors and optional bot names.

paths_allowed: check if a bot has permissions to access page

Description

Usage

Arguments

See Also