rt_get_rtxt

name of the robots.txt files, defaults to a random drawn file ;-)

name

load robots.txt files saved along with the package:
these functions ar very handy for testing (not used otherwise)

internal

Provides a 'Robotstxt' ('R6') class and accompanying methods to
    parse and check 'robots.txt' files. Data fields are provided as
    data frames and vectors. Permissions can be checked by providing
    path character vectors and optional bot names.

rt_get_rtxt: load robots.txt files saved along with the package

Description

Usage

Arguments