check if a bot has permissions to access page
path_allowed(permissions, path = "/", bot = "*")
data.frame generated by rt_get_permissions()
with
three columns: 'useragent' (name of the bot), 'permission' (permission
granted: Disallow/Allow) and 'value' (path for which permission applies)
path for which to check bot's permission, defaults to "/"
name of the bot, defaults to "*"
TRUE permission granted; FALSE permission denied; NA path_allowed() cannot determine if path is allowed or not allowed, will result in a warning as well