Tabular companion to get_scheme and
get_scheme_class, and the scheme-axis counterpart of
check_hosts: for each URL, reports the parsed scheme, its
special/non-special classification, whether it falls inside the built-in
web-acceptance set, an optional allowed column scored against a
caller-supplied allowlist, and a reasons list-column naming the
scheme facts observed.
check_schemes(
url,
allowed_schemes = NULL,
url_standard = "whatwg",
scheme_policy = c("infer", "require"),
scheme_acceptance = c("general", "web")
)A data.frame with one row per input URL (input order
preserved): url, scheme (the parsed scheme, lowercased, or
NA), scheme_class ("special", "non-special" or
"missing-or-error"), web_scheme (is the scheme one of the
five scheme_acceptance = "web" admits), allowed (only when
allowed_schemes is supplied), and reasons --- a list-column
whose i-th element is a character vector of the scheme facts observed
(character(0) when none). The tokens are "no-scheme",
"special-scheme", "non-special-scheme",
"outside-web-acceptance" and "not-in-allowlist".
There is deliberately no token for “carries no authority”. It is
the natural fact to want for mailto: and javascript:, but
it cannot be computed from the public parse record:
get_host("mailto:[email protected]") returns
"example.com", reading the @ as a userinfo delimiter, so a
token derived from host presence would be wrong for exactly the schemes it
is most wanted for. Reporting a fact this package cannot compute correctly
would be worse than not reporting it.
A character vector of URLs.
Optional character vector of schemes the caller will
act on, compared case-insensitively. When supplied, the result gains a
logical allowed column (FALSE for a URL with no parsed
scheme) and the token "not-in-allowlist" where it is FALSE.
When NULL (default) no allowed column is emitted and no
allowlist judgement is made.
Standard profile governing scheme interpretation:
"whatwg" (default) or "rfc3986". Unlike most of the package
this argument has a non-NULL default, because a scheme
classification is undefined without a named standard.
Controls whether scheme-less, host-shaped input is
accepted (an input-acceptance axis, distinct from protocol_handling,
which only controls how the scheme is presented, and from url_standard,
which controls interpretation). Defaults to "infer".
"infer": (Default) Fabricate http:// for scheme-less host-shaped
input (e.g. example.com parses as http://example.com), a
browser-omnibox-style affordance. This is the historical behavior.
"require": Reject scheme-less input — a scheme-less host-shaped
value becomes parse_status = "error" rather than gaining a fabricated
scheme. Use this for a strict, pure-parser posture. Note this governs
only bare host input; scheme-relative //host input is governed
separately by scheme_relative_handling.
Which scheme tokens may enter parsing (a
scheme-acceptance axis, distinct from scheme_policy, which governs
scheme-less input, and from url_standard, which governs interpretation).
Defaults to "web".
"web": (Default) Only the curated web-scheme allowlist
(http/https/ftp/ftps/file) is admitted; a scheme-bearing input
outside it is parse_status = "error". This is the historical,
byte-for-byte compatible behavior.
"general": Admit any syntactically valid scheme token and parse
opaque (mailto:x), non-special (foo://host), and RFC-generic URLs.
Requires an explicit url_standard ("rfc3986" or "whatwg"), which
decides the interpretation; general with url_standard = NULL is an
error. Non-special / opaque hosts receive no www-stripping, no domain/TLD
derivation, and are never run through the IDNA/punycode helpers.
A non-special scheme with no // is an opaque path: it has no
authority, so host, user, port and the domain/tld columns are
all NA and the entire remainder is the path (query/fragment are
still split off). This includes mailto: — the recipient's @ never
re-triggers authority parsing. To decompose a mailto: recipient, use
the accessors (get_host() / get_domain() / get_user(), ADR 0012 D7)
or get_mailto_recipients(); those deliberately return a recipient's
parts where this table presents NA, because a recipient domain is
extraction metadata, not the URL's authority.
Note that file: is admitted under both values, including the default.
A file: URL denotes local-filesystem access, and one with a non-empty
host (file://server/share/x) is a UNC path on Windows, so dereferencing
it reaches a remote SMB share. rurl parses file: URLs; it never opens
them. Restricting schemes before anything dereferences them is the
caller's job — see SECURITY.md.
Like check_hosts this is a policy layer, not parser
conformance: it never changes how a URL parses. Restricting which schemes
your application will act on is the caller's decision, and this helper
supplies the facts to make it --- it does not make it for you.
Every token names something observable about the parse. None is a risk
label, and none is intended as one: whether a scheme is safe depends on what
the caller does with the URL, which this package cannot observe. A
scp: URL is unremarkable to a mirroring tool and unacceptable in an
HTTP fetcher, and no property of the string distinguishes those cases.
check_hosts for the host axis,
get_scheme_class, get_url_diagnostics.
urls <- c(
"https://example.com/a", # special, inside web acceptance
"ftp://example.com/a", # special, inside web acceptance
"scp://host/a", # non-special, outside web acceptance
"smb://server/share", # non-special, outside web acceptance
"mailto:[email protected]", # non-special, opaque path
"javascript:alert(1)", # non-special, opaque path
"notaurl" # no scheme
)
check_schemes(urls)
# Score against the schemes an application will actually act on:
check_schemes(urls, allowed_schemes = c("https", "http"))
Run the code above in your browser using DataLab