Companion helper that separates the three independent questions
get_parse_status collapses into one value: did the input
present well-formed URL syntax (layer 1), was the parsed object admitted
under the active policy (layer 2), and what did the Public Suffix List
annotation find (layer 3).
get_parse_verdicts(
url,
url_standard = NULL,
protocol_handling = c("keep", "none", "strip", "http", "https"),
scheme_relative_handling = c("keep", "http", "https", "error"),
scheme_policy = c("infer", "require"),
scheme_acceptance = c("web", "general"),
tld_source = c("all", "private", "icann")
)A data frame with one row per element of url and three
character columns: layer1_syntax_verdict,
layer2_policy_verdict, and layer3_annotation_state.
A character vector of URLs.
Standard profile governing interpretation: NULL
(default), "rfc3986", or "whatwg". Unlike the other
companion helpers this argument does not gate the result.
A character string specifying how to handle
protocols. Defaults to "keep".
Regardless of this option, rurl only processes authority-based URLs whose
scheme is one of http, https, ftp, or ftps; a scheme-bearing input with any
other scheme (e.g. mailto:, tel:, ws:) yields
parse_status = "error". Scheme inference (below) also requires the
input to be host-shaped: a scheme-less string that is not a host (e.g.
"asdfghjkl", "12345", "/path") or is a non-canonical
IP literal (integer/hex/octal/short forms, or leading-zero octets like
"192.168.010.1") is rejected as "error" rather than having a
scheme fabricated for it.
"keep": If a supported scheme exists (http, https, ftp, ftps), it's
used. If no scheme and the input is host-shaped, "http://" is added;
otherwise the input is not a URL and yields "error".
"none": If a supported scheme exists, it's used. If no scheme, then no scheme is used (scheme component will be NA).
"strip": Any existing scheme is removed (scheme component will be NA).
"http": The scheme is forced to be "http".
"https": The scheme is forced to be "https".
How to handle URLs starting with "//". Defaults to "keep".
"keep": Parse using http but return scheme as NA and set status to "ok-scheme-relative".
"http": Assume http for parsing and output.
"https": Assume https for parsing and output.
"error": Treat scheme-relative URLs as invalid.
Controls whether scheme-less, host-shaped input is
accepted (an input-acceptance axis, distinct from protocol_handling,
which only controls how the scheme is presented, and from url_standard,
which controls interpretation). Defaults to "infer".
"infer": (Default) Fabricate http:// for scheme-less host-shaped
input (e.g. example.com parses as http://example.com), a
browser-omnibox-style affordance. This is the historical behavior.
"require": Reject scheme-less input — a scheme-less host-shaped
value becomes parse_status = "error" rather than gaining a fabricated
scheme. Use this for a strict, pure-parser posture. Note this governs
only bare host input; scheme-relative //host input is governed
separately by scheme_relative_handling.
Which scheme tokens may enter parsing (a
scheme-acceptance axis, distinct from scheme_policy, which governs
scheme-less input, and from url_standard, which governs interpretation).
Defaults to "web".
"web": (Default) Only the curated web-scheme allowlist
(http/https/ftp/ftps/file) is admitted; a scheme-bearing input
outside it is parse_status = "error". This is the historical,
byte-for-byte compatible behavior.
"general": Admit any syntactically valid scheme token and parse
opaque (mailto:x), non-special (foo://host), and RFC-generic URLs.
Requires an explicit url_standard ("rfc3986" or "whatwg"), which
decides the interpretation; general with url_standard = NULL is an
error. Non-special / opaque hosts receive no www-stripping, no domain/TLD
derivation, and are never run through the IDNA/punycode helpers.
A non-special scheme with no // is an opaque path: it has no
authority, so host, user, port and the domain/tld columns are
all NA and the entire remainder is the path (query/fragment are
still split off). This includes mailto: — the recipient's @ never
re-triggers authority parsing. To decompose a mailto: recipient, use
the accessors (get_host() / get_domain() / get_user(), ADR 0012 D7)
or get_mailto_recipients(); those deliberately return a recipient's
parts where this table presents NA, because a recipient domain is
extraction metadata, not the URL's authority.
Note that file: is admitted under both values, including the default.
A file: URL denotes local-filesystem access, and one with a non-empty
host (file://server/share/x) is a UNC path on Windows, so dereferencing
it reaches a remote SMB share. rurl parses file: URLs; it never opens
them. Restricting schemes before anything dereferences them is the
caller's job — see SECURITY.md.
Which TLD source to use for TLD extraction: "all", "icann", or "private". Defaults to "all".
get_host_type and get_scheme_class
require a selector and error without one (ADR 0015). That
requirement is not a house style --- it is a consequence of what those
helpers report. Their content is standard-relative: whether a host
is an IPv4 address, or a scheme is special, is a question only a standard
can answer, so without a selector there is no fact to return.
Verdict layers are not standard-relative in that way. A syntax failure
(layer 1) and an admission rejection (layer 2) are facts about the parse
that this call actually performed, under whatever options were
supplied. Those facts exist at url_standard = NULL exactly as they
do under a selector, so there is nothing to withhold.
Gating anyway would also break the helper's central guarantee. Because
parse_status is the projection of these three layers, projecting
the reported layers must reproduce the reported status. Refusing to answer
at url_standard = NULL while parse_status still reports a
real value would make the companion contradict the column it exists to
explain --- on the most common call, and precisely when a caller is asking
why a default-options parse failed.
Layer 1 --- syntax ("pass" / "fail"). Whether a
syntax failure was observed. "pass" is the absence of an
observed failure, not a proof of conformance --- a row the admission gate
rejected before any parse ran reports its rejection in layer 2 and is not
additionally reported as a syntax failure.
Layer 2 --- policy ("admitted",
"admitted-scheme-relative", "admitted-ftp",
"rejected-scheme", "warn-userinfo"). Whether the object is
admitted under the active scheme_acceptance /
scheme_relative_handling / userinfo policy, and with what note.
"rejected-scheme" rejects; "warn-userinfo" accepts with a
note; the admitted* values accept.
Layer 3 --- annotation ("not-applicable",
"known", "unknown", and the currently unproduced
"not-requested", "invalid-input", "dependency-error").
The PSL registrability fact as a typed state rather than a bare
NA, and never fatal: a host with no public suffix is a
"unknown" annotation, not a parse failure. A host form with no
registrable-domain concept at all --- an IP literal, a file: host,
an opaque authority --- is "not-applicable", never "unknown".
The single parse_status value is a lossy projection of these
three. The guaranteed loss is that a structural syntax failure and a policy
rejection both surface as "error": under the default
scheme_acceptance = "web", "mailto:[email protected]" and
"http://" are both "error", but the first is
layer2_policy_verdict = "rejected-scheme" (rurl declined to accept the
scheme) while the second is layer1_syntax_verdict = "fail" (it did not
parse). Recovering that distinction is what this helper is for.
Like the other companion helpers this never widens the
safe_parse_url frame --- the parse table keeps its 18 columns
and the verdicts live here. Unlike get_host_type and
get_scheme_class, which require a selector, it is fully
defined at url_standard = NULL: layers 1 and 2 describe the parse
that actually occurred, which happens with or without a standard selector.
get_parse_status, get_url_diagnostics,
safe_parse_url
# Both are parse_status "error" -- for entirely different reasons.
get_parse_verdicts(c("mailto:[email protected]", "http://"))
# A PSL miss is an annotation state, never a fatal verdict.
get_parse_verdicts("http://example.invalidtld/")
Run the code above in your browser using DataLab