Companion helper for the url_standard selector: reports the
non-fatal validation/safety facts rurl observed while parsing each
URL (for example an IPv4 host written in a numeric or non-decimal shorthand,
or a path segment carrying an encoded reserved byte). Diagnostics are facts,
not policy: they are emitted keyed to host/path shape in both
standard modes so a security-sensitive consumer can reject a footgun URL
regardless of which selector it chose, while a link-graph consumer can ignore
them. The complete token vocabulary is enumerated below under
Diagnostic vocabulary (canonical).
get_url_diagnostics(
url,
url_standard,
scheme_policy = c("infer", "require"),
scheme_acceptance = c("web", "general")
)For a length-1 url, a character vector of zero or more
diagnostic tokens for that URL. For a length-n url (including
n == 0), a list of length n whose i-th element is the character
vector of that URL's tokens (character(0) when it has none).
An empty result means exactly one thing: that URL raised no
diagnostics under the standard you named. It can no longer also mean that
no selector was passed, because omitting url_standard is an error
rather than a mode (ADR 0015). character(0) is therefore evidence
that the URL is clean under that profile.
A character vector of URLs.
Standard profile governing interpretation: either
"rfc3986" or "whatwg". Required, with no default
(ADR 0015): the vocabulary is profile-dependent --- several tokens fire
under one standard only --- so there is no profile-neutral set of findings
a default could stand for.
Controls whether scheme-less, host-shaped input is
accepted (an input-acceptance axis, distinct from protocol_handling,
which only controls how the scheme is presented, and from url_standard,
which controls interpretation). Defaults to "infer".
"infer": (Default) Fabricate http:// for scheme-less host-shaped
input (e.g. example.com parses as http://example.com), a
browser-omnibox-style affordance. This is the historical behavior.
"require": Reject scheme-less input — a scheme-less host-shaped
value becomes parse_status = "error" rather than gaining a fabricated
scheme. Use this for a strict, pure-parser posture. Note this governs
only bare host input; scheme-relative //host input is governed
separately by scheme_relative_handling.
Which scheme tokens may enter parsing (a
scheme-acceptance axis, distinct from scheme_policy, which governs
scheme-less input, and from url_standard, which governs interpretation).
Defaults to "web".
"web": (Default) Only the curated web-scheme allowlist
(http/https/ftp/ftps/file) is admitted; a scheme-bearing input
outside it is parse_status = "error". This is the historical,
byte-for-byte compatible behavior.
"general": Admit any syntactically valid scheme token and parse
opaque (mailto:x), non-special (foo://host), and RFC-generic URLs.
Requires an explicit url_standard ("rfc3986" or "whatwg"), which
decides the interpretation; general with url_standard = NULL is an
error. Non-special / opaque hosts receive no www-stripping, no domain/TLD
derivation, and are never run through the IDNA/punycode helpers.
A non-special scheme with no // is an opaque path: it has no
authority, so host, user, port and the domain/tld columns are
all NA and the entire remainder is the path (query/fragment are
still split off). This includes mailto: — the recipient's @ never
re-triggers authority parsing. To decompose a mailto: recipient, use
the accessors (get_host() / get_domain() / get_user(), ADR 0012 D7)
or get_mailto_recipients(); those deliberately return a recipient's
parts where this table presents NA, because a recipient domain is
extraction metadata, not the URL's authority.
Note that file: is admitted under both values, including the default.
A file: URL denotes local-filesystem access, and one with a non-empty
host (file://server/share/x) is a UNC path on Windows, so dereferencing
it reaches a remote SMB share. rurl parses file: URLs; it never opens
them. Restricting schemes before anything dereferences them is the
caller's job — see SECURITY.md.
The diagnostics are deliberately a selected set of facts, not a complete validator. The absence of a diagnostic never implies the URL conforms to its scheme's specification or to WHATWG/RFC 3986. Full per-standard conformance validation is out of scope (ADR 0012 D5).
Two WHATWG-generic facts gate on the interpreting standard, not the
acceptance axis, so they are reported whenever url_standard =
"whatwg" --- including the default "web" acceptance path (they are
route-independent, string-level facts; RURL-sgjzbqzk):
invalid-URL-unit / invalid-credentials --- WHATWG
validation errors (WHATWG-verbatim names): a malformed %-escape
or a non-URL code point, and any credentials (userinfo) present.
Bounded detection.
The default combination ("web" acceptance with url_standard =
NULL) emits no diagnostics at all, so it is unaffected.
With scheme_acceptance = "general" (the general-parser posture) a
further set of selected facts is reported. These fire only under
"general"; the default "web" acceptance never emits them:
unicode-outside-rfc3986-uri --- under "rfc3986", a
directly-written non-ASCII scalar value accepted by the sole RFC 3986
generic-grammar tolerance (not RFC 3987/IRI conformance).
transform-skipped-ineligible-scheme --- the scheme is
non-HTTP(S) and so ineligible for the SEO/semantic Stage-B transforms.
scheme-specific facts: ws-fragment-forbidden /
ws-userinfo-forbidden (RFC 6455),
mailto-fragment-discouraged (RFC 6068),
tel-missing-phone-context (RFC 3966),
data-missing-comma (RFC 2397), and, under "rfc3986",
file-non-absolute-path,
file-userinfo-extension (userinfo, permitted by RFC 8089
Appendix E.1's non-normative extended grammar), and
file-component-outside-rfc8089 (a port, query or fragment,
which RFC 8089's grammar does not mention and which are therefore
inherited generic RFC 3986 components).
This section is the
single authoritative enumeration of the diagnostics vocabulary.
It is held to the runtime registry (.URL_DIAGNOSTICS) in both
directions by tools/diagnostics-doc-consistency.R, a CI gate: a
token cannot be added, renamed, or removed without this list moving with
it. Earlier design documents (including the v1 selector PRD's section 7
table) are historical records of what the vocabulary was when they were
accepted --- they are not registries and do not track it.
Every token below is emitted only when url_standard is not
NULL. Tokens marked general additionally require
scheme_acceptance = "general"; the rest fire under both acceptance
postures and, unless noted, under both "rfc3986" and
"whatwg".
Host --- IPv4 shape. Facts about a host written as, or coerced to, an IPv4 address; security filters typically reject all of them.
ipv4-number-form --- numeric IPv4 shorthand instead of
dotted decimal.
ipv4-non-dotted --- a whole-host number parsed/coerced to
IPv4 in WHATWG mode.
ipv4-short-form --- fewer than four dotted parts.
ipv4-non-decimal --- hex or octal notation participated in
IPv4 parsing.
ipv4-octal --- octal interpretation changed the apparent
address value.
ipv4-leading-zero --- a dotted decimal-looking part had a
leading zero.
ipv4-out-of-range --- a dotted part exceeds 255 (fatal
under "whatwg"; flags a numeric-looking reg-name under
"rfc3986", e.g. 256.1.1.1).
Host --- DNS length, UTS-46 and charset. Probed against the resolved host; IP literals are excluded.
domain-label-too-long --- a label exceeds the DNS 63-byte
limit.
domain-name-too-long --- the whole name exceeds the DNS
253-byte limit.
domain-empty-label --- the host contains an empty label
(a ".." run, or a leading dot).
domain-hyphen-violation --- a label breaks the UTS-46
hyphen rules (leading/trailing hyphen, or "--" in positions
3--4 of a non-xn-- label).
domain-std3-violation --- a label carries a code point
outside the STD3 LDH set.
host-charset-shimmed --- the host carries one of the 15
code points WHATWG keeps but the historical parser rejected,
accepted by the shim (ADR 0009: ! $ & ( ) * + , ; =, plus the
ASCII quotation mark, apostrophe, grave accent, and the two braces).
"whatwg" only.
Path.
encoded-dot-segment --- an encoded-dot segment
(%2e / %2e%2e, any hex case) that the profile's dot
handling acted on.
encoded-reserved-path-byte --- the preserved path still
carries an encoded reserved byte (%2F, %3F,
%23) held as data rather than as a separator.
Port. Facts about the raw port versus the resolved scheme's
WHATWG default, independent of the port_handling knob.
explicit-default-port --- the port was written out and
equals the scheme's default.
non-default-port --- a port is present and is not the
scheme's default (including any port on a scheme with no defined
default).
Input shape --- WHATWG cleanup. All three are "whatwg"
only; "rfc3986" has no strip or rewrite step.
invalid-reverse-solidus --- a literal \ was
reinterpreted as / (special schemes only).
control-char-stripped --- an ASCII tab/LF/CR was removed
from the interior of the input (step 1, second half).
leading-trailing-stripped --- a leading and/or trailing run
of C0-control-or-SPACE was removed (step 1, first half).
Layer 5 --- selected per-standard and per-scheme facts (ADR 0012 D5). Described in full under Selected facts, not a conformance oracle above.
invalid-URL-unit --- WHATWG validation error: a malformed
%-escape or a non-URL code point. "whatwg" only.
invalid-credentials --- WHATWG validation error:
credentials (userinfo) are present. "whatwg" only.
unicode-outside-rfc3986-uri --- general; a
directly-written non-ASCII scalar value under "rfc3986".
transform-skipped-ineligible-scheme --- general;
a non-HTTP(S) scheme, ineligible for the Stage-B transforms.
ws-fragment-forbidden --- general; a fragment on a
ws:/wss: URL (RFC 6455).
ws-userinfo-forbidden --- general; userinfo on a
ws:/wss: URL (RFC 6455).
mailto-fragment-discouraged --- general; a fragment
on a mailto: URL (RFC 6068).
tel-missing-phone-context --- general; a local
tel: number with no phone-context (RFC 3966).
data-missing-comma --- general; a data: URL
with no "," separator (RFC 2397).
file-non-absolute-path --- general; a
non-absolute file: path under "rfc3986".
file-userinfo-extension --- general; userinfo on a
file: URL, permitted by RFC 8089 Appendix E.1's non-normative
extended grammar.
file-component-outside-rfc8089 --- general; a port,
query or fragment on a file: URL, inherited from generic
RFC 3986.
A single URL can carry several diagnostics, so the return shape is not a
plain scalar-per-URL vector (see Value). parse_status stays
coarse; diagnostics are never encoded into it.
get_host_type, safe_parse_url
get_url_diagnostics("http://example.com/", url_standard = "rfc3986")
get_url_diagnostics(
c("http://example.com/", "http://2130706433/"),
url_standard = "whatwg"
)
Run the code above in your browser using DataLab