rurl (version 3.0.1)

check_hosts: Report practical host validation verdicts for a set of URLs

Description

Tabular companion to is_valid_host: for each URL, reports the parsed host, its get_host_type classification, a logical column for each requested policy rule, and a reasons list-column naming the host facts observed. Useful for auditing a URL set before choosing what to keep --- see why a host is not a practical web/SEO host, not just that it is not.

Usage

check_hosts(
  url,
  rules = c("url", "dns", "web", "registrable", "seo"),
  url_standard = "whatwg",
  scheme_policy = c("infer", "require"),
  scheme_acceptance = c("web", "general")
)

Value

A data.frame with one row per input URL (input order preserved): url, host (the parsed host, or NA), host_type, one logical column per requested rule (NA when the URL has no host to judge; the "url" rule's column is named url_valid so it does not shadow the input url column), and reasons --- a list-column whose i-th element is a character vector of the host facts observed for that URL (character(0) when none). The reasons evidence is reported for the row as a whole, independent of which rules were requested, and combines the host-shape diagnostics that fired (see get_url_diagnostics) with the policy tokens "ip-literal", "not-registrable", and "underscore-label".

Arguments

url

A character vector of URLs.

rules

A character vector of one or more rules to score, drawn from "url", "dns", "web", "registrable", "seo" (all five by default). See is_valid_host for each rule's meaning.

url_standard

Standard profile governing host interpretation: "whatwg" (default) or "rfc3986".

scheme_policy

Controls whether scheme-less, host-shaped input is accepted (an input-acceptance axis, distinct from protocol_handling, which only controls how the scheme is presented, and from url_standard, which controls interpretation). Defaults to "infer".

• "infer": (Default) Fabricate http:// for scheme-less host-shaped input (e.g. example.com parses as http://example.com), a browser-omnibox-style affordance. This is the historical behavior.

• "require": Reject scheme-less input — a scheme-less host-shaped value becomes parse_status = "error" rather than gaining a fabricated scheme. Use this for a strict, pure-parser posture. Note this governs only bare host input; scheme-relative //host input is governed separately by scheme_relative_handling.

scheme_acceptance

Which scheme tokens may enter parsing (a scheme-acceptance axis, distinct from scheme_policy, which governs scheme-less input, and from url_standard, which governs interpretation). Defaults to "web".

• "web": (Default) Only the curated web-scheme allowlist (http/https/ftp/ftps/file) is admitted; a scheme-bearing input outside it is parse_status = "error". This is the historical, byte-for-byte compatible behavior.

• "general": Admit any syntactically valid scheme token and parse opaque (mailto:x), non-special (foo://host), and RFC-generic URLs. Requires an explicit url_standard ("rfc3986" or "whatwg"), which decides the interpretation; general with url_standard = NULL is an error. Non-special / opaque hosts receive no www-stripping, no domain/TLD derivation, and are never run through the IDNA/punycode helpers. A non-special scheme with no // is an opaque path: it has no authority, so host, user, port and the domain/tld columns are all NA and the entire remainder is the path (query/fragment are still split off). This includes mailto: — the recipient's @ never re-triggers authority parsing. To decompose a mailto: recipient, use the accessors (get_host() / get_domain() / get_user(), ADR 0012 D7) or get_mailto_recipients(); those deliberately return a recipient's parts where this table presents NA, because a recipient domain is extraction metadata, not the URL's authority.

Note that file: is admitted under both values, including the default. A file: URL denotes local-filesystem access, and one with a non-empty host (file://server/share/x) is a UNC path on Windows, so dereferencing it reaches a remote SMB share. rurl parses file: URLs; it never opens them. Restricting schemes before anything dereferences them is the caller's job — see SECURITY.md.

Details

Like is_valid_host this is a policy layer, not parser conformance and not a conformance oracle: it never changes how a URL parses and the absence of a reasons token is not a validity guarantee (see is_valid_host's “A policy layer” section).

See Also

is_valid_host, get_host_type, get_url_diagnostics.

Examples

urls <- c(
  "http://example.com",         # registrable domain
  "http://_dmarc.example.com",  # valid DNS owner name, not a web hostname
  "http://a+b.example",         # valid RFC reg-name, neither web nor dns
  "http://-example.com",        # hyphen hygiene failure
  "http://a..com",              # empty label
  "http://localhost",           # web hostname, but not registrable
  "http://192.168.0.1",         # IP literal
  "http://xn--nxasmq6b.example.com" # IDN (A-label)
)
check_hosts(urls)
# Score a single rule, or a subset:
check_hosts(urls, rules = c("web", "dns"))
# RFC 3986 reg-name semantics instead of whatwg:
check_hosts("http://2130706433", url_standard = "rfc3986")