rdomains (version 0.5.0)

page_signals: What kind of page is this?

Description

Inspects already-fetched HTML for the three things that are not a classifiable site: an anti-bot interstitial, a domain-parking placeholder, and a server's "nothing here" page. No network access.

Usage

page_signals(html, text = NULL, domain = "", status = NULL)

Value

A one-row tibble: `page_state` (one of `"content"`, `"blocked"`, `"parked"`, `"unavailable"`, `"thin"`), `blocked`, `block_vendor`, `block_reason`, `parked`, `unavailable`, `thin`, `n_tokens`.

Arguments

html

Raw response body.

text

Extracted page text. If `NULL`, derived from `html`.

domain

The domain requested, used to spot a page whose only content is its own name.

status

HTTP status code, if known.

Details

`parked` and `unavailable` are **answers, not failures** -- they are facts about the domain, plainly stated in the page, that a caller can act on. They are also free: no classification service needs to be consulted.

See Also

[fetch_error_codes()] for how these map onto run outcomes.

Examples

# A Cloudflare challenge, not a website
page_signals("Just a moment...cf_chl_opt")

# A parking page
page_signals("This domain is for sale. Inquire now.")

# A real page that merely embeds reCAPTCHA is not blocked
page_signals(paste0("Reddit",
                    paste(rep("real discussion content", 200), collapse = " "),
                    ""))