collocationAnalysis,KorAPConnection-method: Collocation analysis

Description

Performs a collocation analysis for the given node (or query) in the given virtual corpus.

Usage

# S4 method for KorAPConnection
collocationAnalysis(
  kco,
  node,
  vc = "",
  lemmatizeNodeQuery = FALSE,
  minOccur = 5,
  leftContextSize = 5,
  rightContextSize = 5,
  topCollocatesLimit = 200,
  searchHitsSampleLimit = 20000,
  ignoreCollocateCase = FALSE,
  withinSpan = ifelse(exactFrequencies, "base/s=s", ""),
  exactFrequencies = TRUE,
  stopwords = append(RKorAPClient::synsemanticStopwords(), node),
  seed = 7,
  expand = length(vc) != length(node),
  maxRecurse = 0,
  addExamples = FALSE,
  thresholdScore = "logDice",
  threshold = 2,
  localStopwords = c(),
  collocateFilterRegex = "^[:alnum:]+-?[:alnum:]*$",
  minObservedExpectedRatio = 1,
  queryMissingScores = FALSE,
  missingScoreQuantile = 0.05,
  vcLabel = NA_character_,
  cacheAs = NULL,
  ...
)

Value

A tibble where each row represents a candidate collocate for the requested node. Columns include (depending on the selected association measures):

node, collocate, vc, label: identifiers for the query node, collocate, virtual corpus, and optional label.
Frequency and contingency information such as frequency, O, O1, O2, E, leftContextSize, rightContextSize, and w.
Association measures (e.g. logDice, ll, mi, ...), one column per requested scorer.
Per-labelled association scores produced by multi-VC comparisons using the pattern <measure>_<label>.
Ranks per label/measure with the pattern rank_<label>_<measure> (1 is best) and the corresponding percentile ranks percentile_rank_<label>_<measure>.
Pairwise contrasts for two-label comparisons, e.g. delta_<measure>, delta_rank_<measure>, and delta_percentile_rank_<measure>.
Summary columns describing the strongest labels per measure (winner_*, runner_up_*, loser_*, and max_delta_*), including winner/loser webUIRequestUrl columns. In multi-VC comparisons, missing per-label concordance URLs are derived from another available row URL for the same node/collocate by replacing the cq parameter with the target label's virtual corpus. Unsuffixed winner_webUIRequestUrl and loser_webUIRequestUrl columns are populated only when the score-based URL choices agree.
imputed_<label>: whether the score for that label was imputed rather than observed (see missingScoreQuantile). n_imputed counts them, and imputed is n_imputed > 0: it describes the node/collocate pair across all labels, not the label of the row it stands in. A row can therefore carry imputed = TRUE while its own scores are perfectly attested, because the pair is missing from some other virtual corpus - use imputed_<label> for the row itself. Filter with dplyr::filter(!imputed) to keep only collocates attested in every compared virtual corpus.
Optional helper columns such as query, example, or url when example retrieval is requested.

Arguments

kco

KorAPConnection() object (obtained e.g. from KorAPConnection()

node

target word or query as a single character string

vc

character vector describing the virtual corpus or corpora in which the query should be performed. An empty string (default) means the whole corpus, as far as it is license-wise accessible.

lemmatizeNodeQuery

if TRUE, node query will be lemmatized, i.e. x -> [tt/l=x]

minOccur

minimum absolute number of observed co-occurrences to consider a collocate candidate

leftContextSize

size of the left context window

rightContextSize

size of the right context window

topCollocatesLimit

limit analysis to the n most frequent collocates in the search hits sample

searchHitsSampleLimit

limit the size of the search hits sample

ignoreCollocateCase

logical, set to TRUE if collocate case should be ignored

withinSpan

KorAP span specification (see https://korap.ids-mannheim.de/doc/ql/poliqarp-plus?embedded=true#spans) for collocations to be searched within. Defaults to base/s=s.

exactFrequencies

if FALSE, extrapolate observed co-occurrence frequencies from frequencies in search hits sample, otherwise retrieve exact co-occurrence frequencies

stopwords

vector of stopwords not to be considered as collocates

seed

seed for random page collecting order

expand

if TRUE, node and vc parameters are expanded to all of their combinations

maxRecurse

apply collocation analysis recursively maxRecurse times

addExamples

If TRUE, examples for instances of collocations will be added in a column example. This makes a difference in particular if node is given as a lemma query.

thresholdScore

association score function (see association-score-functions) to use for computing the threshold that is applied for recursive collocation analysis calls (only applied when maxRecurse > 0)

threshold

minimum value of thresholdScore function call to apply collocation analysis recursively (only applied when maxRecurse > 0). Note that the default score, logDice, expresses how salient a pair is rather than how surprising, so that a frequent collocate can pass it while co-occurring less often than expected. minObservedExpectedRatio keeps those out. See the "Salience versus surprise" section of association-score-functions.

localStopwords

vector of stopwords that will not be considered as collocates in the current function call, but that will not be passed to recursive calls

collocateFilterRegex

allow only collocates matching the regular expression

minObservedExpectedRatio

minimum ratio of observed to expected co-occurrence frequency a collocate must reach. Defaults to 1, which keeps only collocates that occur at least as often as expected by chance, corresponding to a non-negative pmi. Without it, frequent words can end up among the top collocates by logDice although the node does not attract them at all (see the "Salience versus surprise" section of association-score-functions). Raise it to demand a stronger contrast, e.g. 2 for collocates occurring at least twice as often as expected, or set it to 0 to switch the filter off and obtain the unfiltered result of earlier versions, e.g. in order to study repulsion.

queryMissingScores

if TRUE, attempt to retrieve corpus-based association scores for vc/collocate combinations that would otherwise be imputed, by re-querying the KorAP backend without applying the collocate frequency threshold

missingScoreQuantile

lower quantile (evaluated per association measure over the pooled result set) that anchors the adaptive floor used for imputing missing scores between virtual corpora; a robust spread is subtracted from this anchor so the imputed values stay at or below the weakest observed scores. Imputed cells are marked in the imputed* columns; see the section on interpreting multi-VC comparisons below

vcLabel

optional label override for the current virtual corpus (used internally when named VC collections are expanded)

cacheAs

path to an RDS file to keep the result in. If the file exists and records the same call, it is read back instead of contacting the server; otherwise the query is run and its result stored there. Unlike the connection's cache, this file belongs to the caller, which is what keeps an analysis reproducible once the corpus has grown or the scores have changed. Defaults to NULL (no file).

The analysis parameters are stored alongside the result. If they differ from those of the current call, the cached result would not be the one that was asked for, so it is recomputed and the file overwritten, with a warning naming the parameters that differ. Pass a different cacheAs file name to keep an existing analysis. Cache files written by RKorAPClient 1.3.0 do not contain the parameters yet and are used as they are.

...

more arguments will be passed to collocationScoreQuery()

Interpreting multi-VC comparisons

The comparison columns produced when vc holds more than one virtual corpus are experimental: their names and semantics may still change in a future release without a deprecation cycle. Code that has to keep working across versions should select the columns it needs explicitly.

They are an exploration aid, not a significance test. When reading them, keep three properties in mind.

Imputed scores describe presence/absence, not contrast. A collocate that passes the minOccur and topCollocatesLimit thresholds in one virtual corpus but not in another has no observed score for the latter. Such cells are imputed from a floor derived from the pooled result set (see missingScoreQuantile), so the corresponding delta_* and max_delta_* values measure the distance to that floor rather than an attested difference. The imputed, n_imputed and imputed_<label> columns mark these rows; dplyr::filter(!imputed) restricts the result to collocates attested everywhere, and queryMissingScores = TRUE replaces most imputed cells with scores actually retrieved from the backend. Mind what imputed is about: the pair, not the row. It is TRUE as soon as one label lacks the collocate, and stays TRUE on the rows of the labels where it is attested, which is what makes dplyr::filter(!imputed) drop the pair as a whole. Whether the row at hand rests on an imputed score is what imputed_<label> says.

Per-label columns carry syntactic names. The label in <measure>_<label>, rank_<label>_<measure> and imputed_<label> is the one the caller gave, put through make.names(), so that the result stays a well formed data frame: a virtual corpus named 1976-1980 appears as logDice_X1976.1980. The label column and the winner_* / loser_* columns keep the name as it was given, so mapping between the two means applying the same transformation, e.g. stats::setNames(make.names(labels), labels).

Imputed values are relative to one analysis. The floor is computed from the scores present in the result at hand. Analysing a node on its own and analysing it together with other nodes therefore yield different imputed values, and deltas involving imputed cells are not comparable across separate calls. Deltas between observed scores are unaffected.

Winners carry no uncertainty. Unlike ci(), which attaches confidence intervals to relative frequencies, the winner_* / loser_* columns simply order point estimates. A collocate wins by a hair on six occurrences exactly as decisively as one that wins by a wide margin on thousands. Consult the observed frequencies (O, O1, O2) and the webUIRequestUrl concordance links before drawing conclusions from a small difference.

Note also that rank_<label>_<measure> and percentile_rank_<label>_<measure> are computed within each label, over that label's own candidate set. Candidate sets usually differ in size between virtual corpora, so rank-based deltas compare positions in populations of different sizes.

Details

The collocation analysis is currently implemented on the client side, as some of the functionality is not yet provided by the KorAP backend. Mainly for this reason it is very slow (several minutes, up to hours), but on the other hand very flexible. You can, for example, perform the analysis in arbitrary virtual corpora, use complex node queries, and look for expression-internal collocates using the focus function (see examples and demo).

To increase speed at the cost of accuracy and possible false negatives, you can decrease searchHitsSampleLimit and/or topCollocatesLimit and/or set exactFrequencies to FALSE.

Note that some outdated non-DeReKo back-ends might not yet support returning tokenized matches (warning issued). In this case, the client library will fall back to client-side tokenization which might be slightly less accurate. This might lead to false negatives and to frequencies that differ from corresponding ones acquired via the web user interface.

Examples

Run this code

if (FALSE) {

# Find top collocates of "Packung" inside and outside the sports domain.
KorAPConnection(verbose = TRUE) |>
  collocationAnalysis("Packung",
    vc = c("textClass=sport", "textClass!=sport"),
    leftContextSize = 1, rightContextSize = 1, topCollocatesLimit = 20
  ) |>
  dplyr::filter(logDice >= 5)
}

if (FALSE) {

# Identify the most prominent light verb construction with "in ... setzen".
# Note that, currently, the use of focus function disallows exactFrequencies.
KorAPConnection(verbose = TRUE) |>
  collocationAnalysis("focus(in [tt/p=NN] {[tt/l=setzen]})",
    leftContextSize = 1, rightContextSize = 0, exactFrequencies = FALSE, topCollocatesLimit = 20
  )
}

Run the code above in your browser using DataLab