optimal_threshold

Optimize threshold for clean data extraction.

A framework used to detect and handle outliers during data analysis workflows. Outlier detection is a statistical concept with applications in data analysis workflows, highlighting records that are suspiciously high or low. Outlier detection in distribution models was initiated by Chapman (1991) (available at <https://www.researchgate.net/publication/332537800_Quality_control_and_validation_of_point-sourced_environmental_resource_data>), who developed the reverse jackknifing method. The concept was further developed and incorporated into different R packages, including 'flexsdm' (Velazco et al., 2022, <doi:10.1111/2041-210X.13874>) and 'biogeo' (Robertson et al., 2016 <doi:10.1111/ecog.02118>). We compiled various outlier detection methods obtained from the literature, including those elaborated in Dastjerdy et al. (2023) <doi:10.3390/geotechnics3020022> and Liu et al. (2008) <doi:10.1109/ICDM.2008.17>. In this package, we introduced the ensembling aspect, where multiple outlier detection methods are used to flag the record as either an absolute outlier. The concept can also be applied in general data analysis, as well as during the development of species distribution models.

Anthony Basooma

specleanr

Detecting Environmental Outliers in Data Analysis Pipelines

Thomas Hein

Astrid Schmidt-Kloiber

Merret Buurman

Sami Domisch

Martin Tschikof

Florian Borgwardt

optimal_threshold function

<dl><dt>refdata</dt>
<dd><code>dataframe</code>. Species data frame from precleaned analysis.</dd>
<dt>outliers</dt>
<dd><code>datacleaner</code>. Datacleaner output with outliers flagged in <code>multidetect</code> function.</dd>
<dt>var_col</dt>
<dd><code>string</code>. A column with species names if <code>dataset</code> for species is a dataframe not a list.
See <code>pred_extract</code> for extracting environmental data.</dd>
<dt>warn</dt>
<dd><code>logical</code>. If TRUE, warning on whether absolute
outliers obtained at a low threshold is indicated. Default TRUE.</dd>
<dt>verbose</dt>
<dd><code>logical</code>. If true, then messages about the outlier flagging will be displayed.</dd>
<dt>plotsetting</dt>
<dd><code>list</code>. to show plot of loess fitted function with local and global maxima (optimal threshold and clean data).
The list had two parameters. 1) plot to indicate the plot and group to provide the plot title.</dd>
<dt>cutoff</dt>
<dd><code>numeric</code>. Ranging from 0.5 to 0.8 indicating the cutoff to initiate the
LOESS model to optimize the identification of absolute outliers.</dd></dl>

Arguments

Optimize threshold for clean data extraction. — optimal_threshold

<dl>

<dt>refdata</dt>
<dd><code>dataframe</code>. Species data frame from precleaned analysis.</dd>


<dt>outliers</dt>
<dd><code>datacleaner</code>. Datacleaner output with outliers flagged in <code>multidetect</code> function.</dd>


<dt>var_col</dt>
<dd><code>string</code>. A column with species names if <code>dataset</code> for species is a dataframe not a list.
See <code>pred_extract</code> for extracting environmental data.</dd>


<dt>warn</dt>
<dd><code>logical</code>. If TRUE, warning on whether absolute
outliers obtained at a low threshold is indicated. Default TRUE.</dd>


<dt>verbose</dt>
<dd><code>logical</code>. If true, then messages about the outlier flagging will be displayed.</dd>


<dt>plotsetting</dt>
<dd><code>list</code>. to show plot of loess fitted function with local and global maxima (optimal threshold and clean data).
The list had two parameters. 1) plot to indicate the plot and group to provide the plot title.</dd>


<dt>cutoff</dt>
<dd><code>numeric</code>. Ranging from 0.5 to 0.8 indicating the cutoff to initiate the
LOESS model to optimize the identification of absolute outliers.</dd>

</dl>

optimal_threshold: Optimize threshold for clean data extraction.

Description

Usage

Value

Arguments