default_tokenize

This tokenizer uses <code><a href="/link/stringi%3A%3Astri_split_boundaries()?package=lime&version=0.5.3" data-mini-rdoc="lime::stringi::stri_split_boundaries()">stringi::stri_split_boundaries()</a></code> to tokenize a
<code>character</code> vector. To be used with [explain.character()`.

When building complex models, it is often difficult to explain why
the model should be trusted. While global measures such as accuracy are
useful, they cannot be used for explaining why a model made a specific
prediction. 'lime' (a port of the 'lime' 'Python' package) is a method for
explaining the outcome of black box models by fitting a local model around
the point in question an perturbations of this point. The approach is
described in more detail in the article by Ribeiro et al. (2016)
<arXiv:1602.04938>.

Emil Hvitfeldt

lime

Local Interpretable Model-Agnostic Explanations

Thomas Lin Pedersen

Michaël Benesty

default_tokenize function

<dl><dt>text</dt>
<dd>text to tokenize as a <code>character</code> vector</dd></dl>

Arguments

This tokenizer uses <code><a href='https://rdrr.io/pkg/stringi/man/stri_split_boundaries.html'>stringi::stri_split_boundaries()</a></code> to tokenize a
<code>character</code> vector. To be used with [explain.character()`.

Default function to tokenize — default_tokenize

<dl>

<dt>text</dt>
<dd>text to tokenize as a <code>character</code> vector</dd>

</dl>

default_tokenize: Default function to tokenize

Description

Usage

Value

Arguments

Examples