SparseEmbedder

<p>Generates sparse BM25 embeddings for keyword search</p>

A lightweight vector database for text retrieval in R with embedded
machine learning models and no external API (Application Programming
Interface) keys. Supports dense and hybrid search, optional HNSW
(Hierarchical Navigable Small World) approximate nearest-neighbor indexing,
faceted filters with ACL (Access Control List) metadata, command-line
tools, and a local dashboard built with 'shiny'. The HNSW method is
described by Malkov and Yashunin (2018) <doi:10.1109/TPAMI.2018.2889473>.

Kwadwo Daddy Nyame Owusu Boakye

VectrixDB

Lightweight Vector Database with Embedded Machine Learning
Models

SparseEmbedder function

<p><div class="r6-fields"></div></p><dl>
<dt><code>vocab</code></dt>
<dd><p>Vocabulary</p></dd><p></p><p>
<dt><code>language</code></dt>
<dd><p>Language setting ("en" or "ml")</p></dd></p><p>
</p></dl><p></p>

Public fields


<div class="section" id="public-methods">
<h3>Public methods</h3><p></p><p><ul>
<li><p><a href="/link/SparseEmbedder%24new()?package=VectrixDB&version=1.1.2" data-mini-rdoc="VectrixDB::SparseEmbedder$new()"><code>SparseEmbedder$new()</code></a></p></li>
<li><p><a href="/link/SparseEmbedder%24fit()?package=VectrixDB&version=1.1.2" data-mini-rdoc="VectrixDB::SparseEmbedder$fit()"><code>SparseEmbedder$fit()</code></a></p></li>
<li><p><a href="/link/SparseEmbedder%24embed()?package=VectrixDB&version=1.1.2" data-mini-rdoc="VectrixDB::SparseEmbedder$embed()"><code>SparseEmbedder$embed()</code></a></p></li>
<li><p><a href="/link/SparseEmbedder%24query_terms()?package=VectrixDB&version=1.1.2" data-mini-rdoc="VectrixDB::SparseEmbedder$query_terms()"><code>SparseEmbedder$query_terms()</code></a></p></li>
<li><p><a href="/link/SparseEmbedder%24clone()?package=VectrixDB&version=1.1.2" data-mini-rdoc="VectrixDB::SparseEmbedder$clone()"><code>SparseEmbedder$clone()</code></a></p></li>
</ul></p><p></p></div><p><hr>
<a id="method-SparseEmbedder-new"></a></p><div class="section" id="method-new-">
<h3>Method <code>new()</code></h3>
<p>Create a new SparseEmbedder</p><div class="section" id="usage">
<h4>Usage</h4>
<p><div class="r"></div></p><pre><code>SparseEmbedder$new(language = "en")</code></pre><p></p></div><p></p>
</div><p></p><p><div class="section" id="arguments">
<h4>Arguments</h4>
<p><div class="arguments"></div></p><dl>
<dt><code>language</code></dt>
<dd><p>Language behavior ("en" = ASCII-focused, "ml" = Unicode-aware)</p></dd></dl></div></p><p>
</p><p></p>
<p></p><p></p><p><hr>
<a id="method-SparseEmbedder-fit"></a></p><div class="section" id="method-fit-">
<h3>Method <code>fit()</code></h3>
<p>Fit the embedder on a corpus</p><div class="section" id="usage">
<h4>Usage</h4>
<p><div class="r"></div></p><pre><code>SparseEmbedder$fit(texts)</code></pre><p></p></div><p></p>
</div><p></p><p><div class="section" id="arguments">
<h4>Arguments</h4>
<p><div class="arguments"></div></p><dl>
<dt><code>texts</code></dt>
<dd><p>Character vector of texts</p></dd></dl></div></p><p>
</p><p></p>
<p></p><p></p><p><hr>
<a id="method-SparseEmbedder-embed"></a></p><div class="section" id="method-embed-">
<h3>Method <code><a href="/link/embed()?package=VectrixDB&version=1.1.2" data-mini-rdoc="VectrixDB::embed()">embed()</a></code></h3>
<p>Embed texts to sparse vectors</p><div class="section" id="usage">
<h4>Usage</h4>
<p><div class="r"></div></p><pre><code>SparseEmbedder$embed(texts)</code></pre><p></p></div><p></p>
</div><p></p><p><div class="section" id="arguments">
<h4>Arguments</h4>
<p><div class="arguments"></div></p><dl>
<dt><code>texts</code></dt>
<dd><p>Character vector of texts</p></dd></dl></div></p><p>
</p><p></p>

<div class="section" id="returns">
<h4>Returns</h4>
<p>Sparse matrix of BM25 scores</p>
</div><p></p><p></p><p><hr>
<a id="method-SparseEmbedder-query_terms"></a></p><div class="section" id="method-query-terms-">
<h3>Method <code>query_terms()</code></h3>
<p>Get term scores for a query</p><div class="section" id="usage">
<h4>Usage</h4>
<p><div class="r"></div></p><pre><code>SparseEmbedder$query_terms(query)</code></pre><p></p></div><p></p>
</div><p></p><p><div class="section" id="arguments">
<h4>Arguments</h4>
<p><div class="arguments"></div></p><dl>
<dt><code>query</code></dt>
<dd><p>Query text</p></dd></dl></div></p><p>
</p><p></p>

<div class="section" id="returns">
<h4>Returns</h4>
<p>Named vector of term scores</p>
</div><p></p><p></p><p><hr>
<a id="method-SparseEmbedder-clone"></a></p><div class="section" id="method-clone-">
<h3>Method <code>clone()</code></h3>
<p>The objects of this class are cloneable with this method.</p><div class="section" id="usage">
<h4>Usage</h4>
<p><div class="r"></div></p><pre><code>SparseEmbedder$clone(deep = FALSE)</code></pre><p></p></div><p></p>
</div><p></p><p><div class="section" id="arguments">
<h4>Arguments</h4>
<p><div class="arguments"></div></p><dl>
<dt><code>deep</code></dt>
<dd><p>Whether to make a deep clone.</p></dd></dl></div></p><p>
</p><p></p>
<p></p><p>
</p>

Methods

Sparse Embedder (BM25/TF-IDF) — SparseEmbedder

<p><div class="r6-fields"></p><dl>
<dt><code>vocab</code></dt>
<dd><p>Vocabulary</p></dd>


<dt><code>language</code></dt>
<dd><p>Language setting ("en" or "ml")</p></dd>


</dl><p></div></p>


<div class='section' id='public-methods'>
<h3>Public methods</h3>

<ul>
<li><p><a href='#method-SparseEmbedder-new'><code>SparseEmbedder$new()</code></a></p></li>
<li><p><a href='#method-SparseEmbedder-fit'><code>SparseEmbedder$fit()</code></a></p></li>
<li><p><a href='#method-SparseEmbedder-embed'><code>SparseEmbedder$embed()</code></a></p></li>
<li><p><a href='#method-SparseEmbedder-query_terms'><code>SparseEmbedder$query_terms()</code></a></p></li>
<li><p><a href='#method-SparseEmbedder-clone'><code>SparseEmbedder$clone()</code></a></p></li>
</ul>

</div><p><hr>
<a id="method-SparseEmbedder-new"></a></p><div class='section' id='method-new-'>
<h3>Method <code>new()</code></h3>
<p>Create a new SparseEmbedder</p><div class='section' id='usage'>
<h4>Usage</h4>
<p><div class="r"></p><pre><code>SparseEmbedder$new(language = "en")</code></pre><p></div></p>
</div>

<div class='section' id='arguments'>
<h4>Arguments</h4>
<p><div class="arguments"></p><dl>
<dt><code>language</code></dt>
<dd><p>Language behavior ("en" = ASCII-focused, "ml" = Unicode-aware)</p></dd>


</dl><p></div></p>
</div>

</div><p><hr>
<a id="method-SparseEmbedder-fit"></a></p><div class='section' id='method-fit-'>
<h3>Method <code>fit()</code></h3>
<p>Fit the embedder on a corpus</p><div class='section' id='usage'>
<h4>Usage</h4>
<p><div class="r"></p><pre><code>SparseEmbedder$fit(texts)</code></pre><p></div></p>
</div>

<div class='section' id='arguments'>
<h4>Arguments</h4>
<p><div class="arguments"></p><dl>
<dt><code>texts</code></dt>
<dd><p>Character vector of texts</p></dd>


</dl><p></div></p>
</div>

</div><p><hr>
<a id="method-SparseEmbedder-embed"></a></p><div class='section' id='method-embed-'>
<h3>Method <code><a href='https://rdrr.io/r/stats/embed.html'>embed()</a></code></h3>
<p>Embed texts to sparse vectors</p><div class='section' id='usage'>
<h4>Usage</h4>
<p><div class="r"></p><pre><code>SparseEmbedder$embed(texts)</code></pre><p></div></p>
</div>

<div class='section' id='arguments'>
<h4>Arguments</h4>
<p><div class="arguments"></p><dl>
<dt><code>texts</code></dt>
<dd><p>Character vector of texts</p></dd>


</dl><p></div></p>
</div>
<div class='section' id='returns'>
<h4>Returns</h4>
<p>Sparse matrix of BM25 scores</p>
</div>

</div><p><hr>
<a id="method-SparseEmbedder-query_terms"></a></p><div class='section' id='method-query-terms-'>
<h3>Method <code>query_terms()</code></h3>
<p>Get term scores for a query</p><div class='section' id='usage'>
<h4>Usage</h4>
<p><div class="r"></p><pre><code>SparseEmbedder$query_terms(query)</code></pre><p></div></p>
</div>

<div class='section' id='arguments'>
<h4>Arguments</h4>
<p><div class="arguments"></p><dl>
<dt><code>query</code></dt>
<dd><p>Query text</p></dd>


</dl><p></div></p>
</div>
<div class='section' id='returns'>
<h4>Returns</h4>
<p>Named vector of term scores</p>
</div>

</div><p><hr>
<a id="method-SparseEmbedder-clone"></a></p><div class='section' id='method-clone-'>
<h3>Method <code>clone()</code></h3>
<p>The objects of this class are cloneable with this method.</p><div class='section' id='usage'>
<h4>Usage</h4>
<p><div class="r"></p><pre><code>SparseEmbedder$clone(deep = FALSE)</code></pre><p></div></p>
</div>

<div class='section' id='arguments'>
<h4>Arguments</h4>
<p><div class="arguments"></p><dl>
<dt><code>deep</code></dt>
<dd><p>Whether to make a deep clone.</p></dd>


</dl><p></div></p>
</div>

</div>


SparseEmbedder: Sparse Embedder (BM25/TF-IDF)

Description

Arguments

Public fields

Methods

Public methods

Method `new()`

Usage

Arguments

Method `fit()`

Usage

Arguments

Method `embed()`

Usage

Arguments

Returns

Method `query_terms()`

Usage

Arguments

Returns

Method `clone()`

Usage

Arguments

Description

Arguments

Public fields

Methods

Public methods

Method new()

Usage

Arguments

Method fit()

Usage

Arguments

Method embed()

Usage

Arguments

Returns

Method query_terms()

Usage

Arguments

Returns

Method clone()

Usage

Arguments

Method `new()`

Method `fit()`

Method `embed()`

Method `query_terms()`

Method `clone()`