whisper_encoder

Full Whisper encoder: Conv stem + positional encoding + transformer layers.

Speech-to-text transcription using a native R 'torch' implementation
of 'OpenAI' 'Whisper' model <https://github.com/openai/whisper>. Supports
multiple model sizes from tiny (39M parameters) to large-v3 (1.5B parameters)
with integrated download from 'HuggingFace' <https://huggingface.co/> via the
'hfhub' package. Provides automatic speech recognition with optional language
detection and translation to English. Audio preprocessing, mel spectrogram
computation, and transformer-based encoder-decoder inference are all
implemented in R using the 'torch' package.

Troy Hernandez

whisper

Native R 'torch' Implementation of 'OpenAI' 'Whisper'

cornball.ai 

OpenAI 

whisper_encoder function

<dl><dt>n_mels</dt>
<dd>Number of mel spectrogram bins</dd>
<dt>n_ctx</dt>
<dd>Maximum context length (1500 for 30s audio)</dd>
<dt>n_state</dt>
<dd>Hidden dimension</dd>
<dt>n_head</dt>
<dd>Number of attention heads</dd>
<dt>n_layer</dt>
<dd>Number of transformer layers</dd></dl>

Arguments

Audio Encoder — whisper_encoder

<dl>

<dt>n_mels</dt>
<dd>Number of mel spectrogram bins</dd>


<dt>n_ctx</dt>
<dd>Maximum context length (1500 for 30s audio)</dd>


<dt>n_state</dt>
<dd>Hidden dimension</dd>


<dt>n_head</dt>
<dd>Number of attention heads</dd>


<dt>n_layer</dt>
<dd>Number of transformer layers</dd>

</dl>

whisper_encoder: Audio Encoder

Description

Usage

Arguments