ggml_rope_ext_back

Backward pass for extended RoPE (Rotary Position Embedding).
Used during training to compute gradients through RoPE.

Provides 'R' bindings to the 'GGML' tensor library for machine
learning, designed primarily for 'Vulkan' GPU acceleration with full CPU
fallback. 'Vulkan' support is auto-detected at build time on Linux (when
'libvulkan-dev' and 'glslc' are installed) and on Windows (when 'Vulkan'
'SDK' is installed and 'VULKAN_SDK' environment variable is set); all
operations fall back to CPU transparently when no GPU is available.
Implements tensor operations, neural network layers, quantization, and a
'Keras'-like sequential model API for building and training networks.
Includes 'AdamW' (Adam with Weight decay) and 'SGD' (Stochastic Gradient
Descent) optimizers with 'MSE' (Mean Squared Error) and cross-entropy
losses. Also provides a dynamic 'autograd' engine ('PyTorch'-style) with
data-parallel training via 'dp_train()', broadcast arithmetic, 'f16'
(half-precision) support on 'Vulkan' GPU, and a multi-head attention layer
for building Transformer architectures. Serves as backend for 'LLM' (Large
Language Model) inference via 'llamaR' and Stable Diffusion image
generation via 'sdR'. See <https://github.com/ggml-org/ggml> for more
information about the underlying library.

Yuri Baramykov

ggmlR

'GGML' Tensor Operations for Machine Learning

Georgi Gerganov

Jeffrey Quesnelle

Bowen Peng

Mozilla Foundation 

ggml_rope_ext_back function

<dl><dt>ctx</dt>
<dd>GGML context</dd>
<dt>a</dt>
<dd>Gradient tensor from upstream (gradients of ggml_rope_ext result)</dd>
<dt>b</dt>
<dd>Position tensor (same as forward pass)</dd>
<dt>c</dt>
<dd>Optional frequency factors tensor (NULL for default)</dd>
<dt>n_dims</dt>
<dd>Number of dimensions for rotation</dd>
<dt>mode</dt>
<dd>RoPE mode</dd>
<dt>n_ctx_orig</dt>
<dd>Original context length</dd>
<dt>freq_base</dt>
<dd>Base frequency</dd>
<dt>freq_scale</dt>
<dd>Frequency scale factor</dd>
<dt>ext_factor</dt>
<dd>Extension factor (YaRN)</dd>
<dt>attn_factor</dt>
<dd>Attention factor</dd>
<dt>beta_fast</dt>
<dd>YaRN fast beta</dd>
<dt>beta_slow</dt>
<dd>YaRN slow beta</dd></dl>

Arguments

RoPE Extended Backward (Graph) — ggml_rope_ext_back

<dl>

<dt>ctx</dt>
<dd>GGML context</dd>


<dt>a</dt>
<dd>Gradient tensor from upstream (gradients of ggml_rope_ext result)</dd>


<dt>b</dt>
<dd>Position tensor (same as forward pass)</dd>


<dt>c</dt>
<dd>Optional frequency factors tensor (NULL for default)</dd>


<dt>n_dims</dt>
<dd>Number of dimensions for rotation</dd>


<dt>mode</dt>
<dd>RoPE mode</dd>


<dt>n_ctx_orig</dt>
<dd>Original context length</dd>


<dt>freq_base</dt>
<dd>Base frequency</dd>


<dt>freq_scale</dt>
<dd>Frequency scale factor</dd>


<dt>ext_factor</dt>
<dd>Extension factor (YaRN)</dd>


<dt>attn_factor</dt>
<dd>Attention factor</dd>


<dt>beta_fast</dt>
<dd>YaRN fast beta</dd>


<dt>beta_slow</dt>
<dd>YaRN slow beta</dd>

</dl>

ggml_rope_ext_back: RoPE Extended Backward (Graph)

Description

Usage

Value

Arguments