ggmlR (version 0.6.1)

rope_types: RoPE Mode Constants

Description

RoPE (Rotary Position Embedding) Type Constants

Usage

GGML_ROPE_TYPE_NORM
GGML_ROPE_TYPE_NEOX
GGML_ROPE_TYPE_MROPE
GGML_ROPE_TYPE_VISION

Value

An integer constant representing a RoPE type

Format

Integer constants

An object of class integer of length 1.

An object of class integer of length 1.

An object of class integer of length 1.

Details

Constants for RoPE (Rotary Position Embedding) modes used in transformer models. Different models use different RoPE implementations.

• GGML_ROPE_TYPE_NORM (0): Standard RoPE as in original paper (LLaMA, Mistral)

• GGML_ROPE_TYPE_NEOX (2): GPT-NeoX style RoPE with different interleaving

• GGML_ROPE_TYPE_MROPE (8): Multi-RoPE for multimodal models (Qwen2-VL)

• GGML_ROPE_TYPE_VISION (24): Vision model RoPE variant

Examples

# \donttest{
GGML_ROPE_TYPE_NORM    # 0 - Standard RoPE (LLaMA, Mistral)
GGML_ROPE_TYPE_NEOX    # 2 - GPT-NeoX style
GGML_ROPE_TYPE_MROPE   # 8 - Multi-RoPE (Qwen2-VL)
GGML_ROPE_TYPE_VISION  # 24 - Vision models
# }