ggmlR (version 0.6.1)

ggml_mul_mat_id: Matrix Multiplication with Expert Selection (Graph)

Description

Indirect matrix multiplication for Mixture of Experts architectures. Selects expert weights based on indices and performs batched matmul.

Usage

ggml_mul_mat_id(ctx, as, b, ids)

Value

Output tensor after expert-selected matrix multiplication

Arguments

ctx

GGML context

as

Stacked expert weight matrices [n_embd, n_ff, n_experts]

b

Input tensor

ids

Expert selection indices tensor (I32)

Examples

# \donttest{
ctx <- ggml_init(64 * 1024 * 1024)
# 4 experts, each with 8x16 weights (small for example)
experts <- ggml_new_tensor_3d(ctx, GGML_TYPE_F32, 8, 16, 4)
ggml_set_f32(experts, rnorm(8 * 16 * 4))
input <- ggml_new_tensor_2d(ctx, GGML_TYPE_F32, 8, 2)
ggml_set_f32(input, rnorm(16))
# Select expert 0 for token 0, expert 2 for token 1
ids <- ggml_new_tensor_1d(ctx, GGML_TYPE_I32, 2)
ggml_set_i32(ids, c(0L, 2L))
output <- ggml_mul_mat_id(ctx, experts, input, ids)
graph <- ggml_build_forward_expand(ctx, output)
ggml_graph_compute(ctx, graph)
ggml_free(ctx)
# }