ggmlR (version 0.6.1)

ggml_top_k: Top-K Indices (Graph)

Description

Returns the indices of top K elements per row. Useful for sampling strategies in language models (top-k sampling). Note: the resulting indices are in no particular order within top-k.

Usage

ggml_top_k(ctx, a, k)

Value

Tensor containing I32 indices of top-k elements (not values)

Arguments

ctx

GGML context

a

Input tensor (F32)

k

Number of top elements to return per row

Examples

# \donttest{
ctx <- ggml_init(16 * 1024 * 1024)
# Logits from model output
logits <- ggml_new_tensor_1d(ctx, GGML_TYPE_F32, 100)
ggml_set_f32(logits, rnorm(100))
# Get top 5 logits for sampling
top5 <- ggml_top_k(ctx, logits, 5)
graph <- ggml_build_forward_expand(ctx, top5)
ggml_graph_compute(ctx, graph)
ggml_free(ctx)
# }