R6 class for managing local Small Language Model inference. Provides a unified interface for loading model weights, running inference, and managing model lifecycle.
model_pathPath to the model weights file.
model_nameHuman-readable model name.
backendThe inference backend ("onnx", "torch", "gguf").
configModel configuration parameters.
loadedWhether the model is currently loaded in memory.
new()Create a new SLM Engine instance.
SlmEngine$new(model_path, backend = "gguf", config = list())model_pathPath to the model weights file (GGUF, ONNX, or PT format).
backendInference backend to use: "gguf" (default), "onnx", or "torch".
configOptional list of configuration parameters.
A new SlmEngine object.
Self (invisibly).
unload()Unload the model from memory.
SlmEngine$unload()Self (invisibly).
generate()Generate text completion from a prompt.
SlmEngine$generate(
prompt,
max_tokens = 256,
temperature = 0.7,
top_p = 0.9,
stop = NULL
)promptThe input prompt text.
max_tokensMaximum number of tokens to generate.
temperatureSampling temperature (0.0 to 2.0).
top_pNucleus sampling parameter.
stopOptional stop sequences.
A list with generated text and metadata.
stream()Stream text generation with a callback function.
SlmEngine$stream(
prompt,
callback,
max_tokens = 256,
temperature = 0.7,
top_p = 0.9,
stop = NULL
)promptThe input prompt text.
callbackFunction called with each generated token.
max_tokensMaximum number of tokens to generate.
temperatureSampling temperature.
top_pNucleus sampling parameter.
stopOptional stop sequences.
A list with the complete generated text and metadata.
info()Get model information and statistics.
SlmEngine$info()A list with model metadata.
clone()The objects of this class are cloneable with this method.
SlmEngine$clone(deep = FALSE)deepWhether to make a deep clone.