ggmlR (version 0.6.1)

ggml_layer_dropout: Add Dropout Layer

Description

Applies dropout regularization. During training, multiplies all activations by (1 - rate) (deterministic expected-value scaling). During inference (training = FALSE), the layer is an identity (no change).

Usage

ggml_layer_dropout(
  model,
  rate,
  stochastic = FALSE,
  name = NULL,
  trainable = FALSE
)

Value

The model with the dropout layer appended, or a new tensor node.

Arguments

model

A ggml_sequential_model or ggml_tensor_node.

rate

Dropout rate in [0, 1). Fraction of units to "drop".

stochastic

Logical. If TRUE, use inverted dropout with a random Bernoulli mask regenerated each epoch (proper regularization). If FALSE (default), use deterministic scaling by (1 - rate) — cheaper but weaker regularization.

name

Optional layer name.

trainable

Ignored for dropout (no weights); kept for API consistency.

Difference from Keras / inverted dropout

Keras implements inverted dropout: during training it applies a random Bernoulli mask and scales surviving activations up by 1 / (1 - rate), so the expected value of each unit is preserved and no scaling is needed at inference.

This implementation uses deterministic scaling (multiply by (1 - rate) at training, identity at inference) — equivalent in expected value but without stochastic noise. Consequences:

• No random mask → the regularization signal is weaker (no co-adaptation breaking).

• Activations at training are scaled down, not up — the magnitude seen by subsequent layers differs from Keras behaviour.

• Results are fully deterministic and reproducible without setting a seed.

Examples

# \donttest{
model <- ggml_model_sequential() |>
  ggml_layer_dense(128, activation = "relu", input_shape = 784L) |>
  ggml_layer_dropout(0.5, stochastic = TRUE) |>
  ggml_layer_dense(10, activation = "softmax")
# }