kindling (version 0.2.0)

dials-kindling: Tunable hyperparameters for kindling models

Description

These parameters extend the dials framework to support hyperparameter tuning of neural networks built with the {kindling} package. They control network architecture, activation functions, optimization, and training behavior.

Usage

n_hlayers(range = c(1L, 2L), trans = NULL)
hidden_neurons(range = c(8L, 512L), disc_values = NULL, trans = NULL)
activations(
  values = c("relu", "relu6", "elu", "selu", "celu", "leaky_relu", "gelu", "softplus",
    "softshrink", "softsign", "tanhshrink", "hardtanh", "hardshrink", "hardswish",
    "hardsigmoid", "silu", "mish", "logsigmoid")
)
output_activation(
  values = c("relu", "elu", "selu", "softplus", "softmax", "log_softmax", "logsigmoid",
    "hardtanh", "hardsigmoid", "silu")
)
optimizer(values = c("adam", "sgd", "rmsprop", "adamw"))
bias(values = c(TRUE, FALSE))
validation_split(range = c(0, 0.5), trans = NULL)
bidirectional(values = c(TRUE, FALSE))

Value

Each function returns a dials parameter object:

n_hlayers()

A quantitative parameter for the number of hidden layers

hidden_neurons()

A quantitative parameter for hidden units per layer

activations()

A qualitative parameter for activation function names

output_activation()

A qualitative parameter for output activation

optimizer()

A qualitative parameter for optimizer type

bias()

A qualitative parameter for bias inclusion

validation_split()

A quantitative parameter for validation proportion

bidirectional()

A qualitative parameter for bidirectional RNN

Arguments

range

A two-element numeric vector with the default lower and upper bounds.

trans

An optional transformation; NULL for none.

disc_values

NULL (default) or an integer vector of specific possible disc_values (e.g., c(32L, 64L, 128L, 256L)). When provided, tuning will be restricted to these discrete values. The range is automatically derived from these values if not explicitly given. The trans parameter would still be ignored by this parameter when supplied.

values

Logical vector of possible values.

Architecture Strategy

Since tidymodels tuning works with independent parameters, we use a simplified approach where:

• hidden_neurons specifies a single value that will be used for ALL layers

• activations specifies a single activation that will be used for ALL layers

• n_hlayers controls the depth

For more complex architectures with different neurons/activations per layer, users should manually specify these after tuning or use custom tuning logic.

Parameters

n_hlayers

Number of hidden layers in the network.

hidden_neurons

Number of units per hidden layer (applied to all layers).

activation

Single activation function applied to all hidden layers.

output_activation

Activation function for the output layer.

optimizer

Optimizer algorithm.

bias

Whether to include bias terms in layers.

validation_split

Proportion of training data held out for validation.

bidirectional

Whether RNN layers are bidirectional.

Number of Hidden Layers

Controls the depth of the network. When tuning, this will determine how many layers are created, each with hidden_neurons units and activations function.

Hidden Units per Layer

Specifies the number of units per hidden layer.

Activation Function (Hidden Layers)

Activation functions for hidden layers.

Output Activation Function

Activation function applied to the output layer. Values must correspond to torch::nnf_* functions.

Optimizer Type

The optimization algorithm used during training.

Include Bias Terms

Whether layers should include bias parameters.

Validation Split Proportion

Fraction of the training data to use as a validation set during training.

Bidirectional RNN

Whether recurrent layers should process sequences in both directions.

Examples

# \donttest{
library(dials)
library(tune)

# Create a tuning grid
grid = grid_regular(
    n_hlayers(range = c(1L, 4L)),
    hidden_neurons(range = c(32L, 256L)),
    activations(c('relu', 'elu', 'selu')),
    levels = c(4, 5, 3)
)

# Use in a model spec
mlp_spec = mlp_kindling(
    mode = "classification",
    hidden_neurons = tune(),
    activations = tune(),
    epochs = tune(),
    learn_rate = tune()
)
# }