Setup hyperparameters for LightGBM training.
setup_LightGBM(
max_nrounds = 1000L,
force_nrounds = NULL,
early_stopping_rounds = 10L,
num_leaves = 8L,
max_depth = -1L,
learning_rate = 0.01,
feature_fraction = 1,
bagging_fraction = 1,
bagging_freq = 1L,
lambda_l1 = 0,
lambda_l2 = 0,
max_cat_threshold = 32L,
min_data_per_group = 32L,
linear_tree = FALSE,
ifw = FALSE,
objective = NULL,
early_stopping_min_delta = 0,
deterministic = FALSE,
objective_seed = 5L,
force_row_wise = FALSE,
histogram_pool_size = -1,
top_k = NULL,
pos_bagging_fraction = 1,
neg_bagging_fraction = 1,
bagging_seed = 3L,
feature_fraction_bynode = 1,
feature_fraction_seed = 2L,
cegb_tradeoff = 1,
cegb_penalty_split = 0,
cegb_penalty_feature_lazy = NULL,
cegb_penalty_feature_coupled = NULL,
use_quantized_grad = FALSE,
num_grad_quant_bins = NULL,
quant_train_renew_leaf = NULL,
stochastic_rounding = NULL,
monotone_constraints = NULL,
monotone_constraints_method = NULL,
monotone_penalty = NULL,
interaction_constraints = NULL,
feature_contri = NULL,
min_data_in_leaf = 20L,
min_sum_hessian_in_leaf = 0.001,
min_gain_to_split = 0,
max_delta_step = 0,
path_smooth = 0,
extra_trees = FALSE,
extra_seed = 6L,
linear_lambda = NULL,
max_bin = 255L,
min_data_in_bin = 3L,
use_missing = TRUE,
zero_as_missing = FALSE,
cat_l2 = 10,
cat_smooth = 10,
max_cat_to_onehot = 4L,
boosting = "gbdt",
data_sample_strategy = "bagging",
drop_rate = NULL,
max_drop = NULL,
skip_drop = NULL,
uniform_drop = NULL,
xgboost_dart_mode = NULL,
drop_seed = NULL,
top_rate = NULL,
other_rate = NULL,
alpha = NULL,
tweedie_variance_power = NULL,
fair_c = NULL,
poisson_max_delta_step = NULL,
sigmoid = NULL,
boost_from_average = NULL,
reg_sqrt = NULL,
device_type = NULL,
tree_learner = "serial",
force_col_wise = TRUE
)LightGBMHyperparameters object.
Integer [1, Inf): Maximum number of boosting rounds.
Optional Integer [1, Inf): Use this many boosting rounds. Disables search for nrounds. NULL selects nrounds during tuning from the best iteration each tuning fit reports, up to max_nrounds: the fits stop early on their validation cases except under "dart" boosting, and a fit that reports no best iteration counts as 100 rounds.
Integer [1, Inf): Number of rounds without improvement to stop training.
(Tunable) Integer [1, Inf): Maximum number of leaves in one tree.
(Tunable) Integer: Maximum depth of trees. -1 = no limit.
(Tunable) Numeric (0, 1]: Learning rate.
(Tunable) Numeric (0, 1]: Fraction of features to use.
(Tunable) Numeric (0, 1]: Fraction of cases sampled per tree.
(Tunable) Integer [1, Inf): Resample every this many iterations.
(Tunable) Numeric [0, Inf): L1 regularization.
(Tunable) Numeric [0, Inf): L2 regularization.
(Tunable) Integer [1, Inf): Maximum number of categories for categorical features.
(Tunable) Integer [1, Inf): Minimum number of observations per categorical group.
(Tunable) Logical: If TRUE, use linear trees.
(Tunable) Logical: If TRUE, use Inverse Frequency Weighting in classification.
Optional Character: Objective function. NULL = set from outcome type.
(Tunable) Numeric [0, Inf): Smallest improvement that resets the early-stopping counter.
Logical: Force reproducible results at some cost in speed.
Integer: Random seed for the objective, where it draws.
Logical: Force row-wise histogram building. Cheaper in memory than column-wise, slower per iteration.
Numeric: Memory cap in MB for the histogram cache; -1 leaves it uncapped.
(Tunable) Optional Integer [1, Inf): Features each machine votes for in
the voting tree learner. Requires tree_learner = "voting". NULL uses LightGBM's
default of 20.
(Tunable) Numeric (0, 1]: Fraction of positive cases sampled per tree, for a binary outcome.
(Tunable) Numeric (0, 1]: Fraction of negative cases sampled per tree, for a binary outcome.
Integer: Random seed for bagging.
(Tunable) Numeric (0, 1]: Fraction of features sampled at each node, on top of the per-tree fraction.
Integer: Random seed for feature sampling.
(Tunable) Numeric [0, Inf): Overall multiplier on every cost-efficiency penalty. Its scale is not comparable between boosting modes.
(Tunable) Numeric [0, Inf): Cost charged per split, in units of loss reduction.
Optional List: Per-feature cost charged the first time a case uses it, in [0, Inf). One vector applied to every tree, or one vector per tree. NULL charges no cost.
Optional List: Per-feature cost charged the first time any case uses it, in [0, Inf). One vector applied to every tree, or one vector per tree. NULL charges no cost.
Logical: Discretize gradients before finding splits. Faster, and changes the fit.
(Tunable) Optional Integer [2, Inf): Bins the gradients are
discretized into. Requires use_quantized_grad = TRUE. NULL uses LightGBM's default of 4.
Optional Logical: Recompute leaf values from the exact
gradients after each iteration. Requires use_quantized_grad = TRUE. NULL uses
LightGBM's default of FALSE.
Optional Logical: Round gradients stochastically rather than
to nearest. Requires use_quantized_grad = TRUE. NULL uses LightGBM's default of TRUE.
Optional Integer [-1, 1] vector: Per-feature monotonicity: 1 increasing, -1 decreasing, 0 unconstrained. One value per feature. NULL constrains no feature.
Optional Character {"basic", "intermediate",
"advanced"}: How monotonicity is enforced. Requires monotone_constraints. NULL uses LightGBM's default, "basic".
(Tunable) Optional Numeric [0, Inf): Depth penalty on splits
that would break monotonicity. Requires monotone_constraints. NULL uses LightGBM's
default of 0.
Optional Character: Feature groups permitted to interact, in LightGBM's own bracketed form, e.g. "[0,1],[2,3]". NULL permits every interaction.
Optional Numeric vector: Per-feature multiplier on split gain, one value per feature. NULL uses 1 for every feature.
(Tunable) Integer [0, Inf): Minimum number of cases in a leaf.
(Tunable) Numeric [0, Inf): Minimum sum of hessians in a leaf: the smooth counterpart of a case count.
(Tunable) Numeric [0, Inf): Minimum loss reduction a split must produce for the split to be made.
(Tunable) Numeric: Cap on each leaf's output; 0 or less leaves it uncapped.
(Tunable) Numeric [0, Inf): Smooth each leaf towards its parent, more strongly the fewer cases it holds.
(Tunable) Logical: Choose one split threshold per feature at random rather than the best.
Integer: Random seed for the thresholds extra_trees draws.
(Tunable) Optional Numeric [0, Inf): L2 penalty on the linear models
fitted at the leaves. Requires linear_tree = TRUE. NULL uses LightGBM's default of 0.
(Tunable) Integer [2, Inf): Most bins a feature is discretized into. Fewer is faster and more regularized.
(Tunable) Integer [1, Inf): Minimum number of cases per bin.
Logical: Handle missing values. FALSE ignores them.
Logical: Treat zeros as missing, rather than only NA.
(Tunable) Numeric [0, Inf): L2 regularization on a categorical split.
(Tunable) Numeric [0, Inf): Smoothing over categorical levels, which reduces the effect of rare ones.
(Tunable) Integer [1, Inf): Below this many levels, split one-versus-rest rather than by partition.
Character {"gbdt", "rf", "dart"}: Boosting algorithm: gradient boosting, random forest, or DART (dropout).
Character {"bagging", "goss"}: How cases are
subsampled: at random, or by gradient magnitude (GOSS). GOSS cannot be combined
with bagging, so it requires bagging_fraction = 1.
(Tunable) Optional Numeric [0, 1]: Fraction of existing trees
to drop per iteration. Requires boosting = "dart". NULL uses LightGBM's
default of 0.1.
(Tunable) Optional Integer: Most trees dropped in one iteration;
0 or less removes the cap. Requires boosting = "dart". NULL uses LightGBM's
default of 50.
(Tunable) Optional Numeric [0, 1]: Probability of skipping
dropout for an iteration. Requires boosting = "dart". NULL uses LightGBM's
default of 0.5.
(Tunable) Optional Logical: Drop trees uniformly rather than
weighted. Requires boosting = "dart". NULL uses LightGBM's default of FALSE.
(Tunable) Optional Logical: Use the XGBoost DART
normalization. Requires boosting = "dart". NULL uses LightGBM's default of
FALSE.
Optional Integer: Random seed for choosing which trees to drop.
Requires boosting = "dart". NULL uses LightGBM's default of 4.
(Tunable) Optional Numeric [0, 1]: Share of large-gradient cases
GOSS retains. Requires data_sample_strategy = "goss". NULL uses LightGBM's
default of 0.2.
(Tunable) Optional Numeric [0, 1]: Share of the remaining cases
GOSS retains; with top_rate it must not exceed 1. Requires
data_sample_strategy = "goss". NULL uses LightGBM's default of 0.1.
(Tunable) Optional Numeric (0, Inf): Huber delta or quantile level.
Requires objective to be "huber" or "quantile". NULL uses LightGBM's default of
0.9.
(Tunable) Optional Numeric [1, 2): Tweedie power: 1 is
Poisson-like, near 2 is gamma-like. Requires objective = "tweedie". NULL uses
LightGBM's default of 1.5.
(Tunable) Optional Numeric (0, Inf): Fair loss scale. Requires
objective = "fair". NULL uses LightGBM's default of 1.
(Tunable) Optional Numeric (0, Inf): Step cap
safeguarding Poisson optimization. Requires objective = "poisson". NULL uses
LightGBM's default of 0.7.
(Tunable) Optional Numeric (0, Inf): Sigmoid slope, for a binary or one-vs-all objective. NULL uses LightGBM's default of 1.
Optional Logical: Start boosting from the constant score that best fits the outcome under the objective, such as its mean for regression. NULL uses LightGBM's default of TRUE.
Optional Logical: Fit the square root of the outcome and square the prediction back, for the "regression", "regression_l1", "fair", "quantile" or "mape" objective. NULL uses LightGBM's default of FALSE.
Optional Character: Deprecated and ignored. Set the compute device in the
execution_config passed to train, e.g.
setup_FutureExecution(device = "cuda"); "gpu" is device = "opencl" there.
Character {"serial", "feature", "data", "voting"}: Tree learner type.
Logical: Use only with CPU - If TRUE, force col-wise histogram building.
EDG
Get more information from lightgbm::lgb.train.
nrounds is auto-tuned using early stopping (up to max_nrounds) unless
force_nrounds is set.
lightgbm_hyperparams <- setup_LightGBM(
max_nrounds = 500L,
learning_rate = tune_over(0.001, 0.01, 0.05), ifw = TRUE
)
lightgbm_hyperparams
Run the code above in your browser using DataLab