kerasR (version 0.6.1)

Optimizers: Optimizers

Description

Optimization functions to use in compiling a keras model.

Usage

SGD(lr = 0.01, momentum = 0, decay = 0, nesterov = FALSE,
  clipnorm = -1, clipvalue = -1)
RMSprop(lr = 0.001, rho = 0.9, epsilon = 1e-08, decay = 0,
  clipnorm = -1, clipvalue = -1)
Adagrad(lr = 0.01, epsilon = 1e-08, decay = 0, clipnorm = -1,
  clipvalue = -1)
Adadelta(lr = 1, rho = 0.95, epsilon = 1e-08, decay = 0,
  clipnorm = -1, clipvalue = -1)
Adam(lr = 0.001, beta_1 = 0.9, beta_2 = 0.999, epsilon = 1e-08,
  decay = 0, clipnorm = -1, clipvalue = -1)
Adamax(lr = 0.002, beta_1 = 0.9, beta_2 = 0.999, epsilon = 1e-08,
  decay = 0, clipnorm = -1, clipvalue = -1)
Nadam(lr = 0.002, beta_1 = 0.9, beta_2 = 0.999, epsilon = 1e-08,
  schedule_decay = 0.004, clipnorm = -1, clipvalue = -1)

Arguments

lr

float >= 0. Learning rate.

momentum

float >= 0. Parameter updates momentum.

decay

float >= 0. Learning rate decay over each update.

nesterov

boolean. Whether to apply Nesterov momentum.

clipnorm

float >= 0. Gradients will be clipped when their L2 norm exceeds this value. Set to -1 to disable.

clipvalue

float >= 0. Gradients will be clipped when their absolute value exceeds this value. Set to -1 to disable.

rho

float >= 0 to be used in RMSprop

epsilon

float >= 0. Fuzz factor.

beta_1

float, 0 < beta < 1. Generally close to 1.

beta_2

float, 0 < beta < 1. Generally close to 1.

schedule_decay

float >= 0. Learning rate decay over each schedule in Nadam.

Author

Taylor B. Arnold,

References

Chollet, Francois. 2015. Keras: Deep Learning library for Theano and TensorFlow.

Examples

if(keras_available()) {
  X_train <- matrix(rnorm(100 * 10), nrow = 100)
  Y_train <- to_categorical(matrix(sample(0:2, 100, TRUE), ncol = 1), 3)
  
  mod <- Sequential()
  mod$add(Dense(units = 50, input_shape = dim(X_train)[2]))
  mod$add(Activation("relu"))
  mod$add(Dense(units = 3))
  mod$add(Activation("softmax"))
  
  keras_compile(mod,  loss = 'categorical_crossentropy', optimizer = SGD())
  keras_fit(mod, X_train, Y_train, batch_size = 32, epochs = 5,
            verbose = 0, validation_split = 0.2)
  
  keras_compile(mod,  loss = 'categorical_crossentropy', optimizer = RMSprop())
  keras_fit(mod, X_train, Y_train, batch_size = 32, epochs = 5,
            verbose = 0, validation_split = 0.2)
  
  keras_compile(mod,  loss = 'categorical_crossentropy', optimizer = Adagrad())
  keras_fit(mod, X_train, Y_train, batch_size = 32, epochs = 5,
            verbose = 0, validation_split = 0.2)
  
  keras_compile(mod,  loss = 'categorical_crossentropy', optimizer = Adadelta())
  keras_fit(mod, X_train, Y_train, batch_size = 32, epochs = 5,
            verbose = 0, validation_split = 0.2)
  
  keras_compile(mod,  loss = 'categorical_crossentropy', optimizer = Adam())
  keras_fit(mod, X_train, Y_train, batch_size = 32, epochs = 5,
            verbose = 0, validation_split = 0.2)
  
  keras_compile(mod,  loss = 'categorical_crossentropy', optimizer = Adamax())
  keras_fit(mod, X_train, Y_train, batch_size = 32, epochs = 5,
            verbose = 0, validation_split = 0.2)
  
  keras_compile(mod,  loss = 'categorical_crossentropy', optimizer = Nadam())
  keras_fit(mod, X_train, Y_train, batch_size = 32, epochs = 5,
            verbose = 0, validation_split = 0.2)
}