isgd

0th

Percentile

Implicit Stochastic Gradient Descent Optimizer

This optimizer performs an implicit stochastic gradient descent algorithm. It is mainly used within a bamlss call.

Keywords
regression
Usage
isgd(x, y, family, weights = NULL, offset = NULL,
     gammaFun = function(i) 1/(1 + i), shuffle = TRUE,
     CFun = function(beta) diag(length(beta)),
     start = NULL, i.state = 0)
Arguments
x

For function boost() the x list, as returned from function bamlss.frame, holding all model matrices and other information that is used for fitting the model.

y

The model response, as returned from function bamlss.frame.

family

A bamlss family object, see family.bamlss.

weights

Prior weights on the data, as returned from function bamlss.frame.

offset

Can be used to supply model offsets for use in fitting, returned from function bamlss.frame.

gammaFun

Function specifying the step length.

shuffle

Should the data be shuffled?

CFun

Hessian approximating function.

start

Vector of starting values.

i.state

Added to gammaFUN().

Details

tpf

Value

For function isgd() a list containing the following objects:

fitted.values

A named list of the fitted values based on the last iteration of the modeled parameters of the selected distribution.

parameters

A matrix, each row corresponds to the parameter values of one iteration.

sgd.summary

The summary of the stochastic gradient descent algorithm which can be printed and plotted.

Note

Motivated by the lecture 'Regression modelling with large data sets' given by Ioannis Kosmidis in Innsbruck, January 2017.

Warning

CAUTION: Arguments weights and offset are not implemented yet!

References

Toulis, P and Airoldi, EM (2015): Scalable estimation strategies based on stochastic approximations: Classical results and new insights. Statistics and Computing, 25, no. 4, 781--795. doi: 10.1007/s11222-015-9560-y

See Also

bamlss.frame, bamlss

Aliases
  • isgd
Examples
# NOT RUN {
set.seed(111)
d <- GAMart(n = 10000)
f <- num ~ s(x1) + s(x2) + s(x3) + te(lon, lat)
b <- bamlss(f, data = d, optimizer = bamlss:::isgd, sampler = FALSE)
plot(b, ask = F)

## loop over observations a 2nd time
b <- bamlss(f, data = d, optimizer = bamlss:::isgd, sampler = FALSE, start = parameters(b),
            i.state = b$model.stats$optimizer$sgd.summary$i.state)
plot(b, ask = F)

## try differeent gammaFuns, e.g.,
# gammaFun <- function(i) .3/sqrt((1+i)) + 0.001

## testing some families
f2 <- bin ~ s(x1) + s(x2) + s(x3) + te(lon, lat)
b2 <- bamlss(f2, data = d, optimizer = isgd, sampler = FALSE, family = "binomial")

f3 <- cens ~ s(x1) + s(x2) + s(x3) + te(lon, lat)
b3 <- bamlss(f3, data = d, optimizer = isgd, sampler = FALSE, family = "cnorm")
# }
Documentation reproduced from package bamlss, version 1.1-2, License: GPL-2 | GPL-3

Community examples

Looks like there are no examples yet.