BayesFluxR (version 0.1.3)

madapter.RMSProp: Use RMSProp to adapt the inverse mass matrix.

Description

Use RMSProp as a preconditions/mass matrix adapter. This was proposed in Li, C., Chen, C., Carlson, D., & Carin, L. (2016, February). Preconditioned stochastic gradient Langevin dynamics for deep neural networks. In Thirtieth AAAI Conference on Artificial Intelligence for the use in SGLD and related methods.

Usage

madapter.RMSProp(adapt_steps, lambda = 1e-05, alpha = 0.99)

Value

list with `juliavar` and `juliacode` and all given arguments

Arguments

adapt_steps

number of adaptation steps

lambda

see above paper

alpha

see above paper

Examples

if (FALSE) {
  ## Needs previous call to `BayesFluxR_setup` which is time
  ## consuming and requires Julia and BayesFlux.jl
  BayesFluxR_setup(installJulia=TRUE, seed=123)
  net <- Chain(Dense(5, 1))
  like <- likelihood.feedforward_normal(net, Gamma(2.0, 0.5))
  prior <- prior.gaussian(net, 0.5)
  init <- initialise.allsame(Normal(0, 0.5), like, prior)
  x <- matrix(rnorm(5*100), nrow = 5)
  y <- rnorm(100)
  bnn <- BNN(x, y, like, prior, init)
  madapter <- madapter.RMSProp(100)
  sampler <- sampler.GGMC(madapter = madapter)
  ch <- mcmc(bnn, 10, 1000, sampler)
}