General-purpose unconstrained non-linear optimization
An algorithm for general-purpose unconstrained non-linear optimization. The algorithm is of quasi-Newton type with BFGS updating of the inverse Hessian and soft line search with a trust region type monitoring of the input to the line search algorithm. The interface of ucminf is designed for easy interchange with optim.
ucminf(par, fn, gr = NULL, ..., control = list(), hessian=0)
Initial estimate of minimum for
- Objective function to be minimized.
Gradient of objective function. If
NULLa finite difference approximation is used.
- Optional arguments passed to the objective and gradient functions.
- A list of control parameters. See Details.
- No hessian approximation is returned.
- Returns a numerical approximation of the Hessian using hessian in the package numDeriv.
- Returns final approximation of the inverse Hessian based on the series of BFGS updates during optimization.
- Same at 2, but will also return the Hessian (the inverse of 2).
FALSEvalue is given it will switch between option 1 or 0.
The algorithm is documented in (Nielsen, 2000) (see References below) together with a comparison to the Fortran subroutine MINF and the Matlab function fminunc. The implementation of ucminf in R uses the original Fortran version of the algorithm.
The interface in R is designed so that it is very easy to switch
between using ucminf and optim. The
are all the same (with a few extra options for
ucminf). The difference is that there is no
argument in ucminf and that some of the components in the
control argument are different due to differences in the
The algorithm can be given an initial estimate of the Hessian for the optimization and it is possible to get the final approximation of the Hessian based on the series of BFGS updates. This extra functionality may be useful for optimization in a series of related problems.
gr can return
if the functions cannot be evaluated at the supplied value, but the
functions must be computable at the initial value. The functions
are not allowed to return
NA. Any names given to
par will be
copied to the vectors passed to
other attributes of
par are copied over.
control argument is a list that can supply any of the
- If trace is positive then detailed tracing information is printed for each iteration.
grtol = 1e-6. =$>
xtol = 1e-12.=>
stepmax = 1.
maxeval = 500.
grad = 'forward'.
gradstep = c(1e-6, 1e-8).
H0is the initial hessian matrix then the lower triangle of the inverse of
H0can be found as
invhessian.lt = solve(H0)[lower.tri(H0,diag=TRUE)].
- Computed minimizer.
- Objective function value at computed minimizer.
- Flag for reason of termination:
- Stopped by small gradient (grtol).
- Stopped by small step (xtol).
- Stopped by function evaluation limit (maxeval).
- Stopped by zero step from line search
- Computation did not start: length(par) = 0.
- Computation did not start: stepmax is too small.
- Computation did not start: grtol or xtol <= 0.<="" dd="">
- Computation did not start: maxeval <= 0.<="" dd="">
- Computation did not start: given Hessian not pos. definite. =>
- $||F'(x)||_inf$, the largest element in the absolute value of the gradient at the computed minimizer.
- Length of last step.
- Final maximal allowed step length.
- Number of calls to both objective and gradient function.
Nielsen, H. B. (2000) UCMINF - An Algorithm For Unconstrained, Nonlinear Optimization, Report IMM-REP-2000-18, Department of Mathematical Modelling, Technical University of Denmark. http://www2.imm.dtu.dk/~hbn/publ/TR0019.ps or http://orbit.dtu.dk/recid/200975.
The original Fortran source code can be found at http://www2.imm.dtu.dk/~hbn/Software/ucminf.f. The code has been slightly modified in this package to be suitable for use with R.
The general structure of the implementation in R is based on the package FortranCallsR by Diethelm Wuertz.
## Rosenbrock Banana function fR <- function(x) (1 - x)^2 + 100 * (x - x^2)^2 gR <- function(x) c(-400 * x * (x - x * x) - 2 * (1 - x), 200 * (x - x * x)) # Find minimum and show trace ucminf(par = c(2,.5), fn = fR, gr = gR, control = list(trace = 1))