precision: Measures of Precision and Shrinkage for Ridge Regression

Description

Calculates measures of precision based on the size of the estimated covariance matrices of the parameters and shrinkage of the parameters in a ridge regression model.

Usage

precision(object, ...)
# S3 method for ridge
precision(object, det.fun=c("log","root"), normalize=TRUE, ...)
# S3 method for lm
precision(object, det.fun=c("log","root"), normalize=TRUE, ...)

Arguments

object

An object of class ridge or lm

det.fun

Function to be applied to the determinants of the covariance matrices, one of c("log","root").

normalize

If TRUE the length of the coefficient vector is normalized to a maximum of 1.0.

…

Other arguments (currently unused)

Value

A data.frame with the following columns

lambda

The ridge constant

The equivalent effective degrees of freedom

det

The det.fun function of the determinant of the covariance matrix

trace

The trace of the covariance matrix

max.eig

Maximum eigen value of the covariance matrix

norm.beta

The root mean square of the estimated coefficients, possibly normalized

%% ...

Details

Three measures of (inverse) precision based on the “size” of the covariance matrix of the parameters are calculated. Let $V_{k}$ be the covariance matrix for a given ridge constant, and let $λ_{i}, i = 1, \dots p$ be its eigenvalues

$\log | V_{k} | = \log \prod λ$ or $| V_{k} |^{1 / p} = (\prod λ)^{1 / p}$ measures the linearized volume of the covariance ellipsoid and corresponds conceptually to Wilks' Lambda criterion
$t r a c e (V_{k}) = \sum λ$ corresponds conceptually to Pillai's trace criterion
$λ_{1} = m a x (λ)$ corresponds to Roy's largest root criterion.

Examples

Run this code

# NOT RUN {
longley.y <- longley[, "Employed"]
longley.X <- data.matrix(longley[, c(2:6,1)])

lambda <- c(0, 0.005, 0.01, 0.02, 0.04, 0.08)
lridge <- ridge(longley.y, longley.X, lambda=lambda)
clr <- c("black", rainbow(length(lambda)-1, start=.6, end=.1))
coef(lridge)

(pdat <- precision(lridge))
# plot log |Var(b)| vs. length(beta)
with(pdat, {
	plot(norm.beta, det, type="b", 
	cex.lab=1.25, pch=16, cex=1.5, col=clr, lwd=2,
	xlab='shrinkage: ||b|| / max(||b||)',
	ylab='variance: log |Var(b)|')
	text(norm.beta, det, lambda, cex=1.25, pos=c(rep(2,length(lambda)-1),4))
	text(min(norm.beta), max(det), "Variance vs. Shrinkage", cex=1.5, pos=4)
	})

# plot trace[Var(b)] vs. length(beta)
with(pdat, {
	plot(norm.beta, trace, type="b",
	cex.lab=1.25, pch=16, cex=1.5, col=clr, lwd=2,
	xlab='shrinkage: ||b|| / max(||b||)',
	ylab='variance: trace [Var(b)]')
	text(norm.beta, trace, lambda, cex=1.25, pos=c(2, rep(4,length(lambda)-1)))
#	text(min(norm.beta), max(det), "Variance vs. Shrinkage", cex=1.5, pos=4)
	})

# }

Run the code above in your browser using DataLab

Get 50% off unlimited learning