gvcm.cat (version 1.8)

gvcm.cat: Regularized Categorical Effects/Categorical Effect Modifiers/Continuous/Smooth effects in GLMs

Description

The function fits generalized linear models with regularized categorical effects, categorical effect modifiers, continuous effects and smooth effects. The model is specified by giving a symbolic description of the linear predictor and a description of the error distribution. Estimation employs different regularization and model selection strategies. These strategies are either a penalty or a forward selection strategy employing AIC/BIC. For non-differentiable penalties, a local quadratic approximation is employed, see Oelker and Tutz (2013).

Usage

gvcm.cat(formula, data, family = gaussian, method = c("lqa", "AIC", "BIC"), 
tuning = list(lambda=TRUE, specific=FALSE, phi=0.5, grouped.fused=0.5, 
elastic=0.5, vs=0.5, spl=0.5), weights, offset, start, control, 
model = FALSE, x = FALSE, y = FALSE, plot=FALSE, ...)

pest(x, y, indices, family = gaussian, 
tuning = list(lambda=TRUE, specific=FALSE, phi=0.5, grouped.fused=0.5, 
elastic=0.5, vs=0.5, spl=0.5), weights, offset, start = NULL, 
control = cat_control(), plot=FALSE, ...)

abc(x, y, indices, family = gaussian, tuning = c("AIC", "BIC"), 
weights, offset, start, control = cat_control(), plot=FALSE, ...)

Arguments

formula

an object of class formula: a symbolic description of the model to be fitted. See details

data

a data frame, containing the variables in the model

family

a family object describing the error distribution and link function to be used in the model; this can be a character string naming a family function, a family function or the result of a call to a family function, see

method

fitting method; one out of "lqa", "AIC" or "BIC"; method "lqa" induces penalized estimation; it employs a PIRLS-algorithm (see Fan and Li, 2001; Oelker and Tutz, 2013).

tuning

a list; tuning parameters for penalized estimation; lambda is the scalar, overall penalty parameter; if lambda is a vector of values, these values are cross-validated; if lambda = TRUE, lambda

weights

an optional weight vector (for the observations)

offset

an optional offset

start

initial values for the PIRLS algorithm for method lqa

control

a list of parameters for controlling the fitting process; if emtpy, set to cat_control(); see cat_control

model

for functions gvcm.cat: a logical value indicating whether the employed model frame shall be returned or not

x, y

for function gvcm.cat: logical values indicating whether the response vector and the model matrix used in the fitting process shall be returned or not; for functions pest and abc: y must be a re

plot

logical; if TRUE, estimates needed to plot coefficient paths are computed

indices

for pest and abc only: the to be used index argument; see function index

...

further arguments passed to or from other methods

Value

• gvcm.cat returns an object of class gvcm.cat which inherits from class glm which inherits from class lm. An object of class gvcm.cat contains:
• coefficients
  named vector of coefficients
• coefficients.reduced
  reduced vector of coefficients; selected coefficients/differences of coefficients are set to zero
• coefficients.refitted
  refitted vector of coefficients; i.e. maximum likelihood estimate of that model containing selected covariates only; same length as coefficients.reduced
• coefficients.oml
  maximum likelihood estimate of the full model
• residuals
  deviance residuals
• fitted.values
  fitted mean values
• rank
  degrees of freedom model; for method="lqa" estimated by the trace of the generalized head matrix; for methods "AIC", "BIC" estimated like default in glm.fit
• family
  the family object used
• linear.predictors
  linear fit on link scale
• deviance
  scaled deviance
• aic
  a version of Akaike's Information Criterion; minus twice the maximized log-likelihood plus twice the rank. For binomial and Poison families the dispersion is fixed at one. For a gaussian family the dispersion is estimated from the residual deviance, and the number of parameters is the rank plus one.
• null.deviance
  the deviance for the null model, comparable with deviance; the null model includes a non-varying intercept only
• iter
  number of iterations
• weights
  working weights of the final iteration
• df.residual
  the residual degrees of freedom/degrees of freedom error; computed like rank
• df.null
  the residual degrees of freedom for the null model
• converged
  logical; fulfills the PIRLS-algorithm the given convergence conditions?
• boundary
  logical; is the fitted value on the boundary of the attainable values?
• offset
  the offset vector used
• control
  the value of the control argument used
• contrasts
  the contrasts used
• na.action
  information returned by model.frame on the special handling of NAs; currently always na.omit
• plot
  in principle, a list containing two matrixes needed for different types of plots: if input option plot=TRUE, the first matrix contains estimates needed to plot coefficient paths; if lambda was cross-validated, the second matrix contains the cross-validation scores
• tuning
  a list, employed tuning parameters; if lambda was cross-validated, the optimal value is returned
• indices
  used index argument; see function index
• number.selectable.parameters
  number of coefficients that could be selected
• number.removed.parameters
  number of actual removed coefficients
• x.reduction
  a matrix; transforms model frame x into its reduced version; e.g. needed for refitting
• beta.reduction
  a matrix; transforms the coefficients into its reduced version
• call
  the matched call
• formula
  the formula supplied
• terms
  the terms object used
• data
  the data argument
• x, y
  if requested, the model matrix/the response vector
• model
  if requested, the model frame
• xlevels
  a record of the levels of the factors used in fitting
• bootstrap.errors
  experimental
• method
  same as input argument method
• In addition, non-empty fits will have components qr, R and effects relating to the final weighted linear fit.

Details

A typical formula has the form response ~ 1 + terms; where response is the response vector and terms is a series of terms which specifies a linear predictor. There are some special terms for regularized terms:

• v(x, u, n="L1", bj=TRUE): varying coefficients enter theformulaasv(x,u)whereudenotes the categorical effect modifier andxthe modfied covariate. A varying intercept is denoted byv(1,u). Varying coefficients with categorical effect modifiers are penalized as described in Oelker et. al. 2012. The argumentbjand the elementphiin argumenttuningallow for the described weights.
• p(u, n="L1"): ordinal/nominal covariatesugiven asp(u)are penalized as described in Gertheiss and Tutz (2010). For numeric covariates,p(u)indicates a pure Lasso penalty.
• grouped(u, ...): penalizes a group of covariates with the grouped Lasso penalty of Yuan and Lin (2006); so far, working for categorical covariates only
• sp(x, knots=20, n="L2"): implents a continuousxcovariate non-parametrically as$f(x)$;$f(x)$is represented by centered evaluations of basis functions (cubic B-splines with number of knots =knots); forn="L2", the curvature of$f(x)$is penalized by a Ridge penalty; see Eilers and Marx (1996)
• SCAD(u): penalizes a covariateuwith the SCAD penalty by Fan and Li (2001); for categorical covariatesu, differences of coefficients are penalized by a SCAD penalty, see Gertheiss and Tutz (2010)
• elastic(u): penalizes a covariateuwith the elastic net penalty by Zou and Hastie (2005); for categorical covariatesu, differences of coefficients are penalized by the elastic net penalty, see Gertheiss and Tutz (2010)

If the formula contains no (varying) intercept, gvcm.cat assumes a constant intercept. There is no way to avoid an intercept. For specials p and v, there is the special argument n: if n="L1", the absolute values in the penalty are replaced by squares of the same terms; if n="L2", the absolute values in the penalty are replaced by quadratic, Ridge-type terms; if n="L0", the absolute values in the penalty are replaced by an indicator for non-zero entries of the same terms. For methods "AIC" and "BIC", the coefficients are not penalized but selected by a forward selection strategy whenever it makes sense; for special v(x,u), the selection strategy is described in Oelker et. al. 2012; the approach for the other specials corresponds to this idea. For binomial families the response can also be a success/failure rate or a two-column matrix with the columns giving the numbers of successes and failures. Function pest computes penalized estimates, that is, it implements method "lqa" (PIRLS-algorithm). Function abc implements the forward selection strategy employing AIC/BIC. Categorical effect modifiers and penalized categorical covariates are dummy coded as required by the penalty. If x in v(x,u) is binary, it is effect coded (first category refers to -1). Other covariates are coded like given by getOption. There is a summary function: summary.gvcm.cat

References

Eilers, P. H. C. and B. D. Marx (1996). Flexible smoothing with b-splines and penalties. Statist. Sci. 11 (2), 89-121. Fan, J. and R. Li (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association 96(456), 1348-1360. Gertheiss, J. and G. Tutz (2010). Sparse modeling of categorial explanatory variables. The Annals of Statistics 4(4), 2150-2180. Oelker, M.-R., J. Gertheiss and G. Tutz (2012). Regularization and model melection with categorial predictors and effect modifiers in generalized linear models. Department of Statistics at the University of Munich: Technical Report 122. Oelker, M.-R., J. Gertheiss and G. Tutz (2013). A general family of penalties for combining differing types of penalties in generalized structured models. Department of Statistics at the University of Munich: Technical Report 139. Yuan, M. and Y. Lin (2006). Model selection and estimation in regression with grouped variables. R. Stat. Soc. Ser. B Stat. Methodol. 68 (1), 49-67. Zou, H. and T. Hastie (2005). Regularization and variable selection via the Elastic Net. R. Stat. Soc. Ser. B Stat. Methodol. 67 (2), 301-320.

See Also

Functions index, cat_control, plot.gvcm.cat, predict.gvcm.cat, simulation

Examples

## example for function simulation()
covariates <- list(x1=list("unif", c(0,2)),
                  x2=list("unif", c(0,2)),
                  x3=list("unif", c(0,2)),
                  u=list("multinom",c(0.3,0.4,0.3), "nominal")
                  )
true.f <- y ~ 1 + v(x1,u) + x2
true.coefs <- c(0.2,  0.3,.7,.7, -.5)
data <- simulation(400, covariates, NULL, true.f, true.coefs , binomial(), seed=456)
## example for function gvcm.cat()
f <- y ~ v(1,u) + v(x1,u) + v(x2,u)
m1 <- gvcm.cat(f, data, binomial(), plot=TRUE, control=cat_control(lambda.upper=19))
summary(m1)
## example for function predict.gvcm.cat
newdata <- simulation(200, covariates, NULL, true.f, true.coefs , binomial(), seed=789)
prediction <- predict.gvcm.cat(m1, newdata) 
## example for function plot.gvcm.cat 
plot(m1)
plot(m1, type="score")
plot(m1, type="coefs")