DMR: Delete or Merge Regressors

Description

Fits a path of linear (family="gaussian") or logistic (family="binomial") regression models, where the number of parameters changes from 1 to p (p is the number of columns in the model matrix). Models are subsets of continuous predictors and partitions of levels of factors in X.

Usage

DMR(
  X,
  y,
  family = "gaussian",
  clust.method = "complete",
  lam = 10^(-7),
  lambda = NULL
)

Value

An object with S3 class "DMR", which is a list with the ingredients:

beta: Matrix p times p of estimated parameters; each column corresponds to a model on the nested path having from p to 1 parameter (denoted as df).
df: Vector of degrees of freedom; from p to 1.
rss/loglik: Measure of fit for the nested models: rss (residual sum of squares) is returned for family="gaussian" and loglik (loglikelihood) is returned for family="binomial".
n: Number of observations.
levels.listed: Minimal set of levels of respective factors present in data.
lambda: The net of lambda values used in the screening step, empty vector in case of DMR.
arguments: List of the chosen arguments from the function call.
interc: If the intercept was fitted: for DMR always equal to TRUE.

Arguments

X: Input data frame; each row is an observation vector; each column can be numerical or integer for a continuous predictor or a factor for a categorical predictor; DMR works only if p<n (n is the number of observations, p the number of columns in the model matrix), for p>=n see DMRnet.
y: Response variable; Numerical for family="gaussian" or a factor with two levels for family="binomial". For family="binomial" the last level in alphabetical order is the target class.
family: Response type; one of: "gaussian", "binomial".
clust.method: Clustering method used for partitioning levels of factors; see function hclust in package stats for details. clust.method="complete" is the default.
lam: The amount of penalization in ridge regression (used for logistic regression in order to allow for parameter estimation in linearly separable setups) or the amount of matrix regularization in case of linear regression. Used only for numerical reasons. The default is 1e-7.
lambda: The net of lambda values. It is optional and serves only for consistency with DMRnet. It is not used in DMR.

Details

DMR algorithm is based on a traditional stepwise method. A nested family of models is built based on the values of squared Wald statistics:

1. For each continuous variable the squared Wald statistic is calculated for a hypothesis that the variable is equal to zero (it should be deleted).

2. For each factor a dissimilarity matrix is constructed using squared Wald statistics for hypotheses that two parameters are equal (the two levels of factor should be merged). Next, hierarchical clustering is preformed using the dissimilarity matrix. All cutting heights are recorded.

3. Squared Wald statistics and cutting heights and values of from steps 2 and 3 are concatenated and sorted, resulting in vector h.

4. Nested family of models of size 1 to p is built by accepting hypotheses according to increasing values in vector h.

Examples

Run this code

## DMR for linear regression
data(miete)
ytr <- miete[1:1500,1]
Xtr <- miete[1:1500,-1]
Xte <- miete[1501:2053,-1]
m1 <- DMR(Xtr, ytr)
print(m1)
plot(m1)
g <- gic.DMR(m1, c = 2.5)
plot(g)
coef(m1, df = g$df.min)
ypr <- predict(m1, newx = Xte, df = g$df.min)

## DMR for logistic regression
# notice that only part of dataset promoter was used: DMR works only if p=n use DMRnet
data(promoter)
ytr <- factor(promoter[1:80,1])
Xtr <- promoter[1:80,2:11]
Xte <- promoter[81:106,2:11]
m2 <- DMR(Xtr, ytr, family = "binomial")
print(m2)
plot(m2)
g <- gic.DMR(m2, c = 2)
plot(g)
coef(m2, df = g$df.min)
ypr <- predict(m2, newx = Xte, df = g$df.min)