earth

0th

Percentile

Multivariate Adaptive Regression Splines

Build a regression model using the techniques in Friedman's papers "Multivariate Adaptive Regression Splines" and "Fast MARS".

See the package vignette ../doc/earth-notes.pdf{Notes on the earth package}.

Keywords
models, regression, smooth
Usage
## S3 method for class 'formula':
earth(formula = stop("no 'formula' argument"), data = NULL,
   weights = NULL, wp = NULL, subset = NULL,
   na.action = na.fail,
   pmethod = c("backward", "none", "exhaustive", "forward", "seqrep", "cv"),
   keepxy = FALSE, trace = 0, glm = NULL, degree = 1, nprune = NULL,
   ncross=1, nfold=0, stratify=TRUE,
   varmod.method = "none", varmod.exponent = 1,
   varmod.conv = 1, varmod.clamp = .1, varmod.minspan = -3,
   Scale.y = (NCOL(y)==1), ...)

## S3 method for class 'default': earth(x = stop("no 'x' argument"), y = stop("no 'y' argument"), weights = NULL, wp = NULL, subset = NULL, na.action = na.fail, pmethod = c("backward", "none", "exhaustive", "forward", "seqrep", "cv"), keepxy = FALSE, trace = 0, glm = NULL, degree = 1, nprune = NULL, ncross=1, nfold=0, stratify=TRUE, varmod.method = "none", varmod.exponent = 1, varmod.conv = 1, varmod.clamp = .1, varmod.minspan = -3, Scale.y = (NCOL(y)==1), ...)

## S3 method for class 'fit': earth(x = stop("no 'x' argument"), y = stop("no 'y' argument"), weights = NULL, wp = NULL, subset = NULL, na.action = na.fail, pmethod = c("backward", "none", "exhaustive", "forward", "seqrep", "cv"), keepxy = FALSE, trace = 0, glm = NULL, degree = 1, penalty = if(degree > 1) 3 else 2, nk = min(200, max(20, 2 * ncol(x))) + 1, thresh = 0.001, minspan = 0, endspan = 0, newvar.penalty = 0, fast.k = 20, fast.beta = 1, linpreds = FALSE, allowed = NULL, nprune = NULL, Object = NULL, Scale.y = (NCOL(y)==1), Adjust.endspan = 2, Force.weights = FALSE, Use.beta.cache = TRUE, Force.xtx.prune = FALSE, Get.leverages = NROW(x) < 1e5, Exhaustive.tol = 1e-10, ...)

Arguments
formula
Model formula.
data
Data frame for formula.
x
Matrix or dataframe containing the independent variables.
y
Vector containing the response variable, or, in the case of multiple responses, a matrix or dataframe whose columns are the values for each response.
subset
Index vector specifying which cases to use, i.e., which rows in x to use. Default is NULL, meaning all.
weights
Case weights. Default is NULL, meaning no case weights. If specified, weights must have length equal to nrow(x) before applying subset. Zero weights are converted to a very small nonzero value.
wp
Response weights. Default is NULL, meaning no response weights. If specified, wp must have an element for each column of y (after factors in y
na.action
NA action. Default is na.fail, and only na.fail is supported.
keepxy
Default is FALSE. Set to TRUE to retain the following in the returned value: x and y (or data), subset, and weights. The function
trace
Trace earth's execution. Default is 0. Values: 0 no tracing .3 variance model (the varmod.method arg) .5 cross validation (the nfold arg) 1 overview
glm
NULL (default) or a list of arguments to pass on to glm. See the documentation of glm for a description of these arguments See Generalized line
degree
Maximum degree of interaction (Friedman's $mi$). Default is 1, meaning build an additive model (i.e., no interaction terms).
penalty
Generalized Cross Validation (GCV) penalty per knot. Default is if(degree>1) 3 else 2. Simulation studies suggest values in the range of about 2 to 4. The FAQ section in the vignette has some information
nk
Maximum number of model terms before pruning, i.e., the maximum number of terms created by the forward pass. Includes the intercept. The actual number of terms created by the forward pass will often be less than nk because of o
thresh
Forward stepping threshold. Default is 0.001. This is one of the arguments used to decide when forward stepping should terminate: the forward pass terminates if adding a term changes RSq by less than thresh.
minspan
Minimum number of observations between knots. (This increases resistance to runs of correlated noise in the input data.) The default minspan=0 is treated specially and means calculate the minspan internally, as per Fr
endspan
Minimum number of observations before the first and after the final knot. The default endspan=0 is treated specially and means calculate the minspan internally, as per the MARS paper equation 45 with $alpha$ = 0.05. S
newvar.penalty
Penalty for adding a new variable in the forward pass (Friedman's $gamma$, equation 74 in the MARS paper). Default is 0, meaning no penalty for adding a new variable. Useful non-zero values typically range from about 0.01
fast.k
Maximum number of parent terms considered at each step of the forward pass. (This speeds up the forward pass. See the Fast MARS paper section 3.0.) Default is 20. A value of 0 is treated specially (as being equival
fast.beta
Fast MARS ageing coefficient, as described in the Fast MARS paper section 3.1. Default is 1. A value of 0 sometimes gives better results.
linpreds
Index vector specifying which predictors should enter linearly, as in lm. The default is FALSE, meaning all predictors enter in the standard MARS fashion, i.e., in hinge functions. This d
allowed
Function specifying which predictors can interact and how. Default is NULL, meaning all standard MARS terms are allowed. During the forward pass, earth calls the allowed function before considering a term for inclusion;
pmethod
Pruning method. One of: backward none exhaustive forward seqrep cv. Default is "backward". New in version 4.4.0: Specify pmethod="cv" to use cross-validation to select the number of terms. This
nprune
Maximum number of terms (including intercept) in the pruned model. Default is NULL, meaning all terms created by the forward pass (but typically not all terms will remain after pruning). Use this to enforce an upper bound on the model size
ncross
Only applies if nfold>1. Number of cross-validations. Each cross-validation has nfold folds. Default 1.
nfold
Number of cross-validation folds. Default is 0, no cross validation. If greater than 1, earth first builds a standard model as usual with all the data. It then builds nfold cross-validated
stratify
Only applies if nfold>1. Default is TRUE. Stratify the cross-validation samples so that an approximately equal number of cases with a non-zero response occur in each cross validation subset. So if the res
varmod.method
Construct a variance model. For details, see varmod and the vignette ../doc/earth-varmod.pdf{Variance models in earth}. Use trace=.3 to trace construction of the va
varmod.exponent
Power transform applied to the rhs before regressing the absolute residuals with the specified varmod.method. Default is 1. For example, with varmod.method="lm", if you expect the standard deviance to increase linear
varmod.conv
Convergence criterion for the Iteratively Reweighted Least Squares used when creating the variance model. Iterations stop when the mean value of the coefficients of the residual model change by less than varmod.conv percent. Default is
varmod.clamp
The estimated standard deviation of the main model errors is forced to be at least a small positive value, which we call min.sd. This prevents negative or absurdly small estimated standard deviations. Clamping takes place in predict.var
varmod.minspan
Only applies when varmod.method="earth" or "x.earth". This is the minspan used in the internal call to earth when creating the variance model (not the main earth model). Default is -3<
Object
Earth object to be updated, for use by update.earth.
Scale.y
Scale y in the forward pass for better numeric stability. Scaling here means subtract the mean and divide by the standard deviation. Default is NCOL(y)==1, i.e., scale y<
Adjust.endspan
New in version 4.2.0. In interaction terms, endspan gets multiplied by this value. This reduces the possibility of an overfitted interaction term supported by just a few cases on the boundary of the predictor space (as sometimes seen i
Force.weights
Default is FALSE. For testing the weights argument. Force use of the code for handling weights in the earth code, even if weights=NULL or all the weights are the same. This will not necessarily generate
Use.beta.cache
Default is TRUE. Using the beta cache takes a little more memory but is faster (by 20% and often much more for large models). The beta cache uses nk * nk * ncol(x) * sizeof(double) bytes. (The beta cache is an in
Force.xtx.prune
Default is FALSE. This argument pertains to subset evaluation in the pruning pass. By default, if y has a single column then earth calls the leaps routines; if
Get.leverages
New in version 4.4.0. Default is TRUE unless the model has more than 100 thousand cases. The leverages are the diagonal hat values for the linear regression of y on bx. The leverages are needed only for certain model
Exhaustive.tol
Default 1e-10. Applies only when pmethod="exhaustive". If the reciprocal of the condition number of bx is less than Exhaustive.tol, earth forces pmethod="backward". See
...
Dots are passed on to earth.fit.
Value

  • An object of class "earth" which is a list with the components listed below. Term refers to a term created during the forward pass (each line of the output from format.earth is a term). Term number 1 is always the intercept.
  • rssResidual sum-of-squares (RSS) of the model (summed over all responses, if y has multiple columns).
  • rsq1-rss/tss. R-Squared of the model (calculated over all responses, and calculated using the weights argument if it was supplied). A measure of how well the model fits the training data. Note that tss is the total sum-of-squares, sum((y - mean(y))^2).
  • gcvGeneralized Cross Validation (GCV) of the model (summed over all responses). The GCV is calculated using the penalty argument. For details of the GCV calculation, see equation 30 in Friedman's MARS paper and earth:::get.gcv.
  • grsq1-gcv/gcv.null. An estimate of the predictive power of the model (calculated over all responses, and calculated using the weights argument if it was supplied). gcv.null is the GCV of an intercept-only model. See Can GRSq be negative? in the vignette.
  • bxMatrix of basis functions applied to x. Each column corresponds to a selected term. Each row corresponds to a row in in the input matrix x, after taking subset. See model.matrix.earth for an example of bx handling. Example bx:(Intercept) h(Girth-12.9) h(12.9-Girth) h(Girth-12.9)*h(... [1,] 1 0.0 4.6 0 [2,] 1 0.0 4.3 0 [3,] 1 0.0 4.1 0 ...
  • dirsMatrix with one row per MARS term, and with with ij-th element equal to 0 if predictor j is not in term i -1 if an expression of the form h(const - xj) is in term i 1 if an expression of the form h(xj - const) is in term i 2 if predictor j should enter term i linearly (either because specified by the linpreds argument or because earth discovered that a knot was unnecessary). This matrix includes all terms generated by the forward pass, including those not in selected.terms. Note that here the terms may not all be in pairs, because although the forward pass add terms as hinged pairs (so both sides of the hinge are available as building blocks for further terms), it also deletes linearly dependent terms before handing control to the pruning pass. Example dirs:Girth Height (Intercept) 0 0 #intercept h(12.9-Girth) -1 0 #2nd term uses Girth h(Girth-12.9) 1 0 #3rd term uses Girth h(Girth-12.9)*h(Height-76) 1 1 #4th term uses Girth and Height ...
  • cutsMatrix with ij-th element equal to the cut point for predictor j in term i. This matrix includes all terms generated by the forward pass, including those not in selected.terms. Note for programmers: the precedent is to use dirs for term names etc. and to only use cuts where cut information needed. Example cuts:Girth Height (Intercept) 0 0 #intercept, no cuts h(12.9-Girth) 12.9 0 #2nd term has cut at 12.9 h(Girth-12.9) 12.9 0 #3rd term has cut at 12.9 h(Girth-12.9)*h(Height-76) 12.9 76 #4th term has two cuts ...
  • prune.termsA matrix specifying which terms appear in which pruning pass subsets. The row index of prune.terms is the model size. (The model size is the number of terms in the model. The intercept is counted as a term.) Each row is a vector of term numbers for the best model of that size. An element is 0 if the term is not in the model, thus prune.terms is a lower triangular matrix, with dimensions nprune x nprune. The model selected by the pruning pass is at row number length(selected.terms). Example prune.terms:[1,] 1 0 0 0 0 0 0 #intercept-only model [2,] 1 2 0 0 0 0 0 #best 2 term model uses terms 1,2 [3,] 1 2 4 0 0 0 0 #best 3 term model uses terms 1,2,4 [4,] 1 2 6 9 0 0 0 #and so on ...
  • selected.termsVector of term numbers in the selected model. Can be used as a row index vector into cuts and dirs. The first element selected.terms[1] is always 1, the intercept.
  • fitted.valuesFitted values. A matrix with dimensions nrow(y) x ncol(y) after factors in y have been expanded.
  • residualsResiduals. A matrix with dimensions nrow(y) x ncol(y) after factors in y have been expanded.
  • coefficientsRegression coefficients. A matrix with dimensions length(selected.terms) x ncol(y) after factors in y have been expanded. Each column holds the least squares coefficients from regressing that column of y on bx. The first row holds the intercept coefficient(s).
  • rss.per.responseA vector of the RSS for each response. Length is the number of responses, i.e., ncol(y) after factors in y have been expanded. The rss component above is equal to sum(rss.per.response).
  • rsq.per.responseA vector of the R-Squared for each response (where R-Squared is calculated using the weights argument if it was supplied). Length is the number of responses.
  • gcv.per.responseA vector of the GCV for each response. Length is the number of responses. The gcv component above is equal to sum(gcv.per.response).
  • grsq.per.responseA vector of the GRSq for each response (calculated using the weights argument if it was supplied). Length is the number of responses.
  • rss.per.subsetA vector of the RSS for each model subset generated by the pruning pass. Length is nprune. For multiple responses, the RSS is summed over all responses for each subset. The rss above is rss.per.subset[length(selected.terms)]. The RSS of an intercept only-model is rss.per.subset[1].
  • gcv.per.subsetA vector of the GCV for each model in prune.terms. Length is nprune. For multiple responses, the GCV is summed over all responses for each subset. The gcv above is gcv.per.subset[length(selected.terms)]. The GCV of an intercept-only model is gcv.per.subset[1].
  • leveragesDiagonal of the hat matrix (from the linear regression of the response on bx).
  • penalty,nk,threshCopies of the corresponding arguments to earth.
  • pmethod,npruneCopies of the corresponding arguments to earth.
  • weights,wpCopies of the corresponding arguments to earth.
  • termcondReason the forward pass terminated (an integer).
  • callThe call used to invoke earth.
  • termsModel frame terms. This component exists only if the model was built using earth.formula.
  • namesxColumn names of x, generated internally by earth when necessary so each column of x has a name. Used, for example, by predict.earth to name columns if necessary.
  • namesx.orgOriginal column names of x.
  • levelsLevels of y if y is a factor c(FALSE,TRUE) if y is logical Else NULL The following fields appear only if earth's argument keepxy is TRUE.
  • x,y,data,subsetCopies of the corresponding arguments to earth. Only exist if keepxy=TRUE. The following fields appear only if earth's glm argument is used.
  • glm.listList of GLM models. Each element is the value returned by earth's internal call to glm for each response. Thus if there is a single response (or a single binomial pair, see Binomial pairs in the vignette) this will be a one element list and you access the GLM model with earth.mod$glm.list[[1]].
  • glm.coefficientsGLM regression coefficients. Analogous to the coefficients field described above but for the GLM model(s). A matrix with dimensions length(selected.terms) x ncol(y) after factors in y have been expanded. Each column holds the coefficients from the GLM regression of that column of y on bx. This duplicates, for convenience, information buried in glm.list.
  • glm.bpairsNULL unless there are paired binomial columns. A logical vector, derived internally by earth, or a copy the bpairs specified by the user in the glm list. See Binomial pairs in the vignette. The following fields appear only if the nfold argument is greater than 1.
  • cv.listList of earth models, one model for each fold (ncross * nfold models). The fold models have two extra fields, icross (an integer from 1 to ncross) and ifold (an integer from 1 to nfold). To save memory, lengthy fields in the fold models are removed unless you use keepxy=TRUE. The lengthy fields are $bx, $fitted.values, and $residuals.
  • cv.ntermsVector of length ncross * nfold + 1. Number of MARS terms in the model generated at each cross-validation fold, with the final element being the mean of these.
  • cv.nvarsVector of length ncross * nfold + 1. Number of predictors in the model generated at each cross-validation fold, with the final element being the mean of these.
  • cv.groupsSpecifies which cases went into which folds. Matrix with two columns and number of rows equal to the the number of cases nrow(x) Elements of the first column specify the cross-validation number, 1:ncross. Elements of the second column specify the fold number, 1:nfold.
  • cv.rsq.tabMatrix with ncross * nfold + 1 rows and nresponse+1 columns, where nresponse is the number of responses, i.e., ncol(y) after factors in y have been expanded. The first nresponse elements of a row are the cv.rsq's on the out-of-fold data for each response of the model generated at that row's fold. (A cv.rsq is calculated from predictions on the out-of-fold data using the best model built from the in-fold data; where best means the model was selected using the in-fold GCV. The R-Squareds are calculated using the weights argument if it was supplied. The final column holds the row mean (a weighted mean if wp if specified)). The final row holds the column means. The values in this final row is the mean cv.rsq printed by summary.earth. Example for a single response model (where the mean column is redundant but included for uniformity with multiple response models): y mean fold1 0.909 0.909 fold2 0.869 0.869 fold3 0.952 0.952 fold4 0.157 0.157 fold5 0.961 0.961 mean 0.769 0.769 Example for a multiple response model: y1 y2 y3 mean fold1 0.915 0.951 0.944 0.937 fold2 0.962 0.970 0.970 0.968 fold3 0.914 0.940 0.942 0.932 fold4 0.907 0.929 0.925 0.920 fold5 0.947 0.987 0.979 0.971 mean 0.929 0.955 0.952 0.946
  • cv.class.rate.tabLike cv.rsq.tab but is the classification rate at each fold i.e. the fraction of classes correctly predicted. Models with discrete response only. Calculated with thresh=.5 for binary responses. For responses with more than two levels, the final row is the overall classification rate. The other rows are the classification rates for each level (the level versus not-the-level), which are usually higher than the overall classification rate (predicting the level versus not-the-level is easier than correctly predicting one of many levels). The weights argument is ignored for all cross-validation stats except R-Squareds.
  • cv.maxerr.tabLike cv.rsq.tab but is the MaxErr at each fold. This is the signed max absolute value at each fold. Results are aggregated for the final column and final row using the signed max absolute value. The signed max absolute value is defined as the maximum of the absolute difference between the predicted and observed response values, multiplied by -1 if the sign of that difference is negative.
  • cv.auc.tabLike cv.rsq.tab but is the AUC at each fold. Binomial models only.
  • cv.cor.tabLike cv.rsq.tab but is the cor at each fold. Poisson models only.
  • cv.deviance.tabLike cv.rsq.tab but is the MeanDev at each fold. Binomial models only.
  • cv.calib.int.tabLike cv.rsq.tab but is the CalibInt at each fold. Binomial models only.
  • cv.calib.slope.tabLike cv.rsq.tab but is the CalibSlope at each fold. Binomial models only.
  • cv.oof.rsq.tabGenerated only if keepxy=TRUE or pmethod="cv". A matrix with ncross * nfold + 1 rows and max.nterms columns, Each element holds an out-of-fold RSq (oof.rsq), calculated from predictions from the out-of-fold observations using the model built with the in-fold data. The final row is the mean over all folds. The R-Squareds are calculated using the weights argument if it was supplied.
  • cv.infold.rsq.tabGenerated only if keepxy=TRUE. Like cv.oof.rsq.tab but from predictions made on the in-fold observations.
  • cv.oof.fit.tabGenerated only if the varmod.method argument is used. Predicted values on the out-of-fold data. Dataframe with nrow(data) rows and ncross columns. The following field appears only if the varmod.method is specified.
  • varmodAn object of class "varmod". See the varmod help page for a description. Only appears if the varmod.method argument is used.

concept

  • regression
  • mars
  • Friedman

References

The primary references are the Friedman papers. Readers may find the MARS section in Hastie, Tibshirani, and Friedman a more accessible introduction. The Wikipedia article is recommended for an elementary introduction. Faraway takes a hands-on approach, using the ozone data to compare mda::mars with other techniques. (If you use Faraway's examples with earth instead of mars, use $bx instead of $x, and check out the book's errata.) Friedman and Silverman is recommended background reading for the MARS paper. Earth's pruning pass uses code from the leaps package which is based on techniques in Miller.

Faraway (2005) Extending the Linear Model with R http://www.maths.bath.ac.uk/~jjf23

Friedman (1991) Multivariate Adaptive Regression Splines (with discussion) Annals of Statistics 19/1, 1--141 https://statistics.stanford.edu/research/multivariate-adaptive-regression-splines Friedman (1993) Fast MARS Stanford University Department of Statistics, Technical Report 110 http://www.milbo.users.sonic.net/earth/Friedman-FastMars.pdf, https://statistics.stanford.edu/research/fast-mars

Friedman and Silverman (1989) Flexible Parsimonious Smoothing and Additive Modeling Technometrics, Vol. 31, No. 1. http://links.jstor.org/sici?sici=0040-1706%28198902%2931%3A1%3C3%3AFPSAAM%3E2.0.CO%3B2-Z

Hastie, Tibshirani, and Friedman (2009) The Elements of Statistical Learning (2nd ed.) http://www-stat.stanford.edu/~hastie/pub.htm

Leathwick, J.R., Rowe, D., Richardson, J., Elith, J., & Hastie, T. (2005) Using multivariate adaptive regression splines to predict the distributions of New Zealand's freshwater diadromous fish Freshwater Biology, 50, 2034-2052 http://www-stat.stanford.edu/~hastie/pub.htm, http://www.botany.unimelb.edu.au/envisci/about/staff/elith.html

Miller, Alan (1990, 2nd ed. 2002) Subset Selection in Regression http://wp.csiro.au/alanmiller/index.html

Wikipedia article on MARS http://en.wikipedia.org/wiki/Multivariate_adaptive_regression_splines

See Also

Start with summary.earth, plot.earth, evimp, and plotmo.

Please see the main package vignette ../doc/earth-notes.pdf{Notes on the earth package}. The vignette can also be downloaded from http://www.milbo.org/doc/earth-notes.pdf.

The vignette ../doc/earth-varmod.pdf{Variance models in earth} is also included with the package. It describes how to build variance models and generate prediction intervals for earth models.

Aliases
  • earth
  • earth.default
  • earth.formula
  • earth.fit
Examples
earth.mod <- earth(Volume ~ ., data = trees)
plotmo(earth.mod)
summary(earth.mod, digits = 2, style = "pmax")
Documentation reproduced from package earth, version 4.4.4, License: GPL-3

Community examples

Looks like there are no examples yet.