qr_lm: Ordinary least squares for the linear regression model

Description

qr_lm, or LS for linear regression models, solves the following optimization problem $$\textrm{min}_\beta ~ \frac{1}{2}\|y-X\beta\|_2^2,$$ for $y\in\mathbb{R}^n$ and $X\in\mathbb{R}^{n\times p}$ witn $n>p$, to obtain a coefficient vector $\widehat{\beta}\in\mathbb{R}^p$. The design matrix $X\in\mathbb{R}^{n\times p}$ contains the observations for each regressor.

Usage

qr_lm(y, X, X_test = NULL)

Value

A named list containing

coeff: a length-$p$ vector containing the solution for the parameters $\beta$.
coeff.se: a length-$p$ vector containing the standard errors for the estimated regression parameters $\beta$.
fitted: a length-$n$ vector of fitted values, $\widehat{y}=X\widehat{\beta}$.
residuals: a length-$n$ vector of residuals, $\varepsilon=y-\widehat{y}$.
residuals_norm2: the squared L2-norm of the residuals, $\Vert\varepsilon\Vert_2^2.$
y_norm2: the squared L2-norm of the response variable, $\Vert y\Vert_2^2.$
R: the $R\in\mathbb{R}^{p\times p}$ upper triangular matrix of the QR decomposition.
L: the inverse of the $R\in\mathbb{R}^{p\times p}$ upper triangular matrix of the QR decomposition $L = R^{-1}$.
XTX: the Gram matrix $X^\top X\in\mathbb{R}^{p\times p}$ of the least squares problem.
XTX_INV: the inverse of the Gram matrix $X^\top X\in\mathbb{R}^{p\times p}$ of the least squares problem $(X^\top X)^{-1}$.
XTy: A vector equal to $X^\top y$, the cross-product of the design matrix $X$ with the response vector $y$.
sigma2_hat: An estimate of the error variance $\sigma^2$, computed as the residual sum of squares divided by the residual degrees of freedom $\widehat{\sigma}^2 = \frac{\|y - X\hat{\beta}\|_2^2}{df}$
df: The residual degrees of freedom, given by $n - p$, where $n$ is the number of observations and $p$ is the number of estimated parameters.
R2: $R^2$, coefficient of determination, measure of goodness-of-fit of the model.
predicted: predicted values for the test set, $X_{\text{test}}\widehat{\beta}$. It is only available if X_test is not NULL.

Arguments

y: a vector of length-$n$ response vector.
X: an $(n\times p)$ full column rank matrix of predictors.
X_test: an $(q\times p)$ full column rank matrix. Test set. By default it set to NULL.

Examples

Run this code


## generate sample data
## create data: n > p
set.seed(1234)
n    <- 12
n0   <- 3
p    <- 7
X    <- matrix(rnorm(n * p), n, p)
b    <- rep(1, p)
sig2 <- 0.25
y    <- X %*% b + sqrt(sig2) * rnorm(n)
summary(lm(y~X))

## test
X_test <- matrix(rnorm(n0 * p), n0, p)

## lm
qr_lm(y = y, X = X, X_test = X_test)
qr_lm(y = y, X = X)

Run the code above in your browser using DataLab