# Gram-Schmidt Orthogonalization and Regression

# same result as QR(X)$Q, but with signs reversed head(Z, 5) head(-QR(X)$Q, 5) all.equal( unname(Z), -QR(X)$Q ) ## Regression with X and Z We carry out two regressions of IQ on the variables in X and in Z. These are equivalent, in the sense that • The$R^2$and MSE are the same in both models • Residuals are the same • The Type I tests given by anova() are the same. class2 <- data.frame(Z, IQ=class$IQ)

Regression of IQ on the original variables in X

mod1 <- lm(IQ ~ height + weight + age + male, data=class) anova(mod1)

Regression of IQ on the orthogonalized variables in Z

mod2 <- lm(IQ ~ height + weight + age + male, data=class2) anova(mod2)

This illustrates that anova() tests for linear models are sequential tests. They test hypotheses about the extra contribution of each variable over and above all previous ones, in a given order. These usually do not make substantive sense, except in testing ordered ("hierarchical") models.