BAS (version 1.4.7)

cv.summary.bas: Summaries for Out of Sample Prediction

Description

Compute average prediction error from out of sample predictions

Usage

cv.summary.bas(pred, ytrue, score = "squared-error")

Arguments

pred

fitted or predicted value from the output from predict.bas

ytrue

vector of left out response values

score

function used to summarize error rate. Either "squared-error", "percent-explained", or "miss-class"

Value

For squared error, the average prediction error for the Bayesian estimator error = sqrt(sum(ytrue - yhat)^2/npred) while for binary data the missclassification rate is more appropriate. For continuous data the "percent-explained" reports ar, similar to an out of sample R2.

See Also

predict.bas

Examples

# NOT RUN {

# }
# NOT RUN {
library(foreign)
cognitive = read.dta("http://www.stat.columbia.edu/~gelman/arm/examples/child.iq/kidiq.dta")
cognitive$mom_work = as.numeric(cognitive$mom_work > 1)
cognitive$mom_hs =  as.numeric(cognitive$mom_hs > 0)
colnames(cognitive) = c("kid_score", "hs","iq", "work", "age")

set.seed(42)
n = nrow(cognitive)
test = sample(1:n, size=round(.20*n), replace=FALSE)
testdata =  cognitive[test,]
traindata = cognitive[-test,]
cog_train = bas.lm(kid_score ~ ., prior="BIC", modelprior=uniform(), data=traindata)
yhat = predict(cog_train, newdata=testdata, estimator="BMA", se=F)
cv.summary.bas(yhat$fit, testdata$kid_score)
# }