Display the effect of
a single variable (pdpgbart) or pair of variables (pd2pgbart).
Note that if response \(y\) is a binary with \(P(Y=1 | x) = F(f(x))\), \(F\) the standard normal cdf, then the plots are all on the \(f\) scale.
pdpgbart(
x.train, y.train,
xind=1:ncol(x.train), levs=NULL, levquants=c(.05,(1:9)/10,0.95),
pl=TRUE, plquants=c(.05,.95),
…)# S3 method for pdpgbart
plot(x,
xind = seq_len(length(x$fd)),
plquants = c(0.05, 0.95), cols = c('black', 'blue'),
…)
pd2pgbart(
x.train, y.train,
xind=1:2, levs=NULL, levquants=c(.05,(1:9)/10,.95),
pl=TRUE, plquants=c(.05,.95),
…)
# S3 method for pd2pgbart
plot(x,
plquants = c(0.05, 0.95), contour.color = 'white',
justmedian = TRUE,
…)
Explanatory variables for training (in sample) data. May be a matrix or a data frame, with (as usual) rows corresponding to observations and columns to variables. If a variable is a factor in a data frame, it is replaced with dummies. Note that q dummies are created if q>2 and one dummy is created if q=2, where q is the number of levels of the factor.
Dependent variable for training (in sample) data. If y is numeric a continous response model is fit (normal errors). If y is a factor (or just has values 0 and 1) then a binary response model with a probit link is fit.
Integer vector indicating which variables are to be plotted.
In pdpgbart, corresponds to the variables (columns of x.train) for which a plot is to be constructed.
In plotpdpgbart, corresponds to the indices in list returned by pdpgbart for which plot is to be constructed.
In pd2pgbart, the indicies of a pair of variables (columns of x.train) to plot.
Gives the values of a variable at which the plot is to be constructed.
Must be a list, where the \(i\)th component gives the values for the \(i\)th variable.
In pdpgbart, it should have same length as xind. In pd2pgbart, it should have length 2.
See also argument levquants.
If levs in NULL, the values of each variable used in the plot are
set to the quantiles (in x.train) indicated by levquants.
Must be a vector of numeric type.
For pdpgbart and pd2pgbart, if TRUE, plot is subsequently made (by calling plot.*).
In the plots, beliefs about \(f(x)\) are indicated by plotting the
posterior median and a lower and upper quantile.
plquants is a double vector of length two giving the lower and upper quantiles.
Additional arguments.
In pdbart and pd2bart, arguments are passed on to pgbart_train.
In plot.pdbart, they are passed on to plot.
In plot.pd2bart, they are passed on to image.
For plot.*, object is returned from pdpgbart or pd2pgbart.
Vector of two colors. The first color is for the median of \(f\), while the second color is for the upper and lower quantiles.
Color for contours plotted on top of the image.
A logical where if TRUE just one plot is created for
the median of \(f(x)\) draws. If FALSE, three plots are created
one for the median and two additional ones for the lower and upper quantiles.
In this case, mfrow is set to c(1,3).
The plot methods produce the plots and don't return anything.
pdpgbart and pd2pgbart return lists with components
given below. The list returned by pdpgbart is assigned class
pdpgbart and the list returned by pd2pgbart is assigned
class pd2pgbart.
A matrix whose \((i, j)\) value is the \(i\)th draw of \(f_s(x_s)\) for the \(j\)th value of \(x_s\). “fd” is for “function draws”.
For pdpgbart, fd is actually a list whose
\(k\)th component is the matrix described above
corresponding to the \(k\)th variable chosen by argument xind.
The number of columns in each matrix will equal the number of values
given in the corresponding component of argument levs (or number of values in levquants).
For pd2pgbart, fd is a single matrix.
The columns correspond to all possible pairs of values for the pair
of variables indicated by xind.
That is, all possible \((x_i, x_j)\) where \(x_i\) is a value in
the levs component corresponding to the first \(x\) and
\(x_j\) is a value in the levs components corresponding to the second one.
The first \(x\) changes first.
The list of levels used, each component corresponding to a variable.
If argument levs was supplied it is unchanged.
Otherwise, the levels in levs are as constructed using argument levquants.
A vector of character strings which are the plotting labels used for the variables.
The remaining components returned in the list are the same as in the value of pgbart_train. They are simply passed on from the pgbart run used to create the partial dependence plot.
We divide the predictor vector \(x\) into a subgroup of interest, \(x_s\) and the complement \(x_c = x \setminus x_s\). A prediction \(f(x)\) can then be written as \(f(x_s, x_c)\). To estimate the effect of \(x_s\) on the prediction, Friedman suggests the partial dependence function $$ f_s(x_s) = \frac{1}{n}\sum_{i=1}^n f(x_s,x_{ic}) $$ where \(x_{ic}\) is the \(i\)th observation of \(x_c\) in the data. Note that \((x_s, x_{ic})\) will generally not be one of the observed data points. Using pgbart it is straightforward to then estimate and even obtain uncertainty bounds for \(f_s(x_s)\). A draw of \(f^*_s(x_s)\) from the induced pgbart posterior on \(f_s(x_s)\) is obtained by simply computing \(f^*_s(x_s)\) as a byproduct of each MCMC draw \(f^*\). The median (or average) of these MCMC draws \(f^*_s(x_s)\) then yields an estimate of \(f_s(x_s)\), and lower and upper quantiles can be used to obtain intervals for \(f_s(x_s)\).
In pdpgbart \(x_s\) consists of a single variable in \(x\) and in
pd2pgbart it is a pair of variables.
This is a computationally intensive procedure.
For example, in pdbart, to compute the partial dependence plot
for 5 \(x_s\) values, we need
to compute \(f(x_s, x_c)\) for all possible \((x_s, x_{ic})\) and there
would be \(5n\) of these where \(n\) is the sample size.
All of that computation would be done for each kept pgbart draw.
For this reason running pgbart with keepevery larger than 1 (eg. 10)
makes the procedure much faster.
Lakshminarayanan B, Roy D, Teh Y W. (2015) Particle Gibbs for Bayesian Additive Regression Trees Artificial Intelligence and Statistics, 553-561.
Chipman, H., George, E., and McCulloch R. (2010) Bayesian Additive Regression Trees. The Annals of Applied Statistics, 4,1, 266-298.
Friedman, J. H. (2001) Greedy Function Approximation: A Gradient Boosting Machine. The Annals of Statistics, 29, 1189--1232.
# NOT RUN {
## simulate data
f <- function(x) { return(0.5 * x[,1] + 2 * x[,2] * x[,3]) }
sigma <- 0.2
n <- 100
set.seed(27)
x <- matrix(2 * runif(n * 3) -1, ncol = 3)
colnames(x) <- c('rob', 'hugh', 'ed')
Ey <- f(x)
y <- rnorm(n, Ey, sigma)
## first two plot regions are for pdbart, third for pd2bart
par(mfrow = c(1, 3))
## pdbart: one dimensional partial dependence plot
set.seed(99)
pdb1 <-
pdpgbart(
x, y, xind=c(1,2),
levs=list(seq(-1,1,.2), seq(-1,1,.2)), pl=FALSE,
keepevery=10, ntree=5, nskip=100, ndpost=200
)
plot(pdb1,ylim=c(-.6,.6))
## pd2bart: two dimensional partial dependence plot
set.seed(99)
pdb2 <-
pd2pgbart(x, y, xind = c(2, 3),
levquants = c(0.05, 0.1, 0.25, 0.5, 0.75, 0.9, 0.95),
pl = FALSE, ntree = 5, keepevery = 10, verbose = FALSE
)
plot(pdb2)
# }
Run the code above in your browser using DataCamp Workspace