The Brier Score was proposed by Glenn W. Brier in 1950 which is a proper score function that measures the accuracy of probabilistic predictions, usually used to measure the accuracy of a model fit for survival data. Brier can calculate the value of Brier Score at any timepoint, regardless of whether it is the event time.
Brier(object, pre_sp, t_star = -1)the Brier Score at time t_star between the true classes and the predicted probabilities.
object of class Surv created by Surv function
or a fitted survival model, including the survival model
fitted by coxph, survreg, and rfsrc.
If the input of the parameter object is a fitted survival model,
this parameter should be a survival dataset on which
you want to caculate the Brier Score of the fitted model.
Or with an object of class Surv as the parameter object,
it should be a vector of predicted values of survival probabilities
of each observation in testing set at time t_star.
the timepoint at which the Brier Score you want to calculate.
If the input of the parameter object is a fitted survival model, the timepoint is necessary to be specified
at which the survival probability is predicted,
and this function will calculate the Brier Score at that moment.
If the input object is a survival object, this parameter can be ignored
and the value of this parameter will not have any effect on the result of this fuction.
Hanpu Zhou zhouhanpu@csu.edu.cn
The Brier Score is the mean square difference between the true classes and the predicted probabilities. So the Brier Score can be thought of as a cost function. Therefore, the lower the Brier Score is for a set of predictions, the better the predictions are calibrated. The Brier Score takes on a value between zero and one, since this is the square of the largest possible difference between a predicted probability and the actual outcome. As we all know, for the cencoring samples, we do not know the real time of death, so the residual cannot be directly calculated when making the prediction. So the Brier Score is widely used in survival analysis.
The Brier Score is a strictly proper score (Gneiting and Raftery, 2007), which means that it takes its minimal value only when the predicted probabilities match the empirical probabilities.
Judging from the sparse empirical evidence, predictions of duration of survival tend to be rather inaccurate. More precision is achieved by using patient-specific survival probabilities and the Brier score as predictions to discriminate future survivors from failures.
Graf, Erika, Schmoor, Claudia, Sauerbrei, & Willi, et al. (1999). Assessment and comparison of prognostic classification schemes for survival data. Statist. Med., 18(1718), 2529-2545.
Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78.
Gneiting, T. , & Raftery, A. E. . (2007). Strictly Proper Scoring Rules, Prediction, and Estimation.
library(survival)
time <- rexp(50)
status <- sample(c(0, 1), 50, replace = TRUE)
pre_sp <- runif(50)
t_star <- runif(1)
Brier(Surv(time, status), pre_sp, t_star)
Run the code above in your browser using DataLab