The scoringfunctions package implements consistent scoring (loss) functions and identification functions.
bregman1_sf: Bregman scoring function (type 1)
bregman2_sf: Bregman scoring function (type 2, Patton scoring
function)
bregman3_sf: Bregman scoring function (type 3, QLIKE scoring
function)
bregman4_sf: Bregman scoring function (type 4, Patton scoring
function)
serr_sf: Squared error scoring function
expectile_sf: Asymmetric piecewise quadratic scoring function
(expectile scoring function, expectile loss function)
aerr_sf: Absolute error scoring function
maelog_sf: MAE-LOG scoring function
maesd_sf: MAE-SD scoring function
gpl1_sf: Generalized piecewise linear power scoring function
(type 1)
gpl2_sf: Generalized piecewise linear power scoring function
(type 2)
quantile_sf: Asymmetric piecewise linear scoring function
(quantile scoring function, quantile loss function)
ghuber_sf: Generalized Huber scoring function
huber_sf: Huber scoring function
aperr_sf: Absolute percentage error scoring function
bmedian_sf: \(\beta\)-median scoring function
linex_sf: LINEX scoring function
lqmean_sf: \(L_q\)-mean scoring function
lqquantile_sf: \(L_q\)-quantile scoring function
nmoment_sf: \(n\)-th moment scoring function
obsweighted_sf: Observation-weighted scoring function
powerweighted_sf: Power-weighted squared error scoring
function
relerr_sf: Relative error scoring function (MAE-PROP scoring
function)
serrexp_sf: Squared error exp scoring function
serrlog_sf: Squared error log scoring function
serrpower_sf: Squared error of power transformations scoring
function
serrsq_sf: Squared error of squares scoring function
sperr_sf: Squared percentage error scoring function
srelerr_sf: Squared relative error scoring function
interval_sf: Interval scoring function (Winkler scoring
function)
mv_sf: Mean - variance scoring function
errorspread_sf: Error - spread scoring function
bregman1_rs: Realised Bregman score (type 1)
bregman2_rs: Realised Bregman score (type 2, Patton score)
bregman4_rs: Realised Bregman score (type 4, Patton score)
mse: Mean squared error (MSE)
qlike: QLIKE
expectile_rs: Realised expectile score
mae: Mean absolute error (MAE)
maelog_rs: Realised MAE-LOG score
maesd_rs: Realised MAE-SD score
gpl1_rs: Realised generalized piecewise linear power score
(type 1)
gpl2_rs: Realised generalized piecewise linear power score
(type 2)
quantile_rs: Realised quantile score
ghuber_rs: Realised generalized Huber score
huber_rs: Realised Huber score
bmedian_rs: Realised \(\beta\)-median score
linex_rs: Realised LINEX score
lqmean_rs: Realised \(L_q\)-mean score
lqquantile_rs: Realised \(L_q\)-quantile score
mape: Mean absolute percentage error (MAPE)
mre: Mean relative error (MRE)
mspe: Mean squared percentage error (MSPE)
msre: Mean squared relative error (MSRE)
nmoment_rs: Realised \(n\)-th moment score
obsweighted_rs: Realised observation-weighted score
serrexp_rs: Realised squared error exp score
serrlog_rs: Realised squared error log score
serrpower_rs: Realised squared error of power
transformations score
serrsq_rs: Realised squared error of squares score
nse: Nash-Sutcliffe efficiency (NSE)
expectile_if: Expectile identification function
hubermean_if: Huber mean identification function
huberquantile_if: Huber quantile identification function
mean_if: Mean identification function
meanexp_if: Exp-transformed identification function
meanlog_if: Log-transformed identification function
meanpower_if: Power-transformed identification function
nmoment_if: \(n\)-th moment identification function
powerweighted_if: Power-weighted identification function
quantile_if: Quantile identification function
mv_if: Mean - variance identification function
quantile_level: Sample quantile level function
capping_function: Capping function
The table below lists a selection of predictive functionals alongside their
pointwise scoring functions (_sf) / realised average scores
(_rs), and identification functions (_if). The complete listing
of the package functions is given in the numbered sections below the table:
| Target Functional | Loss Functions (_sf / _rs) | Identification (_if) |
| Mean | serr_sf / mse,
bregman1_sf / bregman1_rs | mean_if |
bregman2_sf / bregman2_rs,
bregman3_sf / qlike | ||
bregman4_sf / bregman4_rs | ||
| Expectile (\(p\)) | expectile_sf / expectile_rs | expectile_if |
| Median | aerr_sf / mae,
maelog_sf / maelog_rs | quantile_if (\(p=0.5\)) |
maesd_sf / maesd_rs | ||
| \(\beta\)-Median | bmedian_sf / bmedian_rs | --- |
| Quantile (\(p\)) | quantile_sf / quantile_rs,
gpl1_sf / gpl1_rs | quantile_if |
gpl2_sf / gpl2_rs | ||
| Huber Mean | huber_sf / huber_rs | hubermean_if |
| Huber Quantile | ghuber_sf / ghuber_rs | huberquantile_if |
| \(L_q\)-Mean | lqmean_sf / lqmean_rs | --- |
| \(L_q\)-Quantile | lqquantile_sf / lqquantile_rs | --- |
| Interval (\(p\)) | interval_sf | --- |
| Mean - Variance | mv_sf | mv_if |
| Error - Spread | errorspread_sf | --- |
| Relative Error | relerr_sf / mre | --- |
| Percentage Error | aperr_sf / mape | --- |
The package functions are categorised into six classes, each of which has its own section below:
Scoring functions
Realised (average) score functions
Skill score functions
Identification functions
Functions for sample levels
Supporting functions
Banerjee A, Guo X, Wang H (2005) On the optimality of conditional expectation as a Bregman predictor. IEEE Transactions on Information Theory 51(7):2664--2669. tools:::Rd_expr_doi("10.1109/TIT.2005.850145").
Bellini F, Klar B, Muller A, Rosazza Gianin E (2014) Generalized quantiles as risk measures. Insurance: Mathematics and Economics 54:41--48. tools:::Rd_expr_doi("10.1016/j.insmatheco.2013.10.015").
Brehmer JR, Gneiting T (2021) Scoring interval forecasts: Equal-tailed, shortest, and modal interval. Bernoulli 27(3):1993--2010. tools:::Rd_expr_doi("10.3150/20-BEJ1298").
Chen Z (1996) Conditional \(L_p\)-quantiles and their application to the testing of symmetry in non-parametric regression. Statistics and Probability Letters 29(2):107--115. tools:::Rd_expr_doi("10.1016/0167-7152(95)00163-8").
Christensen HM, Moroz IM, Palmer TN (2015) Evaluation of ensemble forecast uncertainty using a new proper score: Application to medium-range and seasonal forecasts. Quarterly Journal of the Royal Meteorological Society 141(687)(Part B):538--549. tools:::Rd_expr_doi("10.1002/qj.2375").
Dimitriadis T, Fissler T, Ziegel JF (2024) Osband's principle for identification functions. Statistical Papers 65:1125--1132. tools:::Rd_expr_doi("10.1007/s00362-023-01428-x").
Dunsmore IR (1968) A Bayesian approach to calibration. Journal of the Royal Statistical Society, Series B (Methodological) 30(2):396--405. tools:::Rd_expr_doi("10.1111/j.2517-6161.1968.tb00740.x").
Ferguson TS (1967) Mathematical Statistics: A Decision-Theoretic Approach. Academic Press, New York.
Fissler T, Pesenti SM (2023) Sensitivity measures based on scoring functions. European Journal of Operational Research 307(3):1408--1423. tools:::Rd_expr_doi("10.1016/j.ejor.2022.10.002").
Fissler T, Ziegel JF (2016) Higher order elicitability and Osband's principle. The Annals of Statistics 44(4):1680--1707. tools:::Rd_expr_doi("10.1214/16-AOS1439").
Fissler T, Ziegel JF (2019) Order-sensitivity and equivariance of scoring functions. Electronic Journal of Statistics 13(1):1166--1211. tools:::Rd_expr_doi("10.1214/19-EJS1552").
Gerber HU (1974) On additive premium calculation principles. ASTIN Bulletin: The Journal of the IAA 7(3):215--222. tools:::Rd_expr_doi("10.1017/S0515036100006061").
Gneiting T (2011) Making and evaluating point forecasts. Journal of the American Statistical Association 106(494):746--762. tools:::Rd_expr_doi("10.1198/jasa.2011.r10138").
Gneiting T, Raftery AE (2007) Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association 102(477):359--378. tools:::Rd_expr_doi("10.1198/016214506000001437").
Houghton-Carr HA (1999) Assessment criteria for simple conceptual daily rainfall-runoff models. Hydrological Sciences Journal 44(2):237--261. tools:::Rd_expr_doi("10.1080/02626669909492220").
Huber PJ (1964) Robust estimation of a location parameter. Annals of Mathematical Statistics 35(1):73--101. tools:::Rd_expr_doi("10.1214/aoms/1177703732").
Koenker R, Bassett Jr G (1978) Regression quantiles. Econometrica 46(1):33--50. tools:::Rd_expr_doi("10.2307/1913643").
Nash JE, Sutcliffe JV (1970) River flow forecasting through conceptual models Part I - A discussion of principles. Journal of Hydrology 10(3):282--290. tools:::Rd_expr_doi("10.1016/0022-1694(70)90255-6").
Newey WK, Powell JL (1987) Asymmetric least squares estimation and testing. Econometrica 55(4):819--847. tools:::Rd_expr_doi("10.2307/1911031").
Park H, Stefanski LA (1998) Relative-error prediction. Statistics and Probability Letters 40(3):227--236. tools:::Rd_expr_doi("10.1016/S0167-7152(98)00088-1").
Patton AJ (2011) Volatility forecast comparison using imperfect volatility proxies. Journal of Econometrics 160(1):246--256. tools:::Rd_expr_doi("10.1016/j.jeconom.2010.03.034").
Raiffa H, Schlaifer R (1961) Applied Statistical Decision Theory. Colonial Press, Clinton.
Saerens M (2000) Building cost functions minimizing to some summary statistics. IEEE Transactions on Neural Networks 11(6):1263--1271. tools:::Rd_expr_doi("10.1109/72.883416").
Savage LJ (1971) Elicitation of personal probabilities and expectations. Journal of the American Statistical Association 66(336):783--801. tools:::Rd_expr_doi("10.1080/01621459.1971.10482346").
Taggart RJ (2022) Point forecasting and forecast evaluation with generalized Huber loss. Electronic Journal of Statistics 16(1):201--231. tools:::Rd_expr_doi("10.1214/21-EJS1957").
Thirel G, Santos L, Delaigue O, Perrin C (2024) On the use of streamflow transformations for hydrological model calibration. Hydrology and Earth System Sciences 28(21):4837--4860. tools:::Rd_expr_doi("10.5194/hess-28-4837-2024").
Thomson W (1979) Eliciting production possibilities from a well-informed manager. Journal of Economic Theory 20(3):360--380. tools:::Rd_expr_doi("10.1016/0022-0531(79)90042-5").
Tyralis H, Papacharalampous G (2026) Variable transformations in consistent loss functions. Knowledge-Based Systems 336:115202. tools:::Rd_expr_doi("10.1016/j.knosys.2025.115202").
Varian HR (1975) A Bayesian approach to real estate assessment. In: Fienberg SE, Zellner A (eds) Studies in Bayesian Econometrics and Statistics in Honor of Leonard J. Savage. Amsterdam: North-Holland, pp 195--208.
Winkler RL (1972) A decision-theoretic approach to interval estimation. Journal of the American Statistical Association 67(337):187--191. tools:::Rd_expr_doi("10.1080/01621459.1972.10481224").
Winkler RL, Murphy AH (1979) The use of probabilities in forecasts of maximum and minimum temperatures. Meteorological Magazine 108(1288):317--329.
Yeh C-C, Yeh H-W, Chan W (2008) Some equivalent forms of the arithematic-geometric mean inequality in probability: A survey. Journal of Inequalities and Applications 2008:386715. tools:::Rd_expr_doi("10.1155/2008/386715").
Zellner A (1986) Bayesian estimation and prediction using asymmetric loss functions. Journal of the American Statistical Association 81(394):446--451. tools:::Rd_expr_doi("10.1080/01621459.1986.10478289").