Writes a PDF report of one object returned by propagate, or of a list of them (e.g. all examples of a demo), with about two pages per evaluation: measurement model, input data with distributions and Type A/B classification, result statement of the GUM uncertainty framework (GUF) and the Monte Carlo method (MCM), results of all methods, validation of the GUF by the MCM, numerical accuracy of the Monte Carlo results, nonlinearity of the model, uncertainty budget with first-order and Sobol' indices, histogram, Q-Q plot and convergence trace of the Monte Carlo output, an optional conformity assessment, derivatives, covariance check and a reproducibility record. If gauge (reference) values are supplied, the report adds a comparison with them.
report(object, file = tempfile(fileext = ".pdf"), title = "Uncertainty evaluation",
measurand = NULL, unit = "", compare = TRUE, nsim.compare = 1E5,
fitDistr = FALSE, fit.nmax = 1E4, gauge.mean = NULL, gauge.u = NULL,
gauge.method = c("GUF", "MC"), gauge.dec = NULL, gauge.M = 1E6,
interval = c("symmetric", "shortest"), ndig = 2, sobol = TRUE,
conformity = NULL, seed = NULL, fontsize = 1)Invisibly, for one object a list with the elements file, results (the table of results), validation (validation of the GUF, or NULL), accuracy (numerical accuracy of the Monte Carlo results, or NULL), budget (the uncertainty budget), sobol (the result of sobol, or NULL), nonlinearity (the five measures of nonlinearity with their ratings), derivatives (symbolic and evaluated sensitivity coefficients), shape (skewness and excess kurtosis), checks (against the gauge values, or NULL), conformity (the result of conformity, or NULL), fit (the distribution fit, or NULL) and reproducibility (the lines of the reproducibility record). For a list of objects, a list with file and analyses, one such list (without file) per evaluation.
an object returned by propagate, or a list of such objects, one per evaluation. The names of the list are used as page titles if title is not given.
the path and name of the PDF file, e.g. 'c:/temp/test.pdf.
the title in the page header.
symbol and unit of the output quantity (default symbol: the name of the expression, else "y"); an underscore gives a subscript, units such as "Ohm" or "degC" are typeset with their symbols.
if TRUE and the object has Type A inputs (type = "stat" or "raw"), the Monte Carlo simulation is repeated with the other input model ("S1" or "mvt") with nsim.compare trials, if applicable.
if fitDistr = TRUE, all distributions of fitDistr are fitted to the Monte Carlo output (at most fit.nmax values) and the best one by BIC is drawn on the histogram.
gauge (reference) values of the estimate and the standard uncertainty, e.g. published or certified values. If given, the report compares them with the result of gauge.method and adds the section 'Comparison with the gauge values'; the histogram shows the gauge value \(\pm k u\).
"GUF": comparison with the first-order GUM uncertainty framework; "MC": with the Monte Carlo method.
the numbers of decimals of gauge.mean and gauge.u, which define the rounding tolerance; by default taken from the values (set it when trailing zeros matter, e.g. 64.0).
the number of Monte Carlo trials of the gauge values (gauge.method = "MC").
the Monte Carlo coverage interval shown and used for the validation and the numerical accuracy: probabilistically symmetric or shortest.
the number of significant digits of the standard uncertainty \(u(y)\) that defines the numerical tolerance \(\delta\) of the validation, as in summary.propagate.
logical. If TRUE (default) and the input quantities are independent, the first-order Sobol' indices of the Monte Carlo output are computed with sobol and shown in the uncertainty budget.
NULL (default) or a list with the elements lower, upper (tolerance limits) and optionally rule (decision rules, default c("stringent", "simple"), "all" for all six), method ("MC", default, or "GUF") and p (coverage probability of the expanded uncertainty, default 1 - alpha), passed to conformity. If given, page 2 shows the conformity assessment. For a list of objects, a list of such lists, one per evaluation (NA or NULL: none).
the seed of the random number generator used before propagate (e.g. set.seed(123)), for the reproducibility record. The propagate object does not contain the seed, so it must be given here; NULL: 'not recorded'.
a multiplier for all font sizes.
Andrej-Nikolai Spiess
Contents. Each evaluation starts on a new page; the sections follow in this order and move to the next page as a whole if they do not fit, so that an evaluation usually takes two pages (models with many input quantities or many decision rules may need a third page).
Measurement model and input data: raw observations for type = "raw" with at most 15 rows; for each input quantity the estimate, standard uncertainty, degrees of freedom, the distribution and the type of evaluation (see below); the correlation matrix of correlated inputs.
Result statement of the GUF and the MCM, rounded to two significant digits of the standard uncertainty (JCGM 100:2008, 7.2.6), and the results of all methods (GUF first and second order, the latter also with the input moments, u.2m (moments extension, see propagate), Monte Carlo with the input models "S1" and "mvt"; GUF first order and "S1" in dark red).
Validation of the GUF and numerical accuracy of the Monte Carlo results (see below).
Nonlinearity of the model and shape of the output (see below), and the comparison with gauge values, if given.
Uncertainty budget: estimates, standard uncertainties, degrees of freedom, sensitivity coefficients \(c_i\), contributions \(|c_i| u(x_i)\), the first-order index \(c_i^2 u^2(x_i)/u^2(y)\) (blue bar), the share of the covariance terms (negative: dark red) and the first-order Sobol' index \(S_i\) of the Monte Carlo output (violet bar), with \(1 - \sum S_i\) as 'interactions'. Bars are scaled to the largest absolute share if one exceeds 100 %.
Histogram of the Monte Carlo output with the Monte Carlo and GUF coverage intervals (and the fitted distribution and the gauge band, if requested); a Q-Q plot of the quantiles of the Monte Carlo output against those of the GUF distribution (scaled and shifted t with \(\nu_{\mathrm{eff}}\) degrees of freedom, or Gaussian), with the endpoints of the coverage intervals marked: points off the diagonal show where the GUF distribution, and with it the coverage factor \(k\), does not describe the output; and a convergence trace of the running mean and standard deviation over the trials, relative to the final values in units of \(u\).
Conformity assessment (optional, argument conformity): tolerance interval, conformance probability \(p_c\) (JCGM 106:2012, 7) and, for each decision rule, a bar with the acceptance (green), uncertainty (grey) and rejection (red) zones, the estimate and the decision (ISO 14253-1, see conformity).
Sensitivity coefficients and Hessian matrix: \(c_i = \partial f/\partial x_i\), symbolic and evaluated, with the evaluated Hessian matrix as additional columns (for at most six input quantities), and the symbolic Hessian (lower triangle) for at most three input quantities (otherwise see summary.propagate); long expressions are shortened with '...'.
Covariance check of the Monte Carlo input samples, as in summary.propagate, and, for Type A inputs, a footnote on the Monte Carlo input models ("S1": t-margins, correlated inputs joined by a Gaussian copula, Possolo 2010; "mvt": multivariate t-distribution).
Reproducibility record (see below).
Distribution and type of evaluation. If the input data carry an uncertainty budget (from makeDat with output = "sim", or corDist), the distributions are taken from it; an input with a t-distribution and finite degrees of freedom is Type A, all others Type B. Otherwise, for type = "raw" all inputs are Type A (t or mvt); for type = "stat", inputs with finite degrees of freedom (\(< 1000\)) are Type A, the others Type B (Gaussian); for type = "sim" without a budget, the distribution is 'sample'. The budget is found in object$input$budget or as the attribute "budget" of object$input$data.
Validation of the GUF by the MCM (JCGM 101:2008, 8). As in summary.propagate: the numerical tolerance \(\delta = \frac{1}{2} 10^l\) for \(u(y) = c \times 10^l\) with ndig significant digits in \(c\), the differences \(d_{\mathrm{low}}\) and \(d_{\mathrm{high}}\) between the endpoints of the GUF and the Monte Carlo coverage intervals, and twice the numerical standard uncertainty \(s\) of the Monte Carlo endpoints (sim.se of the object). The GUF is validated if \(d_{\mathrm{low}}, d_{\mathrm{high}} \le \delta\); the comparison is inconclusive if \(2s > \delta\). In addition, the report gives the Monte Carlo coverage of the GUF interval, the fraction of the Monte Carlo output within the GUF coverage interval, next to the nominal coverage probability: for a nonlinear model it can be much smaller, e.g. about 33 % instead of 95 % for \(Y = X_1^2 + X_2^2\) with \(x_1 = 0.001\), \(x_2 = 0\), \(u(x_1) = u(x_2) = 0.005\) (JCGM 101:2008, 9.4).
Numerical accuracy of the Monte Carlo results (JCGM 101:2008, 7.9). For the estimate, the standard uncertainty and the endpoints of the coverage interval, with the numerical standard uncertainties \(s\) from 10 blocks of the Monte Carlo trials (sim.se): the number of decimals \(d\) for which \(2s \le \frac{1}{2} 10^{-d}\), the value rounded to \(d\) decimals ('stable value') and its number of stable significant digits. Digits beyond these are Monte Carlo noise; increase nsim or use adaptive = TRUE in propagate for more digits.
Nonlinearity of the model and shape of the output. Five measures, each zero for a linear model with Gaussian inputs: the share of the second-order terms in the second-order variance with the input moments, \((u_{2m}^2 - u_1^2)/u_{2m}^2\) (moments extension; for Gaussian inputs \(\frac{1}{2}\mathrm{tr}(\mathbf{H}\mathbf{\Sigma}\mathbf{H}\mathbf{\Sigma})/u_2^2\), JCGM 100:2008, 5.1.2, note); the shift of the estimate \((\bar{y}_2 - \bar{y}_1)/u_1\); the quantile skewness \((q_{0.975} + q_{0.025} - 2 q_{0.5})/(q_{0.975} - q_{0.025})\), which exists also for heavy-tailed output; and the skewness and excess kurtosis (moments) of the Monte Carlo output. They are rated 'negligible', 'moderate' or 'large' (limits 1 % and 10 %, 0.05 and 0.2, 0.05 and 0.2, 0.1 and 0.5, 0.2 and 1); skewness and kurtosis are not rated for inputs with \(\nu \le 4\), whose moments may not exist. Large values indicate that the first-order GUF may be inadequate and the Monte Carlo results should be preferred (JCGM 101:2008, 5.8). Note that the excess kurtosis also reflects t-distributed inputs (e.g. \(6/(\nu - 4)\) for a linear model with one t-input), not only nonlinearity.
Sobol' indices. The first-order index \(S_i\) is the share of the variance of the Monte Carlo output due to \(x_i\) alone, including all nonlinear effects; unlike the first-order index of the GUF it does not vanish for \(c_i = 0\). It is defined only for independent input quantities and is therefore computed with sobol from the Monte Carlo input samples only for them (not for correlated inputs or mc.margins = "mvt").
Reproducibility record (ISO/IEC 17025:2017, 7.5, 7.8). The call of propagate (if stored in the object as call, otherwise reconstructed from the object and marked as such), the number of Monte Carlo trials, alpha, the input model, cov and the coverage interval, the seed (argument seed), the kind of random number generator of the session, the versions of propagate and R, the platform and the date.
Several evaluations. For a list of objects, title, measurand, unit, gauge.mean, gauge.u, gauge.dec, seed and conformity may be vectors or lists with one element per evaluation (recycled); NA means 'not given' for that evaluation, e.g. gauge.mean = c(-0.1494, NA). For gauge.dec, which has two values per evaluation, use a list, e.g. list(c(4, 4), NA); for conformity, a list of lists, e.g. list(list(lower = 994, upper = 1006), NA). All other arguments apply to all evaluations.
Comparison with gauge values. The tolerance is half a unit of the last given decimal of the gauge value; for gauge.method = "MC", four Monte Carlo standard errors of both calculations are added (from sim.se of the object and gauge.M). A check passes if \(|value - gauge| \le\) tolerance.
Evaluation of measurement data - Guide to the expression of uncertainty in measurement.
JCGM 100:2008. Joint Committee for Guides in Metrology (2008).
tools:::Rd_expr_doi("10.59161/JCGM100-2008E")
Evaluation of measurement data - Supplement 1 to the "Guide to the expression of uncertainty in measurement" - Propagation of distributions using a Monte Carlo method.
JCGM 101:2008. Joint Committee for Guides in Metrology (2008).
tools:::Rd_expr_doi("10.59161/JCGM101-2008")
Evaluation of measurement data - Supplement 2 to the "Guide to the expression of uncertainty in measurement" - Extension to any number of output quantities.
JCGM 102:2011. Joint Committee for Guides in Metrology (2011).
tools:::Rd_expr_doi("10.59161/JCGM102-2011")
Evaluation of measurement data - The role of measurement uncertainty in conformity assessment.
JCGM 106:2012. Joint Committee for Guides in Metrology (2012).
tools:::Rd_expr_doi("10.59161/JCGM106-2012")
Geometrical product specifications (GPS) - Inspection by measurement of workpieces and measuring equipment - Part 1: Decision rules for verifying conformity or nonconformity with specifications.
ISO 14253-1:2017. International Organization for Standardization (2017).
General requirements for the competence of testing and calibration laboratories.
ISO/IEC 17025:2017. International Organization for Standardization (2017).
Copulas for uncertainty analysis.
Possolo A.
Metrologia (2010), 47: 262-271.
tools:::Rd_expr_doi("10.1088/0026-1394/47/3/017")
propagate, summary.propagate, sobol, conformity, corDist, makeDat, fitDistr
# \donttest{
## GUM H.3: calibration of a thermometer, compared with the published values.
## The PDF is written to 'file', e.g. file = file.path(tempdir(), "report.pdf").
data(H.3); fit <- lm(bk ~ I(tk - 20), data = H.3)
V12 <- vcov(fit); dimnames(V12) <- list(c("y1", "y2"), c("y1", "y2"))
set.seed(123)
H3 <- propagate(expression(y1 + y2 * (30 - 20)),
cbind(y1 = c(coef(fit)[1], sqrt(V12[1, 1]), 9),
y2 = c(coef(fit)[2], sqrt(V12[2, 2]), 9)),
type = "stat", cov = V12, ws.group = c(1, 1),
mc.margins = "mvt", nsim = 1E5)
f <- tempfile(fileext = ".pdf")
RES <- report(H3, file = f, title = "GUM H.3: calibration of a thermometer",
measurand = "b", unit = "degC", seed = 123,
gauge.mean = -0.1494, gauge.u = 0.0041)
RES$validation ## INCONCLUSIVE at nsim = 1E5 (2s > delta): increase nsim
RES$accuracy
# }
Run the code above in your browser using DataLab