propagate (version 1.0-4)

propagate: Propagation of uncertainty using higher-order Taylor expansion and Monte Carlo simulation

Description

A general function for the calculation of uncertainty propagation by first-/second-order Taylor expansion and Monte Carlo simulation including covariances. Input data can be any symbolic/numeric differentiable expression and data based on replicates, summaries (mean & s.d.) or sampled from a distribution. Uncertainty propagation is based completely on matrix calculus accounting for full covariance structure. Monte Carlo simulation is conducted using multivariate normal or t-distributions with covariance structure.

Usage

propagate(expr, data, type = c("stat", "raw", "sim"), second.order = TRUE,
          do.sim = TRUE, dist.sim = c("norm", "t"), df.t = NULL, 
          use.cov = TRUE, nsim = 100000, alpha = 0.05, ...)

Arguments

expr

an expression, such as expression(x/y).

data

a dataframe or matrix containing either a) the replicates in columns when using type = "raw", b) the means in first and standard deviations in second rows when using type = "stat" or c) sampled data generated from any of R's distributions or those implemented in this package (rDistr), if type = "sim". Column names must match the variable names.

type

either "stat" if means and standard deviations are supplied, "raw" if raw replicates are given or "sim" in case of simulated data.

second.order

logical. If TRUE, error propagation will be calculated with first- and second-order Taylor expansion. See 'Details'.

do.sim

logical. Should Monte Carlo simulation be applied?

dist.sim

"norm" will use a multivariate normal distribution for Monte Carlo simulation, "t" a multivariate t-distribution. See 'Details'.

df.t

degrees of freedom when using dist.sim = "t" and no raw observations are supplied (i.e. type = "stat" or type = "sim"). See 'Details'.

use.cov

logical or variance-covariance matrix with the same column descriptions as data. See 'Details'.

nsim

the number of Monte Carlo simulations to be performed, minimum is 10000.

alpha

the 1 - confidence level.

...

other parameters to be supplied to future methods.

Value

A list with the following components:

datSIM

a vector containing the nsim simulated multivariate values for each variable in column format.

resSIM

a vector containing the nsim values obtained from the row-wise expression evaluations \(\boldsymbol{ f(x_{m, i})}\) of the simulated data in datSIM.

datPROP

nsim values generated from a normal distribution with \(\mu\) and \(\sigma\) as calculated from the propagated error.

gradient

the symbolic gradient vector \(\nabla_x\) of partial first-order derivatives.

evalGrad

the evaluated gradient vector \(\nabla_x\) of partial first-order derivatives.

covMat

the covariance matrix \(\mathbf{C}_x\) used for Monte Carlo simulation and error propagation.

hessian

the symbolic hessian matrix \(\mathbf{H}_{xx}\) of partial second-order derivatives.

evalHess

the evaluated hessian matrix \(\mathbf{H}_{xx}\) of partial second-order derivatives.

prop

a summary vector containing first-/second-order expectations and uncertainties as well as the confidence interval based on alpha from datPROP.

sim

a summary vector containing the mean, standard deviation, median, MAD as well as the confidence interval based on alpha from datSIM.

Details

The implemented methods are: 1) Monte Carlo simulation: For each variable \(\boldsymbol{m}\) in data, simulated data \(\boldsymbol{x = [X_1, X_2, \ldots, X_n]}\) with \(\boldsymbol{n}\) = nsim samples is generated from a multivariate normal distribution \(\boldsymbol{x_{m, n} \sim \mathcal{N}(\mu, \Sigma)}\) or multivariate t-distribution \(\boldsymbol{x_{m, n} \sim t(\mu, \Sigma, \nu)}\) using means \(\boldsymbol{\mu_m}\) and covariance matrix \(\boldsymbol{\Sigma}\) constructed from the standard deviations \(\boldsymbol{\sigma_m}\) of each variable. All data is coerced into a new dataframe that has the same covariance structure as the initial data: \(\boldsymbol{\Sigma(\mathtt{data}) = \Sigma(x_{m, n})}\). Each row \(\boldsymbol{i = 1, \ldots, n}\) of the simulated dataset \(\boldsymbol{x_{m, n}}\) is evaluated with expr, \(\boldsymbol{y_i = f(x_{m, i})}\), and summary statistics (mean, sd, median, mad, confidence interval based on alpha) are calculated on \(\boldsymbol{y}\). If sample sizes for raw replicated data when setting type = "raw" are small, simulation from a multivariate t-distribution is advocated (Possolo, 2010). If this is done, the resulting distribution can have very heavy tails resulting in less stringent estimates of \(\boldsymbol{\sigma_y}\) and larger confidence intervals. In this case, the degrees of freedom used are \(n-1\). However, when setting type = "stat" or type = "sim", the degrees of freedom have to be supplied by the user in df.t.

2) Error propagation: The propagated error is calculated by first-/second-order Taylor expansion accounting for full covariance structure using matrix algebra. The following transformations based on two variables \(x_1, x_2\) illustrate the equivalence of the matrix-based approach with well-known classical notations: First-order mean: \(\rm{E[y]} = f(\bar{x}_i)\) First-order variance: \(\sigma_y^2 = { \color{red} \nabla_x\mathbf{C}_x\nabla_x^T}\): $${ \color{red}[\rm{j_1}\; \rm{j_2}] \left[ \begin{array}{cc} \sigma_1^2 & \sigma_1\sigma_2 \\ \sigma_2\sigma_1 & \sigma_2^2 \end{array} \right] \left[ \begin{array}{c} \rm{j_1} \\ \rm{j_2} \end{array} \right]} = \rm{j_1}^2 \sigma_1^2 + \rm{2 j_1 j_2} \sigma_1 \sigma_2 + \rm{j_2}^2 \sigma_2^2$$ $$= \underbrace{\sum_{i=1}^2 \rm{j_i}^2 \sigma_i^2 + 2\sum_{i=1\atop i \neq k}^2\sum_{k=1\atop k \neq i}^2 \rm{j_i j_k} \sigma_{ik}}_{\rm{classical\;notation}} = \frac{1}{1!} \left(\sum_{i=1}^2 \frac{\partial f}{\partial x_i} \sigma_i \right)^2$$

Second-order mean: \(\rm{E}[y] = f(\bar{x}_i) + {\color{blue} \frac{1}{2}\rm{tr}(\mathbf{H}_{xx}\mathbf{C}_x)}\): $${ \color{blue} \frac{1}{2} \rm{tr} \left[ \begin{array}{cc} \rm{h_1} & \rm{h_2} \\ \rm{h_3} & \rm{h_4} \end{array} \right] \left[ \begin{array}{cc} \sigma_1^2 & \sigma_1\sigma_2 \\ \sigma_2\sigma_1 & \sigma_2^2 \end{array} \right]} = \frac{1}{2} \rm{tr} \left[ \begin{array}{cc} \rm{h_1} \sigma_1^2 + \rm{h_2}\sigma_1\sigma_2 & \rm{h_1}\sigma_1\sigma_2 + \rm{h_2}\sigma_2^2 \\ \rm{h_3} \sigma_1^2 + \rm{h_4} \sigma_1\sigma_2 & \rm{h_3} \sigma_1\sigma_2 + \rm{h_4} \sigma_2^2 \end{array} \right]$$ $$ = \frac{1}{2}(\rm{h_1}\sigma_1^2 + \rm{h_2}\sigma_1\sigma_2 + \rm{h_3}\sigma_1\sigma_2 + \rm{h_4}\sigma_2^2) = \frac{1}{2!} \left(\sum_{i=1}^2 \frac{\partial}{\partial x_i} \sigma_i \right)^2 \it f$$

Second-order variance: \(\sigma_y^2 = {\color{red} \nabla_x\mathbf{C}_x\nabla_x^T} + {\color{blue} \frac{1}{2}\rm{tr}(\mathbf{H}_{xx}\mathbf{C}_x\mathbf{H}_{xx}\mathbf{C}_x)]}\): $${\color{blue}\frac{1}{2} \rm{tr} \left[ \begin{array}{cc} \rm{h_1} & \rm{h_2} \\ \rm{h_3} & \rm{h_4} \end{array} \right] \left[ \begin{array}{cc} \rm{\sigma_1^2} & \rm{\sigma_1\sigma_2} \\ \rm{\sigma_2\sigma_1} & \rm{\sigma_2^2} \end{array} \right] \left[ \begin{array}{cc} \rm{h_1} & \rm{h_2} \\ \rm{h_3} & \rm{h_4} \end{array} \right] \left[ \begin{array}{cc} \rm{\sigma_1^2} & \rm{\sigma_1\sigma_2} \\ \rm{\sigma_2\sigma_1} & \rm{\sigma_2^2} \end{array} \right]} = \ldots$$ $$= \frac{1}{2} (\rm{h_1}^2\sigma_1^4 + \rm{2h_1h_2}\sigma_1^3\sigma_2 + \rm{2h_1h_3}\sigma_1^3\sigma_2 + \rm{h_2}^2\sigma_1^2\sigma_2^2 + \rm{2h_2h_3}\sigma_1^2\sigma_2^2 + \rm{h_3}^2\sigma_1^2\sigma_2^2 + \rm{2h_1h_4}\sigma_1^2\sigma_2^2$$ $$+ \rm{2h_2h_4}\sigma_1\sigma_2^3 + \rm{2h_3h_4}\sigma_1\sigma_2^3 + \rm{h_4}^2\sigma_2^4 = \frac{1}{2} (\rm{h_1}\sigma_1^2 + \rm{h_2}\sigma_1\sigma_2 + \rm{h_3}\sigma_1\sigma_2 + \rm{h_4}\sigma_2^2)^2$$ $$= \frac{1}{2!} \left( \left(\sum_{i=1}^2 \frac{\partial}{\partial x_i} \sigma_i \right)^2 \it f \right)^2$$

with \(\mathrm{E}(y)\) = expectation of \(y\), \(\mathbf{\sigma_y^2}\) = variance of \(y\), \({\color{red} \nabla_x}\) = the p x n gradient matrix with all partial first derivatives \({\color{red} \rm{j_i}}\), \(\mathbf{C}_x\) = the p x p covariance matrix, \({\color{blue}\mathbf{H}_{xx}}\) the Hessian matrix with all partial second derivatives \({\color{blue} \rm{h_i}}\), \(\sigma_i\) = the uncertainties and \(\rm{tr}(\cdot)\) = the trace (sum of diagonal) of a matrix. Note that because the hessian matrices are symmetric matrices, \({\color{blue} \rm{h_2}} = {\color{blue} \rm{h_3}}\). For a detailed derivation, see 'References'. The second-order Taylor expansion corrects for bias in nonlinear expressions as the first-order Taylor expansion assumes linearity around \(\bar{x}_i\). There is also a Python library available for second-order error propagation ('soerp', https://pypi.python.org/pypi/soerp). The 'propagate' package gives exactly the same results, see last example under "Examples". Depending on the input expression, the error propagation may result in an error that is not normally distributed. The Monte Carlo simulation, starting with normal distributions of the variables, can clarify this. For instance, a high tendency from deviation of normality is encountered in formulas in which the error of the denominator is relatively large or in exponential models with a large error in the exponent.

For setups in which there is no symbolic derivation possible (i.e. e <- expression(abs(x)) => "Function 'abs' is not in the derivatives table") the function automatically switches from symbolic (using makeGrad or makeHess) to numeric (numGrad or numHess) differentiation.

The function will try to evaluate the expression in an environment using eval which results in a significant speed enhancement (~ 10-fold). If that fails, evaluation is done over the rows of the simulated data using apply.

References

Error propagation (in general): An Introduction to error analysis. Taylor JR. University Science Books (1996), New York.

Evaluation of measurement data - Guide to the expression of uncertainty in measurement. JCGM 100:2008 (GUM 1995 with minor corrections). http://www.bipm.org/utils/common/documents/jcgm/JCGM_100_2008_E.pdf.

Evaluation of measurement data - Supplement 1 to the Guide to the expression of uncertainty in measurement - Propagation of distributions using a Monte Carlo Method. JCGM 101:2008. http://www.bipm.org/utils/common/documents/jcgm/JCGM_101_2008_E.pdf.

Higher-order Taylor expansion: On higher-order corrections for propagating uncertainties. Wang CM & Iyer HK. Metrologia (2005), 42: 406-410.

Propagation of uncertainty: Expressions of second and third order uncertainty with third and fourth moments. Mekid S & Vaja D. Measurement (2008), 41: 600-609.

Matrix algebra for error propagation: An Introduction to Error Propagation: Derivation, Meaning and Examples of Equation Cy = FxCxFx^t. www.nada.kth.se/~kai-a/papers/arrasTR-9801-R3.pdf.

Second order nonlinear uncertainty modeling in strapdown integration using MEMS IMUs. Zhang M, Hol JD, Slot L, Luinge H. 2011 Proceedings of the 14th International Conference on Information Fusion (FUSION) (2011).

Uncertainty propagation in non-linear measurement equations. Mana G & Pennecchi F. Metrologia (2007), 44: 246-251.

A compact tensor algebra expression of the law of propagation of uncertainty. Bouchot C, Quilantan JLC, Ochoa JCS. Metrologia (2011), 48: L22-L28.

Nonlinear error propagation law. Kubacek L. Appl Math (1996), 41: 329-345.

Monte Carlo simulation (normal- and t-distribution): MUSE: computational aspects of a GUM supplement 1 implementation. Mueller M, Wolf M, Roesslein M. Metrologia (2008), 45: 586-594.

Copulas for uncertainty analysis. Possolo A. Metrologia (2010), 47: 262-271.

Multivariate normal distribution: Stochastic Simulation. Ripley BD. Stochastic Simulation (1987). Wiley. Page 98.

Testing for normal distribution: Testing for Normality. Thode Jr. HC. Marcel Dekker (2002), New York.

Approximating the Shapiro-Wilk W-test for non-normality. Royston P. Stat Comp (1992), 2: 117-119.

Examples

Run this code
# NOT RUN {
## From summary data: 
EXPR1 <- expression(x/y)
x <- c(5, 0.01)
y <- c(1, 0.01)
DF1 <- cbind(x, y)
RES1 <- propagate(expr = EXPR1, data = DF1, type = "stat", 
                  do.sim = TRUE, verbose = TRUE)
RES1$prop
RES1$sim

## From raw data:
EXPR2 <- expression(x/y)
x <- c(2, 2.1, 2.2, 2, 2.3, 2.1)
y <- c(4, 4, 3.8, 4.1, 3.1, NA)
DF2 <- cbind(x, y)  
RES2 <- propagate(expr = EXPR2, data = DF2, type = "raw",  
                  do.sim = TRUE, verbose = TRUE)
RES2$prop
RES2$sim

## Compare to using a multivariate t-distribution
## because of low sample size => larger confidence
## intervals.
RES2b <- propagate(expr = EXPR2, data = DF2, type = "raw",  
                  do.sim = TRUE, verbose = TRUE, dist.sim = "t")
RES2$sim
RES2b$sim

## Example using a recursive function:
## no Taylor expansion possible, only Monte-Carlo.
a <- c(5, 0.1)
b <- c(100, 2)
DAT <- cbind(a, b)

f <- function(a, b) {
  N <- 0
  for (i in 1:100) {
    N <- N + i * log(a) + b^(1/i)
  }
  return(N)
}

propagate(f, DAT, nsim = 100000)

################# GUM 2008 (1) ########################
## Example in Annex H.1 from the GUM 2008 manual
## (see 'References'), an end gauge calibration
## study. At first, we will only use the first-order 
## Taylor expansion.
EXPR3 <- expression(ls + d - ls * (da * the + as * dt))
ls <- c(50000623, 25)
d <- c(215, 9.7)
da <- c(0, 0.58E-6)
the <- c(-0.1, 0.41)
as <- c(11.5E-6, 1.2E-6)
dt <- c(0, 0.029)
DF3 <- cbind(ls, d, da, the, as, dt)
RES3 <- propagate(expr = EXPR3, data = DF3, type = "stat", 
                  do.sim = TRUE, verbose = TRUE, 
                  second.order = FALSE)
RES3$prop
RES3$sim
## propagate: sd.1 = 31.71 
## GUM H.1.4/H.6c: u = 32  

## Proof that covariance of Monte-Carlo
## simulated dataset is the same as from 
## initial data.
RES3$covMat
cov(RES3$datSIM)
all.equal(RES3$covMat, cov(RES3$datSIM))

## Expanded uncertainty GUM H.1.6
## with 16 degrees of freedom.
qt(0.005, 16, lower.tail = FALSE) * 31.71
## propagate: 92.62
## GUM H.1.6: 93  

## Second-order terms GUM H.1.7.
RES4 <- propagate(expr = EXPR3, data = DF3, type = "stat", 
                  do.sim = TRUE, verbose = TRUE, 
                  second.order = TRUE)
RES4$prop
RES4$sim
## propagate: sd.2 = 33.91115
## GUM H.1.7: u = 34.
## Also similar to the non-matrix-based approach
## in Wang et al. (2005, page 408): u1 = 33.91115.
## NOTE: After second-order correction, uncertainty is more
## similar to the value obtained from Monte Carlo simulation!

#################### GUM 2008 (2) #################
## Example in Annex H.2 from the GUM 2008 manual
## (see 'References'), simultaneous resistance
## and reactance measurement.
data(H.2)

## This gives exactly the means, uncertainties and
## correlations as given in Table H.2:
colMeans(H.2)
sqrt(colVarsC(H.2))/sqrt(5)
cor(H.2)

## H.2.3 Approach 1 using mean values and
## standard uncertainties:
EXPR6a <- expression((V/I) *  cos(phi)) ## R
EXPR6b <- expression((V/I) *  sin(phi)) ## X
EXPR6c <- expression(V/I) ## Z
MEAN6 <- colMeans(H.2)
SD6 <- sqrt(colVarsC(H.2))
DF6 <- rbind(MEAN6, SD6)
COV6ab <- cov(H.2) ## covariance matrix of V, I, phi
COV6c <- cov(H.2[, 1:2])  ## covariance matrix of V, I

RES6a <- propagate(expr = EXPR6a, data = DF6, type = "stat", 
                  do.sim = TRUE, verbose = TRUE, use.cov = COV6ab,
                  second.order = TRUE)

RES6b <- propagate(expr = EXPR6b, data = DF6, type = "stat", 
                  do.sim = TRUE, verbose = TRUE, use.cov = COV6ab,
                  second.order = TRUE)
                  
RES6c <- propagate(expr = EXPR6c, data = DF6[, 1:2], type = "stat", 
                  do.sim = TRUE, verbose = TRUE, use.cov = COV6c,
                  second.order = TRUE)

## This gives exactly the same values of mean and sd/sqrt(5)
## as given in Table H.4.
RES6a$prop
RES6b$prop
RES6c$prop

#################### GUM 2008 (3) #######################
## Example in Annex H.3 from the GUM 2008 manual
## (see 'References'), calibration of a thermometer.
## For this example, we can use the predict.lm function
## of R directly to return the error of predicted values.
data(H.3)
LM <- lm(bk ~ I(tk - 20), data = H.3)
## This will give exactly the same values as in H.3.3.
summary(LM)

## This will give exactly the same values as the
## fourth column ("Predicted correction") of Table H.6.
predict(LM)

## This will give exactly the same values as the
## fifth column ("Difference...") of Table H.6.
H.3$bk - predict(LM)

## Uncertainty in a predicted value. This will give
## exactly the values in H.3.4.
predict(LM, newdata = data.frame(tk = 30), se.fit = TRUE)

######### GUM 2008 Supplement 1 (1) #######################
## Example from 9.2.2 of the GUM 2008 Supplement 1
## (see 'References'), normally distributed input
## quantities. Assign values as in 9.2.2.1.
EXPR7 <- expression(X1 + X2 + X3 + X4)
X1 <- c(0, 1)
X2 <- c(0, 1)
X3 <- c(0, 1)
X4 <- c(0, 1)
DF7 <- cbind(X1, X2, X3, X4)
RES7 <- propagate(expr = EXPR7, data = DF7, type = "stat", 
                  do.sim = TRUE, verbose = TRUE, nsim = 1E5)
## This will give exactly the same results as in 
## 9.2.2.6, Table 2.
RES7$prop
RES7$sim

######### GUM 2008 Supplement 1 (2) #######################
## Example from 9.3 of the GUM 2008 Supplement 1
## (see 'References'), mass calibration.
## Formula 24 in 9.3.1.3 and values as in 9.3.1.4, Table 5.
EXPR8 <- expression((Mrc + dMrc) * (1 + (Pa - Pa0) * ((1/Pw) - (1/Pr))) - Mnom)
Mrc <- rnorm(1E5, 100000, 0.050)
dMrc <- rnorm(1E5, 1.234, 0.020)
Pa <- runif(1E5, 1.10, 1.30)  ## E(Pa) = 1.2, (b-a)/2 = 0.1 
Pw <- runif(1E5, 7000, 9000)  ## E(Pw) = 8000, (b-a)/2 = 1000
Pr <- runif(1E5, 7950, 8050) ## E(Pr) = 8000, (b-a)/2 = 50
Pa0 <- 1.2 
Mnom <- 100000
DF8 <- cbind(Mrc, dMrc, Pa, Pw, Pr, Pa0, Mnom)
RES8 <- propagate(expr = EXPR8, data = DF8, type = "sim",
                  do.sim = TRUE, verbose = TRUE, nsim = 1E5)
## This will give exactly the same results as in 
## 9.3.2.3, Table 6
RES8$prop
RES8$sim
 
######### GUM 2008 Supplement 1 (3) #######################
## Example from 9.4 of the GUM 2008 Supplement 1
## (see 'References'), comparioson loss in microwave
## power meter calibration, zero covariance.
## Formula 28 in 9.4.1.5 and values as in 9.4.1.7.
EXPR9 <- expression(X1^2 - X2^2)
X1 <- c(0.050, 0.005)
X2 <- c(0, 0.005)
DF9 <- cbind(X1, X2)
RES9a <- propagate(expr = EXPR9, data = DF9, type = "stat",
                  do.sim = TRUE, verbose = TRUE, nsim = 1E5)
## This will give exactly the same results as in 
## 9.4.2.2.7, Table 8, x1 = 0.050.
RES9a$prop
RES9a$sim

## Using covariance matrix with r(x1, x2) = 0.9
## We convert to covariances using cor2cov.
COR9 <- matrix(c(1, 0.9, 0.9, 1), nrow = 2)
COV9 <- cor2cov(COR9, c(0.005^2, 0.005^2))
colnames(COV9) <- c("X1", "X2")
rownames(COV9) <- c("X1", "X2")
RES9b <- propagate(expr = EXPR9, data = DF9, type = "stat",
                   use.cov = COV9, do.sim = TRUE, 
                   verbose = TRUE, nsim = 1E5)
## This will give exactly the same results as in 
## 9.4.3.2.1, Table 9, x1 = 0.050.
RES9b$prop
RES9b$sim

######### GUM 2008 Supplement 1 (4) #######################
## Example from 9.5 of the GUM 2008 Supplement 1
## (see 'References'), gauge block calibration.
## Assignment of PDF's as in Table 10 of 9.5.2.1.
EXPR10 <- expression(Ls + D + d1 + d2 - Ls *(da *(t0 + Delta) + as * dt) - Lnom)
Lnom <- 50000000
Ls <- propagate:::rst(100000, mean = 50000623, sd  = 25, df = 18)
D <- propagate:::rst(100000, mean = 215, sd = 6, df = 25)
d1 <- propagate:::rst(100000, mean = 0, sd = 4, df = 5)
d2 <- propagate:::rst(100000, mean = 0, sd = 7, df = 8)
as <- runif(100000, 9.5E-6, 13.5E-6)
t0 <- rnorm(100000, -0.1, 0.2)
Delta <- propagate:::rarcsin(100000, -0.5, 0.5)
da <- propagate:::rctrap(100000, -1E-6, 1E-6, 0.1E-6)
dt <- propagate:::rctrap(100000, -0.050, 0.050, 0.025)
DF10 <- cbind(Ls, D, d1, d2, as, t0, Delta, da, dt, Lnom)
RES10 <- propagate(expr = EXPR10, data = DF10, type = "sim",
                   use.cov = FALSE, verbose = TRUE, 
                   alpha = 0.01)
RES10
## This gives the same results as in 9.5.4.2, Table 11.
## However: results are exacter than in the GUM 2008
## manual, especially when comparing sd(Monte Carlo)
## sd(second-order Taylor expansion)!
## GUM 2008 gives 32 and 36, respectively.
RES10$prop
RES10$sim

########## Comparison to Pythons 'soerp' ###################
## Exactly the same results as under 
## https://pypi.python.org/pypi/soerp ! 
EXPR11 <- expression(C * sqrt((520 * H * P)/(M *(t + 460))))
H <- c(64, 0.5)
M <- c(16, 0.1)
P <- c(361, 2)
t <- c(165, 0.5)
C <- c(38.4, 0)
DAT11 <- makeDat(EXPR11)
RES11 <- propagate(expr = EXPR11, data = DAT11, type = "stat", 
                  do.sim = TRUE, verbose = TRUE) 
RES11
# }

Run the code above in your browser using DataLab