Monte Carlo hypothesis testing
qleTest(est, par0 = NULL, obs0 = NULL, ..., sim, criterion = NULL,
nsim = 100, obs = NULL, alpha = 0.05, multi.start = 0L,
na.rm = TRUE, cores = 1L, cl = NULL, iseed = NULL,
verbose = FALSE)object of class qle, the estimation results from function qle
optional, vector of parameter for the null hypothesis
optional, vector of observed statistics corresponding to `par0`
arguments passed to the simulation function `sim`, searchMinimizer and multiSearch
user supplied simulation function, see qle
optional, NULL (default), name of the test statistic, either "qle" or "mahal" which overwrites the function criterion used for estimation of the model parameter
number of bootstrap replications of the statistical model to generate (simulated) statistics
optional, NULL (default), simulated statistics at the hypothesised parameter, if not given, these are generated at `par0` or at `est$par`
significance level for testing the hypothesis
integer, =0,1,2, level of multi start root finding (see details)
logical, TRUE (default), whether to remove `NA` values from the matrix of re-estimated parameters
number of cores for multistart searches for each given/generated observation, only if multi.start>0 enabled and ignored otherwise
cluster object, NULL (default), of class MPIcluster, SOCKcluster, cluster
integer, the seed for initializing the cluster workers for parallel computations
logical, TRUE for intermediate output
An object of class qleTest as a list of:
data frame of estimated parameters and error measures
the test result
name of the test
with attributes:
mean square error matrix of re-estimated parameters
average inverse quasi-information matrix over all re-estimated parameters
inverse quasi-information matrix at the parameter to be tested `est$par`
relative difference of the empirial and predicted standard error of the parameter to be tested
list of simulated statistics either at the estimated parameter or at the optional parameter `par0`
results from re-estimating the model parameters for each simulated observation `obs`
average quasi-score, respectively, average gradient of the MD at the re-estimated parameters
name of criterion function used as a test statistic: "qle" or "mahal"
list of the following elements: indices of re-estimation results where the inversion of the quasi-information matrix failed,
the re-estimated parameters have `NA`s, criterion function minimizations failed or did not converge numerically,
the integer seed value `iseed`
The function tests the null hypothesis \(H_0:\,\hat{\theta}=\theta_0\), that is, whether the statistical model w.r.t. to the estimated parameter explains the observed statistics, against the alternative \(H_1:\,\hat{\theta}\neq\theta_0\) based on a Monte Carlo approach (see vignette). Due to the approximate nature of the assumed statistical model for the observed statistics the exact distribution of the test statistics, that is, the Mahalanobis distance or quasi-deviance, is generally unknown and therefore its asymptotic distribution might be an unrealistic assumption for the null hypothesis. For this reason, and in order to retrieve an empirical p-value for testing, we generate bootstrap observations and re-estimate the model parameter for each observation in the same way as done before when estimating the model parameter. This includes all possible types of variance approximations available (by kriging or average approximations) and types of prediction variances (by kriging or cross-validation).
The function expects an estimation result `est` as returned from the main estimation function qle. If any simulated statistics
are available at the final parameter estimate or at `par0`, then these can be passed by `obs` and used as bootstrapped observations of the
summary statistics. Otherwise the function first generates those using `nsim` model replications. The criterion function approximations are used as
specified in the object `qsd` and will not be further augmented by additional samples or simulations during the test procedure.
The value of the test statistic is either chosen as the current criterion function value at the estimated parameter or it is re-computed at the
given parameter `par0` using, if given, the `real` observed statistics `obs0`. The user can also select a different criterion function
as a test statistic compared to the estimation before which can be set by `criterion`. Apart from the quasi-deviance as a test statistic, in
principle, any supported type of a least squares criterion, more generally, the Mahalanobis distance, can be used which only depends on the prefered type
of variance matrix approximation, see covarTx.
In order to efficiently find the roots of the quasi-score vector we implement a multi start concept for minimizing the criterion function.
If `multi.start=0` no single root finding is initiated from the estimated parameter (as a starting point) for each newly generated observation.
Using `multi.start=1` starts a multi start root finding only in case the local optimization gets stuck into a local minimum or does not
converge and setting `multi.start=2` always triggers a multi start local search for each simulated observation. Practically, the re-estimations
of the parameters might still fail to converge. However, the user can control the convergence conditions of the local solvers
(including the quasi-scoring iteration) by the corresponding control parameters (see searchMinimizer). Any failed re-estimation is
excluded from the test results and stored in the attribute `info`. In addition, as part of the returned data frame `param`
the empirical standard error, predicted standard error (based on the average inverse quasi-information matrix), the root mean square error,
the bias and sample mean value of the re-estimated parameters are also available. For a full example we refer the reader to the package vignette.