mctest: Sequential implementation of Monte Carlo tests with p-valube buckets

Description

Sequential implementation of the Monte Carlo test with p-value buckets.

Implementation of the Robbins-Lai (mctest.RL) and SIMCTEST (mctest.simctest) approaches to compute a decision interval (and decision) with respect to several thresholds/ p-value buckets. The function "mctest" is a wrapper function for both the Robbins-Lai and the SIMCTEST approach which calls one of the two using an additional parameter "method" (method="simctest" for SIMCTEST and method="RL" for Robbins-Lai).

Usage

mctest(gen,J=Jstar,epsilon=0.001,batch=10,batchincrement=1.1,maxbatch=100,
  method=c("simctest","RL"))
mctest.RL(gen,J=Jstar,epsilon=0.001,batch=10,batchincrement=1.1,maxbatch=100)
mctest.simctest(gen,J=Jstar,epsilon=0.001,batch=10,batchincrement=1.1,maxbatch=100)
J
Jstar
# S3 method for mctestres
print(x,...)

Arguments

gen

function that performs one sampling step. Returns 0 (sampled test statistic does not exceed the observation) or 1 (sampled test static exceeds the observation)

method

which method to use for stopping

p-value buckets to use. A matrix with two rows, each column describes an interval bucket. Column names give the code for the interval bucket. Defaults to Jstar.

epsilon

error bound

batch

initial number of samples to use before checking for stopping

batchincrement

factor by which the batch size gets multiplied after each step. 1 would mean no increment

maxbatch

maximum batch size

object of type "mctestres"

...

further arguments

Value

mctest, mctest.RL and mctest.simctest all return an object of class type mctestres, which has a print function (print.mctestres).

An object of class mctestres is a list with the following components: step (total batched number of samples drawn), decision.interval (interval for the p-value), decision (expressing significance), est.p (an estimate of the p-value) and realn (the actual number of samples taken without batching).

References

Ding, D., Gandy, A. and Hahn, G. (2019) Implementing Monte Carlo Tests with P-value Buckets. To appear in Scandinavian Journal of Statistics. arXiv:1703.09305 [stat.ME].

Examples

Run this code

# NOT RUN {
  #Example used in the above paper
  dat <- matrix(nrow=5,ncol=7,byrow=TRUE,
                c(1,2,2,1,1,0,1, 2,0,0,2,3,0,0, 0,1,1,1,2,7,3, 1,1,2,0,0,0,1, 0,1,1,1,1,0,0))
  loglikrat <- function(data){
    cs <- colSums(data)
    rs <- rowSums(data)
    mu <- outer(rs,cs)/sum(rs)
    2*sum(ifelse(data<=0.5, 0,data*log(data/mu)))
  }
  resample <- function(data){
    cs <- colSums(data)
    rs <- rowSums(data)
    n <- sum(rs)
    mu <- outer(rs,cs)/n/n
    matrix(rmultinom(1,n,c(mu)),nrow=dim(data)[1],ncol=dim(data)[2])
  }
  t <- loglikrat(dat);

  # function to generate samples
  gen <- function(){loglikrat(resample(dat))>=t}

  #using simctest
  mctest(gen)
  mctest.simctest(gen)
  mctest.RL(gen)
# }