StSignificanceTesting: Perform significance testing, DBMH or ORH

Description

Performs Dorfman-Berbaum-Metz (DBM) or Obuchowski-Rockette (OR) significance testing with Hillis' improvements, for specified dataset; significance testing refers to analysis designed to assign a P-value for rejecting a null hypothesis (NH); the most common NH is that the reader-averaged figure of merit (FOM) difference between treatments is zero. The results of the analysis are better visualized in the text or, preferably, Excel-formatted, files produced by UtilOutputReport.

Usage

StSignificanceTesting (dataset, FOM = "wJAFROC", alpha = 0.05, 
   method = "DBMH", covEstMethod = "Jackknife", nBoots = 200, option = "ALL", 
   VarCompFlag = FALSE, FPFValue = 0.2)

Arguments

dataset

The dataset to be analyzed, see RJafroc-package

FOM

The figure of merit, default "wJAFROC", see UtilFigureOfMerit

alpha

The significance level of the test of the null hypothesis that all treatment effects are zero; the default alpha is 0.05

method

The significance testing method to be used. There are two options: "DBMH" (the default) or "ORH", representing the Dorfman-Berbaum-Metz and the Obuchowski-Rockette significance testing methods, respectively.

covEstMethod

The method used to estimate the covariance matrix in ORH analysis; it can be "Jackknife", "Bootstrap" or "DeLong", the last assumes FOM = "Wilcoxon", otherwise an error results. This parameter is not relevant if the analysis method is "DBMH"

nBoots

The number of bootstraps (default is 200), relevant only if the "Bootstrap" method is used to estimate the covariance matrix in the ORH method

option

Determines which factors are regarded as random vs. fixed: "RRRC" = random-reader random case, "FRRC" = fixed-reader random case, "RRFC" = random-reader fixed case, "ALL" outputs the results of "RRRC", "FRRC" and "RRFC" analyses

VarCompFlag

If TRUE, only the appropriate (DBM or OR) variance components (six in all) are returned, default is FALSE

FPFValue

Only needed for LROC data; where to evaluate a partial curve based figure of merit. The default is 0.2.

Value

For method = "DBMH" the returned value is a list with 22 members:

fomArray

The figure of merit array for each modality-reader combination

anovaY

The ANOVA table of the pseudovalues over all modalities

anovaYi

The ANOVA table of the pseudovalues for each modality

varComp

The variance components of the pseudovalue model underlying the analysis, 6 values, in the following order: c("Var(R)", "Var(C)", "Var(T*R)", "Var(T*C)", "Var(R*C)", "Var(Error)")

fRRRC

For random-reader random-case (RRRC) analysis, the F-statistic for rejecting the null hypothesis of no treatment effect

ddfRRRC

For RRRC analysis, the denominator degrees of freedom of the F statistic

pRRRC

For RRRC analysis, the p-value of the significance test of the NH

ciDiffTrtRRRC

For RRRC analysis, the confidence intervals and related test statistics for the FOM differences between pairs of modalities

ciAvgRdrEachTrtRRRC

For RRRC analysis, the confidence intervals and related test statistics for rdr. avg. FOM in each modality

fFRRC

For fixed-reader random-case (FRRC) analysis, the F-statistic for rejecting the NH

ddfFRRC

For FRRC analysis, the denominator degrees of freedom of the F-statistic

pFRRC

For FRRC analysis, the p-value of the significance test of the NH

ciDiffTrtFRRC

For FRRC analysis, the confidence intervals and related test statistics for the FOM differences between pairs of modalities

ciAvgRdrEachTrtFRRC

For FRRC analysis, the confidence intervals and related tests for rdr. avg. FOM in each modality

ssAnovaEachRdr

The sum of squares table of the ANOVA of the pseudovalues for each reader (based on data for the specified reader)

msAnovaEachRdr

The mean squares table of the ANOVA of the pseudovalues for each reader (based on data for the specified reader)

ciDiffTrtEachRdr

The confidence intervals and related tests of the FOM differences between pairs of modalities for each reader

fRRFC

For random-reader fixed-case (RRFC) analysis, the F statistic

ddfRRFC

For RRFC analysis, the denominator degrees of freedom of the F statistic

pRRFC

For RRFC analysis, the p-value for rejecting the NH

ciDiffTrtRRFC

For RRFC analysis, the confidence intervals and related test statistics for the FOM differences between pairs of modalities

ciAvgRdrEachTrtRRFC

For RRFC analysis, the confidence intervals and related tests for reader averaged FOM in each modality

For method = "ORH" the return value is a list with with 21 members:

fomArray

Figures of merit array. See the return of UtilFigureOfMerit

msT

Mean square of the figure of merit corresponding to the treatment effect

msTR

Mean square of the figure of merit corresponding to the treatment-reader effect

varComp

The variance components of the pseudovalue model underlying the analysis, 6 values, in the following order: c("Var(R)", "Var(T*R)", "COV1", "COV2", "COV3", "Var(Error)")

fRRRC

Same as DBMH method

ddfRRRC

Same as DBMH method

pRRRC

Same as DBMH method

ciDiffTrtRRRC

Same as DBMH method

ciAvgRdrEachTrtRRRC

Same as DBMH method

fFRRC

Same as DBMH method

ddfFRRC

Same as DBMH method

pFRRC

Same as DBMH method

ciDiffTrtFRRC

Same as DBMH method

ciAvgRdrEachTrtFRRC

Same as DBMH method

ciDiffTrtEachRdr

Same as DBMH method

varCovEachRdr

Obuchowski-Rockette Variance and Cov1 estimates for each reader

fRRFC

Same as DBMH method

ddfRRFC

Same as DBMH method

pRRFC

Same as DBMH method

ciDiffTrtRRFC

Same as DBMH method

ciAvgRdrEachTrtRRFC

Same as DBMH method

References

Dorfman DD, Berbaum KS, Metz CE (1992) ROC characteristic rating analysis: Generalization to the Population of Readers and Patients with the Jackknife method, Invest. Radiol. 27, 723-731.

Obuchowski NA, Rockette HE (1995) Hypothesis Testing of the Diagnostic Accuracy for Multiple Diagnostic Tests: An ANOVA Approach with Dependent Observations, Communications in Statistics: Simulation and Computation 24, 285-308.

Hillis SL (2014) A marginal-mean ANOVA approach for analyzing multireader multicase radiological imaging data, Statistics in medicine 33, 330-360.

Chakraborty DP (2017) Observer Performance Methods for Diagnostic Imaging - Foundations, Modeling, and Applications with R-Based Examples, CRC Press, Boca Raton, FL. https://www.crcpress.com/Observer-Performance-Methods-for-Diagnostic-Imaging-Foundations-Modeling/Chakraborty/p/book/9781482214840

Examples

Run this code

# NOT RUN {
retDbmRoc  <- StSignificanceTesting(dataset02, 
FOM = "Wilcoxon", method = "DBMH") 

# }
# NOT RUN {
retDbmwJAFROC  <- StSignificanceTesting(dataset05) # default is weighted JAFROC

retDbmHrAuc  <- StSignificanceTesting(dataset05, 
FOM = "HrAuc", method = "DBMH") 
print(retDbmHrAuc) 

retDbmSongA1  <- StSignificanceTesting(dataset05, 
FOM = "SongA1", method = "DBMH") 
print(retDbmSongA1)

retDbmSongA2  <- StSignificanceTesting(dataset05, 
FOM = "SongA2", method = "DBMH") 
print(retDbmSongA2)

retDbmwJafroc1  <- StSignificanceTesting(dataset05, 
FOM = "wJAFROC1", method = "DBMH")
print(retDbmwJafroc1)
 
retDbmJafroc1  <- StSignificanceTesting(dataset05, 
FOM = "JAFROC1", method = "DBMH")
print(retDbmJafroc1)
 
retDbmJAFROC  <- StSignificanceTesting(dataset05, 
FOM = "JAFROC", method = "DBMH") 
print(retDbmJAFROC)
 
# }
# NOT RUN {
retOR <- StSignificanceTesting(dataset02, 
FOM = "Wilcoxon", method = "ORH")
print(retOR)


# }

Run the code above in your browser using DataLab