Performs Dorfman-Berbaum-Metz (DBM) or Obuchowski-Rockette (OR)
significance testing (with Hillis' improvements), for specified dataset;
significance testing refers to analysis designed to assign a P-value,
and other statistics, for
rejecting the null hypothesis (NH) that the reader-averaged
figure of merit (FOM) difference between treatments is zero. The results of
the analysis are better visualized in the text or
Excel-formatted files produced by UtilOutputReport.
StSignificanceTesting(
dataset,
FOM,
FPFValue = 0.2,
alpha = 0.05,
method = "DBMH",
covEstMethod = "Jackknife",
nBoots = 200,
option = "ALL",
tempOrgCode = FALSE
)The dataset to be analyzed, see RJafroc-package.
Must have two or more treatments and two or more readers.
The figure of merit, see UtilFigureOfMerit
Only needed for LROC data and FOM = "PCL" or "ALROC";
where to evaluate a partial curve based figure of merit. The default is 0.2.
The significance level of the test of the null hypothesis that all treatment effects are zero; the default is 0.05
The significance testing method to be used. There are two options:
"DBMH" (the default) or "ORH", representing the Dorfman-Berbaum-Metz
and the Obuchowski-Rockette significance testing methods, respectively.
The covariance matrix estimation method
in ORH analysis (for method = "DBMH" the jackknife is always used).
"Jackknife", the default,
"Bootstrap", in which case nBoots (above) is relevant
"DeLong"; requires FOM = "Wilcoxon", otherwise
an error results.
The number of bootstraps (defaults to 200), relevant only if
covEstMethod = "Bootstrap" and method = "ORH"
Determines which factors are regarded as random vs. fixed:
"RRRC" = random-reader random case,
"FRRC" = fixed-reader random case,
"RRFC" = random-reader fixed case,
"ALL" = outputs the results of "RRRC", "FRRC"
and "RRFC" analyses
default FALSE; if TRUE, then code from version 0.0.1 of RJafroc is used (see RJafroc_0.0.1.tar). This is intended to check against errors that crept in subsequent to the original version as I attempted to improve the organization of the code and the output. As implicit in the name of this temporary flag, it will eventually be removed.
For method = "DBMH" the returned list has 15 members:
Figure of merit array: see return of UtilFigureOfMerit
ANOVA table of the pseudovalues, over all treatments
ANOVA table of the pseudovalues, for each treatment
The variance components of the DBM pseudovalue mode;
6 values, in the following order: varR, varC,
varTR, varTC, varRC and varErr
Results of the F-test for RRRC random reader random case
analysis; contains the following items: fRRRC - the value of the F-statistic,
ndfRRRC - the numerator degrees of freedom, ddfRRRC -
the denominator degrees of freedom and pRRRC - the pvalue
For RRRC analysis, the confidence intervals and related statistics for the FOM differences between pairs of treatments
For RRRC analysis, the confidence intervals and related test statistics for rdr. avg. FOM in each treatment
As for FTestStatsRRRC except that this is
for fixed-reader random-case (FRRC) analysis
For FRRC analysis, the confidence intervals and related test statistics for the FOM differences between pairs of treatments
For FRRC analysis, the confidence intervals and related tests for rdr. avg. FOM in each treatment
The mean squares table of the ANOVA of the pseudovalues for each reader (based on data for the specified reader)
The confidence intervals and related tests of the FOM differences between pairs of treatments for each reader
As for FTestStatsRRRC except that this is
for random-reader fixed-case (RRFC) analysis
For RRFC analysis, the confidence intervals and related test statistics for the FOM differences between pairs of treatments
For RRFC analysis, the confidence intervals and related tests for reader averaged FOM in each treatment
For method = "ORH" the return list has 14 members:
Figure of merit array: see return of UtilFigureOfMerit
List with 3 members: msT, msR, msTR
The variance components of the OR figure of merit model;
6 values, listed in the following order: varR, varTR, cov1,
cov2, cov3 and var
Results of the F-test for RRRC random reader random case
analysis; contains the following items: fRRRC - the value of the F-statistic,
ndfRRRC - the numerator degrees of freedom, ddfRRRC -
the denominator degrees of freedom and pRRRC - the pvalue
Same as in DBMH method
Same as in DBMH method
Same as FTestStatsRRRC, but
now treating reader as a fixed effect
Same as in DBMH method
Same as in DBMH method
Same as in DBMH method
Var and Cov1 estimates for each reader
Same as FTestStatsRRRC, but
now treating case as a fixed effect
Same as in DBMH method
Same as in DBMH method
Dorfman DD, Berbaum KS, Metz CE (1992) ROC characteristic rating analysis: Generalization to the Population of Readers and Patients with the Jackknife method, Invest. Radiol. 27, 723-731.
Obuchowski NA, Rockette HE (1995) Hypothesis Testing of the Diagnostic Accuracy for Multiple Diagnostic Tests: An ANOVA Approach with Dependent Observations, Communications in Statistics: Simulation and Computation 24, 285-308.
Hillis SL (2014) A marginal-mean ANOVA approach for analyzing multireader multicase radiological imaging data, Statistics in medicine 33, 330-360.
Chakraborty DP (2017) Observer Performance Methods for Diagnostic Imaging - Foundations, Modeling, and Applications with R-Based Examples, CRC Press, Boca Raton, FL. https://www.crcpress.com/Observer-Performance-Methods-for-Diagnostic-Imaging-Foundations-Modeling/Chakraborty/p/book/9781482214840
# NOT RUN {
StSignificanceTesting(dataset02,FOM = "Wilcoxon", method = "DBMH")
StSignificanceTesting(dataset02,FOM = "Wilcoxon", method = "ORH")
# }
# NOT RUN {
StSignificanceTesting(dataset05, FOM = "wAFROC")
StSignificanceTesting(dataset05, FOM = "HrAuc", method = "DBMH")
StSignificanceTesting(dataset05, FOM = "SongA1", method = "DBMH")
StSignificanceTesting(dataset05, FOM = "SongA2", method = "DBMH")
StSignificanceTesting(dataset05, FOM = "FOM_wAFROC1", method = "DBMH")
StSignificanceTesting(dataset05, FOM = "FOM_AFROC1", method = "DBMH")
StSignificanceTesting(dataset05, FOM = "FOM_AFROC", method = "DBMH")
# }
# NOT RUN {
# }
Run the code above in your browser using DataLab