Modeling, Analysis, Validation and Visualization of Observer
Performance Studies in Diagnostic Radiology
Description
Tools for quantitative assessment of medical imaging systems, radiologists or computer aided ('CAD') algorithms. Implements methods described in a book: 'Chakraborty' 'DP' (2017), "Observer Performance Methods for Diagnostic Imaging - Foundations, Modeling, and Applications with R-Based Examples", Taylor-Francis and its Online Appendices . Data collection paradigms include receiver operating characteristic ('ROC') and its location specific extensions, primarily free-response 'ROC' ('FROC'). 'ROC' data consists of a single rating per image, where the rating is the perceived confidence level the image is of a diseased patient. 'FROC' data consists of a variable number (including zero) of mark-rating pairs per image, where a mark is the location of a clinically reportable suspicious region and the rating is the corresponding confidence level that it is a true lesion. The software supersedes the current Windows version of 'JAFROC' software . 'RJafroc' is derived from it being an enhanced R version of original Windows 'JAFROC'. Implemented are a number of figures of merit quantifying performance, functions for visualizing operating characteristics; three ROC ratings data curve-fitting algorithms: the 'binormal' model ('BM'), the contaminated binormal model ('CBM') and the radiological search model ('RSM'). Unlike the 'BM', the 'CBM' and the 'RSM' predict proper ROC curves that do not cross the chance diagonal or display inappropriate hooks near the upper right corner of the plots. 'RSM' fitting additionally yields measures of search and lesion-classification performances, in addition to the usual case-classification performance measured by the area under the 'ROC' curve. Search performance is the ability to find lesions while avoiding finding non-lesions. Lesion-classification performance is the ability to discriminate between found lesions and non-lesions. For fully crossed study designs, termed multiple-reader multiple-case, significance testing of reader-averaged figure-of-merit differences between modalities is implemented via both 'Dorfman', 'Berbaum' and 'Metz' ('DBM') and the 'Obuchowski' and 'Rockette' ('OR') methods. Single treatment analysis allows comparison of performance of a group of radiologists to a specified value, or comparison of 'CAD' performance to a group of radiologists interpreting the same cases. Sample size estimation tools are provided for 'ROC' and 'FROC' studies that allow estimation of relevant variances from a pilot study to predict required numbers of readers and cases in a pivotal study. Utility and data file manipulation functions allow data to be read in any of the currently used input formats, including Excel, and the results of the analysis can be viewed in text or Excel output files. 'RJafroc' is used extensively in the cited book and its online appendices.