REFERENCES

Andersson, Ingvar, Debra M Ikeda, Sophia Zackrisson, Mark Ruschin, Tony Svahn, Pontus Timberg, and Anders Tingberg. 2008. “Breast Tomosynthesis and Digital Mammography: A Comparison of Breast Cancer Visibility and BIRADS Classification in a Population of Cancers with Subtle Mammographic Findings.” European Radiology 18 (12): 2817–25.

Bamber, Donald. 1975. “The Area Above the Ordinal Dominance Graph and the Area Below the Receiver Operating Characteristic Graph.” Journal of Mathematical Psychology 12 (4): 387–415.

Barlow, William E, Chen Chi, Patricia A Carney, Stephen H Taplin, Carl D’Orsi, Gary Cutter, R Edward Hendrick, and Joann G Elmore. 2004. “Accuracy of Screening Mammography Interpretation by Characteristics of Radiologists.” Journal of the National Cancer Institute 96 (24): 1840–50.

Barnes, GT, EA Sabbagh, DP Chakraborty, PH Nath, RF Luna, C Sanders, and RG Fraser. 1989. “A Comparison of Dual-Energy Digital Radiography and Screen-Film Imaging in the Detection of Subtle Interstitial Pulmonary Disease.” Investigative Radiology 24 (8): 585–91.

Barrett, Harrison H, and Kyle J Myers. 2013. Foundations of Image Science. John Wiley & Sons.

Beam, Craig A, Peter M Layde, and Daniel C Sullivan. 1996. “Variability in the Interpretation of Screening Mammograms by US Radiologists: Findings from a National Sample.” Archives of Internal Medicine 156 (2): 209–13.

Berbaum, Kevin S, Donald D Dorfman, EA Franken Jr, and Robert T Caldwell. 2002. “An Empirical Comparison of Discrete Ratings and Subjective Probability Ratings.” Academic Radiology 9 (7): 756–63.

Bolker, Ben, and R Development Core Team. 2022. Bbmle: Tools for General Maximum Likelihood Estimation. https://github.com/bbolker/bbmle.

Broyden, Charles George. 1970. “The Convergence of a Class of Double-Rank Minimization Algorithms 1. General Considerations.” IMA Journal of Applied Mathematics 6 (1): 76–90.

Burgess, Arthur E. 2011. “Visual Perception Studies and Observer Models in Medical Imaging.” In Seminars in Nuclear Medicine, 41:419–36. Elsevier.

Chakraborty, D. P., E. S. Breatnach, M. V. Yester, B. Soto, G. T. Barnes, and R. G. Fraser. 1986. “Digital and Conventional Chest Imaging: A Modified ROC Study of Observer Performance Using Simulated Nodules.” Journal Article. Radiology 158: 35–39. https://doi.org/10.1148/radiology.158.1.3940394.

Chakraborty, Dev P. 1989. “Maximum Likelihood Analysis of Free-Response Receiver Operating Characteristic (FROC) Data.” Medical Physics 16 (4): 561–68.

———. 2017. Observer Performance Methods for Diagnostic Imaging: Foundations, Modeling, and Applications with r-Based Examples. Boca Raton, FL: CRC Press.

Chakraborty, Dev, and Xuetong Zhai. 2022. RJafroc: Artificial Intelligence Systems and Observer Performance. https://dpc10ster.github.io/RJafroc/.

Chakraborty, DP, and Tony Svahn. 2011. “Estimating the Parameters of a Model of Visual Search from ROC Data: An Alternate Method for Fitting Proper ROC Curves.” In Medical Imaging 2011: Image Perception, Observer Performance, and Technology Assessment, 7966:189–97. SPIE.

Chakraborty, DP, and Hong-Jun Yoon. 2008. “Operating Characteristics Predicted by Models for Diagnostic Tasks Involving Lesion Localization.” Medical Physics 35 (2): 435–45.

Chakraborty, DP, Hong-Jun Yoon, and Claudia Mello-Thoms. 2012. “Inverse Dependence of Search and Classification Performances in Lesion Localization Tasks.” In Medical Imaging 2012: Image Perception, Observer Performance, and Technology Assessment, 8318:129–40. SPIE.

DeLong, Elizabeth R, David M DeLong, and Daniel L Clarke-Pearson. 1988. “Comparing the Areas Under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach.” Biometrics, 837–45.

Dobbins III, James T, H Page McAdams, John M Sabol, Dev P Chakraborty, Ella A Kazerooni, Gautham P Reddy, Jenny Vikgren, and Magnus Båth. 2016. “Multi-Institutional Evaluation of Digital Tomosynthesis, Dual-Energy Radiography, and Conventional Chest Radiography for the Detection and Management of Pulmonary Nodules.” Journal Article. Radiology 282 (1): 236–50.

Dorfman, Donald D, and Edward Alf Jr. 1969. “Maximum-Likelihood Estimation of Parameters of Signal-Detection Theory and Determination of Confidence Intervals—Rating-Method Data.” Journal of Mathematical Psychology 6 (3): 487–96.

Dorfman, Donald D, and Kevin S Berbaum. 2000a. “A Contaminated Binormal Model for ROC Data: Part II. A Formal Model.” Academic Radiology 7 (6): 427–37.

———. 2000b. “A Contaminated Binormal Model for ROC Data: Part III. Initial Evaluation with Detection ROC Data.” Academic Radiology 7 (6): 438–47.

Dorfman, Donald D, Kevin S Berbaum, and Eric A Brandser. 2000. “A Contaminated Binormal Model for ROC Data: Part i. Some Interesting Examples of Binormal Degeneracy.” Academic Radiology 7 (6): 420–26.

Dorfman, Donald D, Kevin S Berbaum, Charles E Metz, Russell V Lenth, James A Hanley, and Hatem Abu Dagga. 1997. “Proper Receiver Operating Characteristic Analysis: The Bigamma Model.” Academic Radiology 4 (2): 138–49.

Edwards, Darrin C, Matthew A Kupinski, Charles E Metz, and Robert M Nishikawa. 2002. “Maximum Likelihood Fitting of FROC Curves Under an Initial-Detection-and-Candidate-Analysis Model.” Medical Physics 29 (12): 2861–70.

Egan, James P. 1975b. Signal Detection Theory and ROC Analysis. Book. First. Academic Press Series in Cognition and Perception. New York: Academic Press, Inc.

———. 1975a. Signal Detection Theory and ROC Analysis. Book. First. Academic Press Series in Cognition and Perception. New York: Academic Press, Inc.

Fenton, Joshua J. 2015. “Is It Time to Stop Paying for Computer-Aided Mammography?” JAMA Internal Medicine 175 (11): 1837–38.

Fenton, Joshua J, Linn Abraham, Stephen H Taplin, Berta M Geller, Patricia A Carney, Carl D’Orsi, Joann G Elmore, William E Barlow, and Breast Cancer Surveillance Consortium. 2011. “Effectiveness of Computer-Aided Detection in Community Mammography Practice.” Journal of the National Cancer Institute 103 (15): 1152–61.

Fenton, Joshua J, Stephen H Taplin, Patricia A Carney, Linn Abraham, Edward A Sickles, Carl D’Orsi, Eric A Berns, et al. 2007. “Influence of Computer-Aided Detection on Performance of Screening Mammography.” New England Journal of Medicine 356 (14): 1399–409.

Fletcher, Roger. 1970. “A New Approach to Variable Metric Algorithms.” The Computer Journal 13 (3): 317–22.

———. 2013. Practical Methods of Optimization. John Wiley & Sons.

Franken, Jr., Edmund A., Kevin S. Berbaum, Susan M. Marley, Wilbur L. Smith, Yutaka Sato, Simon C. S. Kao, and Steven G. Milam. 1992. “Evaluation of a Digital Workstation for Interpreting Neonatal Examinations: A Receiver Operating Characteristic Study.” Journal Article. Investigative Radiology 27 (9): 732–37. http://journals.lww.com/investigativeradiology/Fulltext/1992/09000/Evaluation_of_a_Digital_Workstation_for.16.aspx.

Fryback, Dennis G, and John R Thornbury. 1991. “The Efficacy of Diagnostic Imaging.” Medical Decision Making 11 (2): 88–94.

Genz, Alan, Frank Bretz, Tetsuhisa Miwa, Xuefei Mi, and Torsten Hothorn. 2021. Mvtnorm: Multivariate Normal and t Distributions. http://mvtnorm.R-forge.R-project.org.

Goldfarb, Donald. 1970. “A Family of Variable-Metric Methods Derived by Variational Means.” Mathematics of Computation 24 (109): 23–26.

Green, David Marvin, John A Swets, et al. 1966. Signal Detection Theory and Psychophysics. Vol. 1. Wiley New York.

Gur, David, Andriy I. Bandos, Cathy S. Cohen, Christiane M. Hakim, Lara A. Hardesty, Marie A. Ganott, Ronald L. Perrin, et al. 2008. “The "Laboratory" Effect: Comparing Radiologists’ Performance and Variability During Prospective Clinical and Laboratory Mammography Interpretations.” Journal Article. Radiology 249 (1): 47–53. https://doi.org/10.1148/radiol.2491072025.

Hanley, James A. 1988. “The Robustness of the" Binormal" Assumptions Used in Fitting ROC Curves.” Medical Decision Making 8 (3): 197–203.

Hanley, James A, and Karim O Hajian-Tilaki. 1997. “Sampling Variability of Nonparametric Estimates of the Areas Under Receiver Operating Characteristic Curves: An Update.” Academic Radiology 4 (1): 49–58.

Hanley, James A, and Barbara J McNeil. 1982. “The Meaning and Use of the Area Under a Receiver Operating Characteristic (ROC) Curve.” Radiology 143 (1): 29–36.

Hartmann, Lynn C, Thomas A Sellers, Marlene H Frost, Wilma L Lingle, Amy C Degnim, Karthik Ghosh, Robert A Vierkant, et al. 2005. “Benign Breast Disease and the Risk of Breast Cancer.” New England Journal of Medicine 353 (3): 229–37.

Hupse, Rianne, Maurice Samulski, Marc Lobbes, Ard Heeten, MechliW Imhof-Tas, David Beijerinck, Ruud Pijnappel, Carla Boetes, and Nico Karssemeijer. 2013. “Standalone Computer-Aided Detection Compared to Radiologists’ Performance for the Detection of Mammographic Masses.” Journal Article. European Radiology 23 (1): 93–100. https://doi.org/10.1007/s00330-012-2562-7.

Jiang, Yulei, and Charles E. Metz. 2010. “BI-RADS Data Should Not Be Used to Estimate ROC Curves.” Journal Article. Radiology 256 (1): 29–31. https://doi.org/10.1148/radiol.10091394.

Jiang, Yulei, Charles E Metz, and Robert M Nishikawa. 1996. “A Receiver Operating Characteristic Partial Area Index for Highly Sensitive Diagnostic Tests.” Radiology 201 (3): 745–50.

Kundel, H, K Berbaum, D Dorfman, D Gur, CE Metz, and RG Swensson. 2008. “Receiver Operating Characteristic Analysis in Medical Imaging (ICRU Report 79).” Report, International Commission on Radiation Units & Measurments.

Larsen, Richard J, and Morris L Marx. 2005. An Introduction to Mathematical Statistics. Prentice Hall Hoboken, NJ.

Lehman, Constance D, Robert D Wellman, Diana SM Buist, Karla Kerlikowske, Anna NA Tosteson, Diana L Miglioretti, Breast Cancer Surveillance Consortium, et al. 2015. “Diagnostic Accuracy of Digital Screening Mammography with and Without Computer-Aided Detection.” JAMA Internal Medicine 175 (11): 1828–37.

Liberman, Laura, and Jennifer H Menell. 2002. “Breast Imaging Reporting and Data System (BI-RADS).” Radiologic Clinics 40 (3): 409–30.

Lusted, L. B. 1971. “Signal Detectability and Medical Decision Making.” Journal Article. Science 171: 1217–1219. https://doi.org/10.1126/science.171.3977.1217.

Macmillan, Neil A, and C Douglas Creelman. 2004. Detection Theory: A User’s Guide. Psychology press.

Mann, H. B., and D. R. Whitney. 1947. “On a Test of Whether One of Two Random Variables Is Stochastically Larger Than the Other.” Journal Article. Annals of Mathematical Statistics 18: 50−60.

Marcum, Jess. 1960. “A Statistical Theory of Target Detection by Pulsed Radar.” IRE Transactions on Information Theory 6 (2): 59–267.

Marcum, JI. 1947. “A Statistical Theory of Target Detection by Pulsed Radar.” RAND CORP SANTA MONICA CA.

Metz, C. E. 1978. “Basic Principles of ROC Analysis.” Journal Article. Seminars in Nuclear Medicine 8 (4): 283–98. https://doi.org/10.1016/s0001-2998(78)80014-2.

Metz, Charles E. 1989. “Some Practical Issues of Experimental Design and Data Analysis in Radiological ROC Studies.” Investigative Radiology 24 (3): 234–45.

Metz, Charles E. 1986. “ROC Methodology in Radiologic Imaging.” Journal Article. Investigative Radiology 21 (9): 720–33. http://journals.lww.com/investigativeradiology/Fulltext/1986/09000/ROC_Methodology_in_Radiologic_Imaging.9.aspx.

Metz, Charles E, Benjamin A Herman, and Cheryl A Roe. 1998. “Statistical Comparison of Two ROC-Curve Estimates Obtained from Partially-Paired Datasets.” Medical Decision Making 18 (1): 110–21.

Metz, Charles E, and Helen B Kronman. 1980. “Statistical Significance Tests for Binormal ROC Curves.” Journal of Mathematical Psychology 22 (3): 218–43.

Metz, Charles E, and Xiaochuan Pan. 1999. “‘Proper’ Binormal ROC Curves: Theory and Maximum-Likelihood Estimation.” Journal of Mathematical Psychology 43 (1): 1–33.

Metz, Charles E, Pu-Lan Wang, and Helen B Kronman. 1984. “A New Approach for Testing the Significance of Differences Between ROC Curves Measured from Correlated Data.” In Information Processing in Medical Imaging: Proceedings of the 8th Conference, Brussels, 29 August–2 September 1983, 432–45. Springer.

Metz, C, H Kronman, J Shen, and P Wang. 1989. “CORROC2: A Program for ROC Analysis of Correlated, Inherently Categorical, Rating-Scale Data.” Department of Radiology, University of Chicago.

Miller, George A. 1956. “The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information.” Psychological Review 63 (2): 81.

Neyman, Jerzy, and Egon Sharpe Pearson. 1933. “IX. On the Problem of the Most Efficient Tests of Statistical Hypotheses.” Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character 231 (694-706): 289–337.

Nishikawa, Robert. 2012. “Estimating Sensitivity and Specificity in an ROC Experiment.” Journal Article. Breast Imaging, 690–96.

Pan, Xiaochuan, and Charles E Metz. 1997. “The ‘Proper’ Binormal Model: Parametric Receiver Operating Characteristic Curve Estimation with Degenerate Data.” Academic Radiology 4 (5): 380–89.

Pearson, Karl. 1900. “X. On the Criterion That a Given System of Deviations from the Probable in the Case of a Correlated System of Variables Is Such That It Can Be Reasonably Supposed to Have Arisen from Random Sampling.” Journal Article. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science 50 (302): 157–75. https://doi.org/10.1080/14786440009463897.

Penedo, Monica, Miguel Souto, Pablo G. Tahoces, Jose M. Carreira, Justo Villalon, Gerardo Porto, Carmen Seoane, et al. 2005. “Free-Response Receiver Operating Characteristic Evaluation of Lossy JPEG2000 and Object-Based Set Partitioning in Hierarchical Trees Compression of Digitized Mammograms.” Journal Article. Radiology 237 (2): 450–57.

Philpotts, Liane E. 2009. “Can Computer-Aided Detection Be Detrimental to Mammographic Interpretation?” Radiology 253 (1): 17–22.

Pisano, Etta D, Constantine Gatsonis, Edward Hendrick, Martin Yaffe, Janet K Baum, Suddhasatta Acharyya, Emily F Conant, et al. 2005. “Diagnostic Performance of Digital Versus Film Mammography for Breast-Cancer Screening.” New England Journal of Medicine 353 (17): 1773–83.

Pollack, Irwin. 1952. “The Information of Elementary Auditory Displays.” The Journal of the Acoustical Society of America 24 (6): 745–49.

———. 1953. “The Information of Elementary Auditory Displays. II.” The Journal of the Acoustical Society of America 25 (4): 765–69.

Press, W. H., S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery. 2007. Numerical Recipes: The Art of Scientific Computing. Book. 3rd ed. Cambridge: Cambridge University Press.

Rockette, Howard E, David Gur, and Charles E Metz. 1992. “The Use of Continuous and Discrete Confidence Judgments in Receiver Operating Characteristic Studies of Diagnostic Imaging Techniques.” Investigative Radiology 27 (2): 169–72.

Ruschin, Mark., Pontus. Timberg, Magnus. Bath, Bengt. Hemdal, Tony. Svahn, Rob. Saunders, Ehsan. Samei, et al. 2007. “Dose Dependence of Mass and Microcalcification Detection in Digital Mammography: Free Response Human Observer Studies.” Journal Article. Medical Physics 34: 400–407.

Shanno, David F. 1970. “Conditioning of Quasi-Newton Methods for Function Minimization.” Mathematics of Computation 24 (111): 647–56.

Shanno, David F, and Paul C Kettler. 1970. “Optimal Conditioning of Quasi-Newton Methods.” Mathematics of Computation 24 (111): 657–64.

Sievert, Carson, Chris Parmer, Toby Hocking, Scott Chamberlain, Karthik Ram, Marianne Corvellec, and Pedro Despouy. 2022. Plotly: Create Interactive Web Graphics via Plotly.js. https://CRAN.R-project.org/package=plotly.

Skaane, Per, Andriy I Bandos, Randi Gullien, Ellen B Eben, Ulrika Ekseth, Unni Haakenaasen, Mina Izadi, Ingvild N Jebsen, Gunnar Jahr, and Mona Krager. 2013. “Comparison of Digital Mammography Alone and Digital Mammography Plus Tomosynthesis in a Population-Based Screening Program.” Journal Article. Radiology 267 (1): 47–56.

Swensson, Richard G. 1996. “Unified Measurement of Observer Performance in Detecting and Localizing Target Objects on Images.” Medical Physics 23 (10): 1709–25.

Thompson, John D, Peter Hogg, David J Manning, Katy Szczepura, and Dev P Chakraborty. 2014. “A Free-Response Evaluation Determining Value in the Computed Tomography Attenuation Correction Image for Revealing Pulmonary Incidental Findings: A Phantom Study.” Journal Article. Academic Radiology 21 (4): 538–45.

Thompson, Mary Lou, and Walter Zucchini. 1989. “On the Statistical Analysis of ROC Curves.” Statistics in Medicine 8 (10): 1277–90.

Van Dyke, C. W., R. D. White, N. A. Obuchowski, M. A. Geisinger, R. J. Lorig, and M. A. Meziane. 1993. “Cine MRI in the Diagnosis of Thoracic Aortic Dissection.” Journal Article. 79th RSNA Meetings.

Vannier, Michael W, Thomas K Pilgram, Jeffrey L Marsh, B Balfour Kraemer, Susan C Rayne, Mokhtar H Gado, Christopher J Moran, William H McAlister, Gary D Shackelford, and Robert A Hardesty. 1994. “Craniosynostosis: Diagnostic Imaging with Three-Dimensional CT Presentation.” American Journal of Neuroradiology 15 (10): 1861–69.

Vikgren, Jenny, Sara Zachrisson, Angelica Svalkvist, Ase A. Johnsson, Marianne Boijsen, Agneta Flinck, Susanne Kheddache, and Magnus Bath. 2008. “Comparison of Chest Tomosynthesis and Chest Radiography for Detection of Pulmonary Nodules: Human Observer Study of Clinical Cases.” Journal Article. Radiology 249 (3): 1034–41. https://doi.org/10.1148/radiol.2492080304.

Wagner, Robert F, Sergey V Beiden, and Charles E Metz. 2001. “Continuous Versus Categorical Data for ROC Analysis: Some Quantitative Considerations.” Academic Radiology 8 (4): 328–34.

Warren, Lucy M, Rosalind M Given-Wilson, Matthew G Wallis, Julie Cooke, Mark D Halling-Brown, Alistair Mackenzie, Dev P Chakraborty, Hilde Bosmans, David R Dance, and Kenneth C Young. 2014. “The Effect of Image Processing on the Detection of Cancers in Digital Mammography.” Journal Article. American Journal of Roentgenology 203 (2): 387–93.

Wilcoxon, F. 1945. “Individual Comparison by Ranking Methods.” Journal Article. Biometrics 1: 80–83.

Zanca, Federica, Stephen L Hillis, Filip Claus, Chantal Van Ongeval, Valerie Celis, Veerle Provoost, Hong-Jun Yoon, and Hilde Bosmans. 2012. “Correlation of Free-Response and Receiver-Operating-Characteristic Area-Under-the-Curve Estimates: Results from Independently Conducted FROC/ROC Studies in Mammography.” Medical Physics 39 (10): 5917–29.

Zanca, Federica, Jurgen Jacobs, Chantal Van Ongeval, Filip Claus, Valerie Celis, Catherine Geniets, Veerle Provost, Herman Pauwels, Guy Marchal, and Hilde Bosmans. 2009. “Evaluation of Clinical Image Processing Algorithms Used in Digital Mammography.” Medical Physics 36 (3): 765–75.

Zhai, Xuetong, and Dev P Chakraborty. 2017. “A Bivariate Contaminated Binormal Model for Robust Fitting of Proper ROC Curves to a Pair of Correlated, Possibly Degenerate, ROC Datasets.” Medical Physics 44 (6): 2207–22.

Zhou, Xiao-Hua, Nancy A. Obuchowski, and Donna K. McClish. 2002. Statistical Methods in Diagnostic Medicine. Book. New York: John Wiley; Sons.