Modeling ROC data
2024-04-25
Preface
TBA
0.1 Rationale and Organization
- Intended as an online update to my print book (Dev P. Chakraborty 2017).
- All references in this book to
RJafroc
refer to the R package with that name (case sensitive) (Dev Chakraborty and Zhai 2023). - Since its publication in 2017
RJafroc
, on which theR
code examples in the print book depend, has evolved considerably causing many of the examples to “break” if one uses the most current version ofRJafroc
. The code will still run if one usesRJafroc
0.0.1 but this is inconvenient and misses out on many of the software improvements made since the print book appeared. - This gives me the opportunity to update the print book.
- The online book has been divided into 3 books.
- The RJafrocQuickStartBook book.
- This book: RJafrocRocBook.
- The RJafrocFrocBook book.
0.2 TBA Acknowledgements
Dr. Xuetong Zhai
Dr. Peter Phillips
Online Latex Editor at this site
Dataset contributors
0.3 Temporary comments
This is intended to allow successful builds when a needed file is not in the build. These are indicated by, for example:
Chapter TempComment \@ref(proper-roc-models)
Fix these on final release.
- It is intended as an online update to my “physical” book (Dev P. Chakraborty 2017).
- Since its publication in 2017 the
RJafroc
package, on which theR
code examples in the book depend, has evolved considerably, causing many of the examples to “break”. - This also gives me the opportunity to improve on the book and include additional material.
TBA How much finished
- HMF approximately 70%
- This book is currently (as of December 2021) in preparation.
- Parts labeled TBA and TODOLAST need to be updated on final revision.
The pdf file of the book
Support for the pdf file has been removed (4/8/23) as plots produced by plotly
do not compile when using this format.
The html version of the book
Go here to view the html
version of the book.
A note on the online distribution mechanism of the book
- In the hard-copy version of my book (Dev P. Chakraborty 2017) the online distribution mechanism was
BitBucket
. BitBucket
allows code sharing within a closed group of a few users (e.g., myself and a grad student).- Since the purpose of open-source code is to encourage collaborations, this was, in hindsight, an unfortunate choice. Moreover, as my experience with R-packages grew, it became apparent that the vast majority of R-packages are shared on
GitHub
, notBitBucket
. - For these reasons I have switched to
GitHub
. All previous instructions pertaining toBitBucket
are obsolete. - In order to access
GitHub
material one needs to create a (free)GitHub
account. - Go to this link and click on
Sign Up
.
Structure of the book
The book is divided into parts as follows:
- Part I: Quick Start: intended for existing Windows
JAFROC
users who are seeking a quick-and-easy transition from WindowsJAFROC
toRJafroc
. - Part II: ROC paradigm: this covers the basics of the ROC paradigm
- Part III: Significance Testing: The general procedure used to determine the significance level, and associated statistics, of the observed difference in figure of merit between pairs of treatments or readers
- Part IV: FROC paradigm: TBA
Contributing to this book
I appreciate constructive feedback on this document. To do this raise an Issue
on the GitHub
interface. Click on the Issues
tab under dpc10ster/RJafrocRocBook
, then click on New issue
. When done this way, contributions from users automatically become part of the GitHub
documentation/history of the book.
Is this book relevant to you and what are the alternatives?
- Diagnostic imaging system evaluation
- Detection
- Detection combined with localization
- Detection combined with localization and classification
- Optimization of Artificial Intelligence (AI) algorithms
- CV
- Alternatives
Shelved vs. removed vs. parked folders needing heavy edits
- replace functions with
\text{}
; eg.\text{erf}
and\text{exp}
in all of document - Also for TPF, FPF etc.
- Temporarily shelved 17c-rsm-evidence.Rmd in removed folder
- Now 17-b is breaking; possibly related to changes in RJafroc: had to do with recent changes to RJafroc code - RSM_xFROC etc requiring intrinsic parameters; fixed 17-b
- parked has dependence of ROC/FROC performance on threshold
Coding aids
- sprintf(“%.4f”, proper formatting of numbers
- OpPtStr(, do:
- kbl(dfA, caption = “….”, booktabs = TRUE, escape = FALSE) %>% collapse_rows(columns = c(1, 3), valign = “middle”) %>% kable_styling(latex_options = c(“basic”, “scale_down”, “HOLD_position”), row_label_position = “c”)
- ```{r, attr.source = “.numberLines”}
- kbl(x12, caption = “Summary of optimization results using wAFROC-AUC.”, booktabs = TRUE, escape = FALSE) %>% collapse_rows(columns = c(1), valign = “middle”) %>% kable_styling(latex_options = c(“basic”, “scale_down”, “HOLD_position”), row_label_position = “c”)
- \(\text{exp} \left ( -\lambda' \right )\) space before dollar sign generates a pdf error
- FP errors generated by GitHub actions due to undefined labels: Error: Error: pandoc version 1.12.3 or higher is required and was not found (see the help page ?rmarkdown::pandoc_available). In addition: Warning message: In verify_rstudio_version() : Please install or upgrade Pandoc to at least version 1.17.2; or if you are using RStudio, you can just install RStudio 1.0+. Execution halted