Decoding the Data: ROC Curve and Area Under the Curve
Ryan Tom, M.S., DCLS, MLS(ASCP)CM, provides an overview of the receiver operational characteristic curve (ROC Curve) and area under the curve (AUC).
Video Notes
Key Takeaways
Receiver Operational Characteristic Curve (ROC Curve)
The function of an ROC Curve is to evaluate the diagnostic accuracy of an index test. This determines whether an individual who was tested has a specific disease or not. To utilize an ROC Curve, the index test needs to be dichotomized (split into 2 data points) to achieve the true positive rate and false positive rate that can be plotted on an ROC curve graph. Finding the optimal threshold, where sensitivity and specificity are maximized, is the ultimate goal for this curve.- The ROC Curve is used to evaluate the diagnostic accuracy of an index test.
- For continuous outcomes, there is a number threshold that distinguishes if a test is normal or abnormal (positive or negative).
- A 2x2 contingency table is used to compare index test results with reference test results to identify true and false positives and negatives.
- The results are used to find the true positive rate (TPR) and false positive rate (FPR) for the index test.
- TPR = sensitivity = TP/(TP + FN). TP means true positive and FN means false negative.
- FPR = 1 – specificity = TN/(FP +TN). FP means false positive and TN means true negative.
- A graph is created with the TPR and FPR as the legends for the X and Y axes.
- The TPR and FPR for the index test are plotted on the graph to create a coordinate.
- Next, a new threshold is chosen for the index test.
- Using the new threshold, the index test results are compared to the reference test results again to gain new TPR and FPR results that will give a new coordinate.
- As the threshold changes, new TPR and FPR are created that can be plotted on a graph.
- A line can be drawn through the coordinates to create the curve (The ROC Curve).
- The randomness line is drawn from the bottom left of the graph to the top right of the graph.
- Ideally, the curve should be as far from the line as possible.
- A curve close to the line represents a test that is just as good as randomly selecting whether a result is normal or abnormal (guessing the result).
Area Under the Curve (AUC)
The AUC represents the area under the ROC Curve.- The AUC is calculated using the trapezoidal rule.
- A high AUC represents an index test that can accurately discriminate between normal and abnormal tests. An AUC of 0.5 represents a chance that is not better than randomly choosing if a result is normal or abnormal.
- Additionally, the Youden threshold is a statistical formula used to find the optimal threshold of an index test after the ROC Curve is created.
Author Information
Ryan Tom, M.S., DCLS, MLS(ASCP)CM, Clinical Technologist Level 5, North Central Bronx Hospital.
Stay up to date on all ASM Clinical Public Health Microbiology (CPHM) activities when you sign up for our monthly CPHM Newsletter. Be the first to know about free educational webinars, continuing education credit opportunities, advocacy, updated guidelines and more.
