Traditional versus modern approaches to screening mammography: a comparison of computer-assisted detection for synthetic 2D mammography versus an artificial intelligence algorithm for digital breast tomosynthesis
Computer-assisted detection (CADe) has been used in conjunction with mammography since its approval by the United States Food and Drug Administration (FDA) in 1998 to help radiologists detect breast cancer . Despite promising initial studies suggesting the potential of CADe to improve screening accuracy, research in clinical practice indicates that the diagnostic accuracy of mammography is not improved with the use of CADe, specifically with regard to low specificity and high false-positive rates [ – ]. Furthermore, CADe may not be a cost-effective measure for improving diagnostic accuracy or patient outcomes due to high false-positive rates and equipment costs . Unlike CADe algorithms, deep learning-based artificial intelligence (AI) algorithms for lesion detection and diagnosis on mammography do not rely on human-derived or manually engineered imaging features . Rather, deep learning-based AI algorithms use convolutional neural networks to extract high-level imaging features from large amounts of raw data, with the neural network model itself learning the features and relationships among features that are necessary to identify breast cancers on mammography. Deep learning-based AI algorithms have shown promise in improving accuracy and reducing false-positive rates [ – ]. While traditional CADe algorithms were developed for use with digital 2D mammography, new-generation AI algorithms can be used with 2D mammography and/or digital breast tomosynthesis (DBT) . DBT was approved for clinical use by the FDA in 2011 and, as of April 2024, is offered by 90% of practices across the United States . In clinical practice, DBT images are interpreted with synthetic 2D mammographic images, which are reconstructed from the DBT images, or separately-acquired 2D mammographic images. There are few published studies that directly compare traditional CADe to AI on the same 2D mammography dataset . There is no published literature, to our knowledge, that directly compares the traditional approach, CADe applied to 2D mammography, to the modern approach, AI applied to DBT. The purpose of this study was to compare the performance of a traditional machine learning CADe algorithm for synthetic 2D mammography to a deep learning-based AI algorithm for DBT on the same dataset. 

Study population/case selection Screening mammographic examinations were collected under an IRB-approved case collection protocol with a waiver for written informed consent. Under this protocol, cases of various dispositions were acquired consecutively between January 2016 and August 2018 from five clinical sites, and a stratified random sample of 764 cases was drawn. Of the 764 cases, 106 were biopsy-proven cancers and 658 were cancer-negative cases (97 biopsy-proven benign findings, 81 recalled but not recommended for biopsy cases, and 480 cases assessed as negative). One-year follow-up data were not available for the cases. Cases were included for female subjects of any race/ethnicity who underwent a screening mammography examination. Cases were excluded for subjects with symptomatic lesions, pacemakers in the mammography field of view, breast implants, motion during imaging, skin markers, cut-off anatomy, prior surgery and/or presence of biopsy clips visible on imaging, or missing standard views or reports. CADe and AI systems All cases were analyzed by CADe using ImageChecker v10.0 (Hologic, Inc.) and AI using Genius AI Detection v2.0 (Hologic, Inc.). For CADe analysis, synthetic 70 µm-resolution 2D images generated by Hologic tomosynthesis systems in for-processing format following DBT acquisitions were used. The CADe algorithm offers one operating point for synthetic 2D images. Per image, the number of marks is limited to four for calcifications, two for masses, and two for masses/calcifications in the same location. Per case, the number of marks is limited to eight for calcifications, four for masses, and four for masses/calcifications in the same location. For AI analysis, 1 mm slice-thickness, 70 µm-resolution tomosynthesis slices were used. The AI algorithm offers a single operating point and implements a simple capping mechanism of five marks per image for each type of mark, mass and calcifications. Outcomes Overall performance of 2D CADe and 3D AI was analyzed with the following metrics: area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and the rate of false marks. Both algorithms generated lesion marks with corresponding scores related to the potential likelihood of malignancy. For each algorithm, an overall case score was defined as the highest score of all lesion marks. AUC was determined using overall case scores from both algorithms. Sensitivity was estimated as the percentage of cancer cases correctly marked by CADe or AI. Sensitivity was further divided into lesion-specific and examination-specific analyses. For lesion-specific analysis, sensitivity was estimated as the percentage of cancer cases with at least one lesion mark in the accurate location by AI or CADe. Examination-specific sensitivity was the percentage of cancer cases with at least one lesion mark, regardless of location accuracy by AI or CADe. Specificity was estimated as the percentage of non-cancer cases with no lesion marks. False-positives were defined as the total number of false marks per study calculated for all non-cancer cases. Performance, with respect to sensitivity and specificity, was also analyzed by breast density and categorized into two groups: non-dense (almost entirely fatty and scattered areas of fibroglandular density) and dense (heterogeneously dense and extremely dense). Similar to overall performance, sensitivity was further categorized into lesion-specific and examination-specific analysis groups. Ground truth determination Determination of ground truth overlays on images for all 106 cancer cases was performed by an independent Mammography Quality Standards Act (MQSA)-qualified and board-certified radiologist (30 years of experience) using anonymized copies of clinical reports and associated images available for review. The expert radiologist reviewed the reports for screening and diagnostic imaging that were dictated by radiologists at the clinical sites. The images from screening, diagnostic, and post-biopsy studies were presented when available. The pathology reports of the biopsied lesions were reviewed to identify those particular lesions that were proven malignant by biopsy. The specific biopsied lesions were overlayed and tagged as malignant. Generation of ground truth overlays was performed using a proprietary tool, which enables simultaneous display of four standard-view tomosynthesis images in Digital Imaging and Communications in Medicine (DICOM) format and facilitates drawing overlays on top of the images. The overlays were generated by the expert on the tomosynthesis slice where the lesion was in-focus and best visualized. Statistical analysis The AUCs with 95% CIs for 2D CADe and 3D AI were calculated with a tool that utilizes the Scikit-learn library in Python. The VassarStats p -value calculator for two independent receiver operating characteristic (ROC) curves was used to compare the AUCs between 2D CADe and 3D AI. Overall sensitivity (lesion-specific and examination-specific) and specificity for 2D CADe and 3D AI were analyzed with a Wilson score interval for the 95% CI and a two-tailed Z -test to compare sensitivity and specificity between the two groups. These performance measures were conducted using Statistics Kingdom calculators. A comparison of false-positive rates between 2D CADe and 3D AI was done using a paired t -test in Excel. For all statistical analysis runs, P < 0.05 was considered statistically significant. 

Screening mammographic examinations were collected under an IRB-approved case collection protocol with a waiver for written informed consent. Under this protocol, cases of various dispositions were acquired consecutively between January 2016 and August 2018 from five clinical sites, and a stratified random sample of 764 cases was drawn. Of the 764 cases, 106 were biopsy-proven cancers and 658 were cancer-negative cases (97 biopsy-proven benign findings, 81 recalled but not recommended for biopsy cases, and 480 cases assessed as negative). One-year follow-up data were not available for the cases. Cases were included for female subjects of any race/ethnicity who underwent a screening mammography examination. Cases were excluded for subjects with symptomatic lesions, pacemakers in the mammography field of view, breast implants, motion during imaging, skin markers, cut-off anatomy, prior surgery and/or presence of biopsy clips visible on imaging, or missing standard views or reports. 

All cases were analyzed by CADe using ImageChecker v10.0 (Hologic, Inc.) and AI using Genius AI Detection v2.0 (Hologic, Inc.). For CADe analysis, synthetic 70 µm-resolution 2D images generated by Hologic tomosynthesis systems in for-processing format following DBT acquisitions were used. The CADe algorithm offers one operating point for synthetic 2D images. Per image, the number of marks is limited to four for calcifications, two for masses, and two for masses/calcifications in the same location. Per case, the number of marks is limited to eight for calcifications, four for masses, and four for masses/calcifications in the same location. For AI analysis, 1 mm slice-thickness, 70 µm-resolution tomosynthesis slices were used. The AI algorithm offers a single operating point and implements a simple capping mechanism of five marks per image for each type of mark, mass and calcifications. 

Overall performance of 2D CADe and 3D AI was analyzed with the following metrics: area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and the rate of false marks. Both algorithms generated lesion marks with corresponding scores related to the potential likelihood of malignancy. For each algorithm, an overall case score was defined as the highest score of all lesion marks. AUC was determined using overall case scores from both algorithms. Sensitivity was estimated as the percentage of cancer cases correctly marked by CADe or AI. Sensitivity was further divided into lesion-specific and examination-specific analyses. For lesion-specific analysis, sensitivity was estimated as the percentage of cancer cases with at least one lesion mark in the accurate location by AI or CADe. Examination-specific sensitivity was the percentage of cancer cases with at least one lesion mark, regardless of location accuracy by AI or CADe. Specificity was estimated as the percentage of non-cancer cases with no lesion marks. False-positives were defined as the total number of false marks per study calculated for all non-cancer cases. Performance, with respect to sensitivity and specificity, was also analyzed by breast density and categorized into two groups: non-dense (almost entirely fatty and scattered areas of fibroglandular density) and dense (heterogeneously dense and extremely dense). Similar to overall performance, sensitivity was further categorized into lesion-specific and examination-specific analysis groups. 

Determination of ground truth overlays on images for all 106 cancer cases was performed by an independent Mammography Quality Standards Act (MQSA)-qualified and board-certified radiologist (30 years of experience) using anonymized copies of clinical reports and associated images available for review. The expert radiologist reviewed the reports for screening and diagnostic imaging that were dictated by radiologists at the clinical sites. The images from screening, diagnostic, and post-biopsy studies were presented when available. The pathology reports of the biopsied lesions were reviewed to identify those particular lesions that were proven malignant by biopsy. The specific biopsied lesions were overlayed and tagged as malignant. Generation of ground truth overlays was performed using a proprietary tool, which enables simultaneous display of four standard-view tomosynthesis images in Digital Imaging and Communications in Medicine (DICOM) format and facilitates drawing overlays on top of the images. The overlays were generated by the expert on the tomosynthesis slice where the lesion was in-focus and best visualized. 

The AUCs with 95% CIs for 2D CADe and 3D AI were calculated with a tool that utilizes the Scikit-learn library in Python. The VassarStats p -value calculator for two independent receiver operating characteristic (ROC) curves was used to compare the AUCs between 2D CADe and 3D AI. Overall sensitivity (lesion-specific and examination-specific) and specificity for 2D CADe and 3D AI were analyzed with a Wilson score interval for the 95% CI and a two-tailed Z -test to compare sensitivity and specificity between the two groups. These performance measures were conducted using Statistics Kingdom calculators. A comparison of false-positive rates between 2D CADe and 3D AI was done using a paired t -test in Excel. For all statistical analysis runs, P < 0.05 was considered statistically significant. 

Patient population There were 764 screening DBT examinations in 764 patients with an average age of 58 ± 11 years (Table ). The majority of patients were between 40 and 69 years of age (81.5%, 623/764). Geographic location was relatively evenly spread across the five locations. Racial and ethnicity data were available for 56.4% (431/764) of the population; 81.7% (352/431) were White, 9.5% (41/431) were Black, 4.2% (18/431) were Hispanic or Latino, 3.5% (15/431) were Asian, and 1.2% (5/431) were Other. Breast density was predominantly scattered areas of fibroglandular density or heterogeneously dense for both cancer and non-cancer cases, which is representative of the screening population. In this enriched population of 764 cases, 13.9% were biopsy-proven cancers, and the remaining 86.1% were non-cancer cases (Table ). Within the biopsy-proven cancer group, the majority of breast cancer types were invasive ductal carcinoma (IDC) (39.6%, 42/106), ductal carcinoma in situ (DCIS) (34.0%, 36/106), or IDC/DCIS (17.0%, 18/106). Of the breast cancers that were masses, 81.1% (103/127) had a lesion size ranging between 5 and 15 mm in diameter. Of the breast cancers that presented as calcifications, 76.5% (65/85) had a lesion size ranging between 0 and 10 mm in diameter. AUC The overall AUC for 3D AI was 0.873 (95% CI 0.84–0.90) and for 2D CADe was 0.693 (95% CI 0.64–0.74) ( P < 0.001) (Fig. ). Sensitivity Lesion-specific sensitivity was estimated to ensure that true-positive examinations were those in which the cancer was flagged by the AI or CADe software at the correct location in the breast. Lesion-specific sensitivity of 3D AI versus 2D CADe was 94.3% (95% CI 88.2–97.4%) versus 72.6% (95% CI 63.5–80.2%) ( P = 0.002) (Table ). The exam-specific sensitivity of 3D AI was 96.2% (95% CI 90.7–98.5%) and for 2D CADe was 91.5% (95% CI 84.7–95.5%) ( P = 0.15). Figure shows an example of a cancer detected on mammography by AI but not by CADe. Specificity The specificity of 3D AI versus 2D CADe was 54.3% (95% CI 50.4–58.0%) versus 16.7% (95% CI 14.0–19.7%) ( P < 0.001) (Table ). False marks The rate of false marks on non-cancer cases was 0.91 per exam (95% CI 0.80–1.02) for 3D AI and 3.24 per exam (95% CI 3.00–3.48) for 2D CADe ( P < 0.001). Lesion-specific false-negatives occurred in 6 and 29 cases for 3D AI and 2D CADe, respectively (Table ). Breast cancer types for lesion-specific misses were predominantly IDC/DCIS (50.0%, 3/6) for AI and IDC (48.3%, 14/29) for CADe. Figure shows a mammogram on which multiple false-positive marks were made by CADe but not by AI. Performance by breast density Lesion-specific sensitivity was significantly higher for 3D AI than 2D CADe for both non-dense ( P = 0.003) and dense ( P = 0.002) breast tissue (Table ). Specificity was significantly higher for 3D AI than 2D CADe for both non-dense ( P < 0.001) and dense ( P < 0.001) breast tissue. 

There were 764 screening DBT examinations in 764 patients with an average age of 58 ± 11 years (Table ). The majority of patients were between 40 and 69 years of age (81.5%, 623/764). Geographic location was relatively evenly spread across the five locations. Racial and ethnicity data were available for 56.4% (431/764) of the population; 81.7% (352/431) were White, 9.5% (41/431) were Black, 4.2% (18/431) were Hispanic or Latino, 3.5% (15/431) were Asian, and 1.2% (5/431) were Other. Breast density was predominantly scattered areas of fibroglandular density or heterogeneously dense for both cancer and non-cancer cases, which is representative of the screening population. In this enriched population of 764 cases, 13.9% were biopsy-proven cancers, and the remaining 86.1% were non-cancer cases (Table ). Within the biopsy-proven cancer group, the majority of breast cancer types were invasive ductal carcinoma (IDC) (39.6%, 42/106), ductal carcinoma in situ (DCIS) (34.0%, 36/106), or IDC/DCIS (17.0%, 18/106). Of the breast cancers that were masses, 81.1% (103/127) had a lesion size ranging between 5 and 15 mm in diameter. Of the breast cancers that presented as calcifications, 76.5% (65/85) had a lesion size ranging between 0 and 10 mm in diameter. 

The overall AUC for 3D AI was 0.873 (95% CI 0.84–0.90) and for 2D CADe was 0.693 (95% CI 0.64–0.74) ( P < 0.001) (Fig. ). 

Lesion-specific sensitivity was estimated to ensure that true-positive examinations were those in which the cancer was flagged by the AI or CADe software at the correct location in the breast. Lesion-specific sensitivity of 3D AI versus 2D CADe was 94.3% (95% CI 88.2–97.4%) versus 72.6% (95% CI 63.5–80.2%) ( P = 0.002) (Table ). The exam-specific sensitivity of 3D AI was 96.2% (95% CI 90.7–98.5%) and for 2D CADe was 91.5% (95% CI 84.7–95.5%) ( P = 0.15). Figure shows an example of a cancer detected on mammography by AI but not by CADe. 

The specificity of 3D AI versus 2D CADe was 54.3% (95% CI 50.4–58.0%) versus 16.7% (95% CI 14.0–19.7%) ( P < 0.001) (Table ). 

The rate of false marks on non-cancer cases was 0.91 per exam (95% CI 0.80–1.02) for 3D AI and 3.24 per exam (95% CI 3.00–3.48) for 2D CADe ( P < 0.001). Lesion-specific false-negatives occurred in 6 and 29 cases for 3D AI and 2D CADe, respectively (Table ). Breast cancer types for lesion-specific misses were predominantly IDC/DCIS (50.0%, 3/6) for AI and IDC (48.3%, 14/29) for CADe. Figure shows a mammogram on which multiple false-positive marks were made by CADe but not by AI. 

Lesion-specific sensitivity was significantly higher for 3D AI than 2D CADe for both non-dense ( P = 0.003) and dense ( P = 0.002) breast tissue (Table ). Specificity was significantly higher for 3D AI than 2D CADe for both non-dense ( P < 0.001) and dense ( P < 0.001) breast tissue. 

This retrospective, multi-site study is one of few studies to directly compare AI and CADe for breast cancer detection on the same mammography dataset and, to our knowledge, the first study to compare the traditional approach of CADe applied to synthetic 2D mammography and the modern approach of AI applied to DBT. 3D AI has a significantly higher AUC, lesion-specific sensitivity, and specificity compared to 2D CADe, in addition to a significantly lower number of false-positive marks per non-cancer case. For performance by breast density, 3D AI had a significantly higher lesion-specific sensitivity and specificity compared to 2D CADe for both non-dense and dense tissue. These results suggest that the modern approach to screening mammography—AI applied to DBT—has the potential to improve performance metrics and patient outcomes. Our results show that AI applied to DBT has significantly higher lesion-specific sensitivity than CADe applied to synthetic 2D mammography. Two prior studies comparing CADe and AI on the same acquired 2D mammography dataset found no significant differences in sensitivity, though both studies included very few cancers (3 and 19, respectively) . We also calculated examination-specific sensitivity for our dataset, which is based on the presence of at least one mark on a true-positive mammogram (regardless of whether or not it is in the correct location of the cancer). The difference in lesion-specific and examination-level sensitivities was greater for 2D CADe (72.6 vs 91.5%) than 3D AI (94.3 vs 96.2%). Examination-specific sensitivity, often reported in the literature, is more likely to favor algorithms that make more marks, leading to an overestimation of sensitivity especially for low-specificity algorithms. AI applied to DBT was shown to have a significantly higher specificity and a reduced false-positive rate compared to CADe applied to synthetic 2D mammography, without compromising sensitivity. Specificity was also significantly higher, without compromising sensitivity, in fatty and dense breast tissue. These results are consistent with published literature suggesting that a major advantage of AI algorithms for mammography interpretation is higher specificity and a lower number of false-positive cues, both of which may help facilitate radiologists’ use and acceptance of such systems . The analysis of CADe and AI on the same screening dataset allows for a direct performance comparison of the two approaches. However, our results may also reflect inherent differences in mammographic examination technology, as the CADe algorithm was applied to synthetic 2D mammographic images and the AI algorithm was applied to DBT images. Although combined DBT and 2D mammography has been shown to lead to higher rates of cancer detection and lower rates of false-positives compared to 2D mammography alone [ – ], these improvements in performance metrics have not been achieved uniformly across practices [ – ]. Similarly, subsets of individuals and practices may benefit more from the integration of AI than others. For example, AI may lead to larger improvements in performance for general radiologists compared to specialized breast imaging radiologists . There are several limitations to this study. First, it is a retrospective analysis of a dataset enriched with cancer cases from a single vendor mammography dataset. Second, as discussed above, our study does not fully disentangle the potential impact of CADe vs AI and synthetic 2D mammography vs DBT; however, rather than comparing CADe and AI for synthetic 2D mammography, it provides a more representative comparison in the context of modern-day breast imaging practices in the United States, the vast majority of which utilize DBT. Third, one-year follow-up data were not available for the cases included in this study, thus limiting a comprehensive analysis of false-negative cases. Fourth, our results with regard to false marks may have been impacted by the different capping mechanisms used by the traditional CADe algorithm and the AI algorithm. In this study, we compare a traditional machine learning CADe algorithm applied to synthetic 2D mammography to a new-generation deep learning-based AI algorithm applied to DBT on the same mammography dataset. Our results show that AI applied to DBT has significantly better performance compared to CADe applied to synthetic 2D mammography across three indicators—AUC, lesion-specific sensitivity, and specificity. This work contributes to a growing body of evidence showing the benefits of AI for lesion detection and diagnosis on DBT. Future research evaluating the performance of AI in real-world clinical practice will shed insight into its true impact on patient care. 

