Computer-aided diagnosis system versus conventional reading system in low-dose (< 2 mSv) computed tomography: comparative study for patients at risk of lung cancer

ABSTRACT BACKGROUND: Computer-aided diagnosis in low-dose (≤ 3 mSv) computed tomography (CT) is a potential screening tool for lung nodules, with quality interpretation and less inter-observer variability among readers. Therefore, we aimed to determine the screening potential of CT using a radiation dose that does not exceed 2 mSv. OBJECTIVE: We aimed to compare the diagnostic parameters of low-dose (< 2 mSv) CT interpretation results using a computer-aided diagnosis system for lung cancer screening with those of a conventional reading system used by radiologists. DESIGN AND SETTING: We conducted a comparative study of chest CT images for lung cancer screening at three private institutions. METHODS: A database of low-dose (< 2 mSv) chest CT images of patients at risk of lung cancer was viewed with the conventional reading system (301 patients and 226 nodules) or computer-aided diagnosis system without any subsequent radiologist review (944 patients and 1,048 nodules). RESULTS: The numbers of detected and solid nodules per patient (both P < 0.0001) were higher using the computer-aided diagnosis system than those using the conventional reading system. The nodule size was reported as the maximum size in any plane in the computer-aided diagnosis system. Higher numbers of patients (102 [11%] versus 20 [7%], P = 0.0345) and nodules (154 [15%] versus 17 [8%], P = 0.0035) were diagnosed with cancer using the computer-aided diagnosis system. CONCLUSIONS: The computer-aided diagnosis system facilitates the diagnosis of cancerous nodules, especially solid nodules, in low-dose (< 2 mSv) CT among patients at risk for lung cancer.


INTRODUCTION
Low-dose computed tomography is an effective imaging modality to reduce mortality in patients at high risk of lung cancer. [1][2][3][4] In China, the computed tomography interpretation systems for the management of lung cancer vary among institutions. Moreover, the experiences of radiologists have an impact on computed tomography interpretation. 5 Therefore, standardized computed tomography interpretation and management of nodule screening is crucial. [6][7][8][9] Computer-aided diagnosis is reportedly a potential measurement tool for screening lung nodules, with quality interpretation and fewer variabilities among readers. [10][11][12][13] The European Society of Radiology and European Respiratory Society recommend computer-aided diagnosis of lung cancer nodules. 14 The investigated computed tomography scans were considered low dose at 3 mSv or less; however, the requirement for low-dose computed tomography is actually < 2 mSv. 5,15 However, computer-aided diagnosis in computed tomography can miss lung cancer nodules that are detected by radiologists. 11 A computer-aided diagnosis system has less sensitivity for ground-glass nodules than the conventional reading system. 16 Computer-aided diagnosis systems often miss lesions that are large, endobronchial, and inseparable from the mediastinum or perihilar. In addition, computer-aided diagnosis is typically used to aid radiologists in screening trials; therefore, both methods are used in clinical practice. Hence, the feasibility and efficacy of computer-aided diagnosis in computed tomography for lung cancer nodules should be investigated in detail.

OBJECTIVE
In this retrospective study, we aimed to compare the diagnostic parameters of low-dose (< 2 mSv) computed tomography interpretation results using a computer-aided diagnosis system for lung cancer screening with those of a conventional reading system by radiologists.

Ethics approval and consent to participate
The present study involved chart reviews from a database ( A flowchart of the patient selection is shown in Figure 1.

Imaging protocols of chest computed tomography
The detailed protocols for chest computed tomography were based on individual institutional guidelines; there were no differences between the image acquisition protocols and basic characteristics of each center. The basic configuration comprised a computed tomography scanner with at least 16 detector rows. A whole thoracic scan was performed with a one-breath hold at full inspiration. The slice thickness was 1.5 mm, and the image acquisition settings were 80-120 kVp, 22 mA, and the lowest possible collimation on the scanner; the radiation dose was less than 2 mSv.

Computed tomography image analyses
Computer-aided diagnosis system  Solutions AG, Bremen, Germany), including semi-automated segmentation and measurement of the nodules (the diameter of the nodules was automatically measured by automatic segmentation).

Conventional reading system
The computed tomography images were initially screened for interpretations using the institutional conventional system, and other reformats (sagittal or coronal) were accessible to the radiologists, who had a minimum of three years of experience in thoracic imaging, at each hospital. The nodule diameters were measured manually using an electronic caliper (DIGITAL CALIPER, Model No. DT-300/D-300W, Niigata seiki Co., Ltd., Sanjo, Niigata, Japan).

Lung Imaging Reporting and Data System
The chest computed tomography scan interpretations were based on the Lung Imaging Reporting and Data System (Lung-RADS)

Characteristics of participants and nodules
A database of 1,245 patients was retrospectively reviewed.
Among them, a database of 301 patients was viewed using the conventional reading system with the radiologists unaware of the computer-aided diagnosis system data. In addition, the data of 944 patients were viewed using a computer-aided diagnosis system without any subsequent review by a radiologist. Details of the participants' characteristics are presented in Table 2 Table 3.

Lung-RADS category distribution and positivity rates
A total of 20 (7%) and 102 (11%) patients were diagnosed with cancer using the conventional reading and computer-aided diagnosis systems, respectively. The computer-aided diagnosis system detected a higher number of patients with cancer than the conventional reading system (P = 0.0345, Fisher's test). If nodules were measured in a transverse plane, there were no significant differences between the two systems in the number of patients diagnosed with cancer (P = 0.6150, Fisher's test). However, if nodules were measured in any maximum plane with the computeraided diagnosis system, a higher number of patients with cancers were detected than with the transverse plane measurement using the conventional reading or computer-aided diagnosis systems. The details of the per-patient Lung-RADS category distribution screening results for lung cancers are presented in Table 4.
A total of 17 (8%) and 154 (15%) nodules were diagnosed using the conventional reading and computer-aided diagnosis systems, respectively (P = 0.0035, Fisher's test). If nodules were measured in a transverse plane, there were no significant differences between the two systems in the number of nodules diagnosed with cancer (P = 0.6921, Fisher's test). However, if nodules were measured in any maximum  plane with the computer-aided diagnosis system, then higher numbers of cancerous nodules were detected compared with the transverse plane measurement using the conventional reading or computer-aided diagnosis systems. The details of the per-nodule Lung-RADS category distribution screening results and lung cancers are presented in Table 5.

Diagnostic parameters
Sensitivity and positive predictive values were higher if nodules were measured in any maximum plane of the computer-aided diagnosis system compared with that measured in any plane of any system. The sensitivity, specificity, and positive predictive values did not differ between the transverse plane in the conventional reading and computer-aided diagnosis systems, transverse plane in the conventional reading system, and maximum orthogonal plane in the computer-aided diagnosis system. The details of the diagnostic parameters for the imaging interpretation systems for lung cancer are presented in Table 6.

DISCUSSION
This study revealed that the sensitivity and positive predictive values were higher if nodules were measured with the computeraided diagnosis system than those measured with the conventional reading system. The diagnostic parameter results of the current study are consistent with those of previous retrospective studies. 5,15 Small nodules missed using a conventional reading system can be detected by the computer-aided diagnosis system. 15,19 The computer-aided diagnosis system facilitates the diagnosis of cancerous nodules in patients at risk of lung cancer. We found that the specificity and negative predictive values were the same when nodules were measured with the computer-aided diagnosis or conventional reading systems. There was a difference in nodule sizes measured by the two systems. The radiologists did not measure oversized nodules, and the computer-aided diagnosis system did not measure undersized nodules. Nodule size and the risk of lung cancer are separate issues that require investigation.
In this study, we found that the Lung-RADS screening rate per patient was higher with the computer-aided diagnosis system than that of the conventional reading system. In addition, the Lung-RADS screening rates per nodule differed between imaging interpretation systems for lung cancer. The per patient and per nodule Lung-RADS screening rates in the current study were inconsistent with those of retrospective studies. 5,15 The increased diagnosis of small nodules with cancer resulted in a higher per patient Lung-RADS screening rate. Moreover, the increased diagnosis rate of small nodules significantly changed the per-nodule Lung-RADS screening rate. The use of data from more than one institution, heterogeneity of the patients, 20 and higher numbers of involved radiologists 21 may explain the contradictory results between imaging interpretation systems for lung cancer in the current study and those of other retrospective studies. 5,15 A computer-aided diagnosis system is a more accurate tool for lung cancer screening among at-risk patients.
We found that the diagnostic parameters did not differ between the transverse plane in the conventional reading and   Results were considered significant if the P value was less than 0.05. CRS = conventional reading system; CAD = computer-aided diagnosis system; TP = transverse plane; MOP = maximum orthogonal plane; AMP = maximum plane; PPV = positive predictive value; NPV = negative predictive value.
computer-aided diagnosis systems, transverse plane of the conventional reading system, and maximum orthogonal plane of the computer-aided diagnosis system. The results of the different planes using the computer-aided diagnosis system in the current study were consistent with those of a previous retrospective study. 5,22 Lung cancer can be missed by radiologists using computer-aided diagnosis systems. 11  The current study revealed significantly fewer pure ground nodules and significantly more solid nodules among patients evaluated by the computer-aided diagnosis system compared with patients evaluated by the conventional reading system. The pure-ground and solid nodule results observed in the current study were consistent with those of other retrospective studies. 5,15 A computer-aided diagnosis system has less sensitivity for ground-glass nodules than that of the conventional reading system. 16 Solid nodules that can be detected by a computer-aided diagnosis system are sometimes missed by radiologists. 15,19 The conventional reading system is recommended for pure-ground nodules, whereas computer-aided diagnosis systems are recommended for solid nodules in lung cancer screening among at-risk patients.
We also found insignificant differences in part-solid nodules between patients evaluated with the computer-aided diagnosis and conventional reading systems. The performance results of the imaging systems for the detection of part-solid nodules in the current study were consistent with those of a prospective multicenter study. 24 The conventional reading system showed comparable performance to the computer-aided diagnosis system for part-solid nodules.
This was an interesting study on a highly relevant topic that included a large database with follow-up data on malignancy diagnoses. This study had some limitations, mainly its retrospective design (the datasets of the conventional reading and computer-aided diagnosis systems were different) and lack of cross-sectional analysis. It may be more valuable to compare the performance of both systems using the same dataset. However, the gold standard (biopsy, surgical pathology, or position emission tomography) has not yet been described. This study noted that the data were organized according to a local "risk prediction model" established for a single institution. This is problematic as it did not translate to other universally standardized classifications (United States Preventive Services Task criteria). There was an apparent difference between the small number of cases (n = 301) read by radiologists and an entirely different large (n = 944) set of cases read by the computer. Consequently, this study included the two separate, albeit overlapping, issues of diagnosis and measurement. This might be responsible for the overall differences between the radiologists' and computer's results. A possible justification for this is that the study included clinical features, which showed broad similarities between the patients diagnosed by radiologists and patients diagnosed by computer (P > 0.05). This study used size in maximum length rather than volume, which is not conventionally used when screening populations in the United Kingdom. The possible justification for this is that nodule diameter or volume can be used for lung cancer screening. 25 When comparing radiologist interpretations and computer-aided diagnoses it is critical to use the same images. Given that there were two different image sets in this study, it was not possible to validate their performance because there were many different variables between the two groups. Therefore, the increased diagnosis of lung cancer using a computer-aided system may also reflect differences in underlying risks among patients.

CONCLUSIONS
This study validates a commercial computer-