Reading and interpretation of chest X-ray in adults with community-acquired pneumonia

ORIGINAL ARTICLE

Reading and interpretation of chest X-ray in adults with community-acquired pneumonia

Diana Carolina Moncada^I; Zulma Vanessa Rueda^II; Antonio Macías^III; Tatiana Suárez^IV; Héctor Ortega^V; Lázaro Agustín Vélez^VI

^IMD, Grupo Investigador de Problemas en Enfermedades Infecciosas (GRIPE), Facultad de Medicina, Universidad de Antioquia, Medellín, Colombia

^IIMD; PhD Student of Epidemiology, GRIPE, Facultad de Medicina, Universidad de Antioquia, Medellín, Colombia

^IIIMD, GRIPE, Facultad de Medicina, Universidad de Antioquia, Medellín, Colombia

^IVMD, Radiologist, Hospital Universitario San Vicente Fundación and Universidad de Antioquia, Medellín, Colombia

^VMD, Pneumologist, Clínica Cardiovascular Santa María and Universidad de Antioquia, Medellín, Colombia

^VIMD, Specialty on Internal Medicine, Sub-specialty on Infectious Diseases; Professor of Medicine, GRIPE, Infectious Diseases Section, Facultad de Medicina, Universidad de Antioquia, Medellín, Colombia

^{Correspondence to}

ABSTRACT

INTRODUCTION: Traditional reading of chest X-rays usually has a low prognostic value and poor agreement.

OBJECTIVE: This study aimed to determine the interobserver and intraobserver agreement using two reading formats in patients with community-acquired pneumonia, and to explore their association with etiology and clinical outcomes.

METHODS: A pulmonologist and a radiologist, who were blind to clinical data, interpreted 211 radiographs using a traditional analysis format (type and location of pulmonary infiltrates and pleural findings), and a quantitative analysis (pulmonary damage categorized from 0 to 10). For both, the interobserver and intraobserver agreement was estimated (Kappa statistic and intraclass correlation coefficient). The latter was assessed in a subsample of 25 radiographs three months after the initial reading. Finally, the observers made a joint reading to explore its prognostic usefulness via multivariate analysis.

RESULTS: Seventy-four chest radiographs were discarded due to poor quality. With the traditional reading, the mean interobserver agreement was moderate (0.43). It was considered good when the presence of pleural effusion, and the location of the infiltrates in the right upper lobe and both lower lobes, were evaluated; moderate for multilobar pneumonia; and poor for the type of infiltrates. The mean intraobserver agreement for each reviewer was 0.71 and 0.5 respectively. The quantitative reading had an agreement between good and excellent (interobserver 0.72, intraobserver 0.85 and 0.61). Radiological findings were neither associated to a specific pathogen nor to mortality.

CONCLUSION: In patients with pneumonia, the interpretation of the chest X-ray, especially the smallest of details, depends solely on the reader.

Keywords: radiography, thoracic; pneumonia; reproducibility of results.

INTRODUCTION

Community-acquired pneumonia (CAP) is a common infection and a frequent cause of medical consultation, hospitalization and death in all countries around the world. Its global incidence ranges between 150 and 1500 cases per 100,000 inhabitants/year,¹ and it is estimated to have a mortality rate of 20 cases per 100,000 inhabitants/year. In the United States it is the sixth cause of death in adults and the first related to infectious etiology, while in Colombia, according to the PAHO (Pan American Health Organization), that rate was 52.2/100,000 in 2008.²

Because the clinical presentation can be very variable, the diagnosis is based on the presence of new pulmonary infiltrates in the chest X-ray. Its traditional reading describes the presence, location and type of the infiltrates, and identifies complications associated, such as pleural effusion, formation of abscesses or cavitations. However, this kind of interpretation has some limitations such as low sensitivity and specificity, poor ability for predicting the etiological agent, and a poor to moderate interobserver agreement.^3-8 In an effort to standardize the criteria to evaluate the severity of the infection and the effectiveness of the antimicrobial treatment used, the Japanese Society of Chemotherapy in 1999 suggested using a grading system based on the extension of the pulmonary injury displayed in the chest X-ray.⁹ A later study demonstrated that scores of > 6 at hospital admission were associated with higher mortality.¹⁰

Due to the importance of chest X-rays as a diagnostic tool in CAP and its potential use as predictor of etiology, mortality and complications, it is convenient to assess the interpretation performance using different reading tools. Therefore, we planned this study with the following goals: I) to describe the radiological characteristics of community-acquired pneumonia, and determine the intraobserver and interobserver agreement level in the chest X-ray interpretation between two trained readers, using two different reading methods, qualitative and quantitative; and II) to determine if there is any association between one or both reading methods and the need for intensive-care unit (ICU) admission, death and the specific etiological agent identified in each case.

MATERIALS AND METHODS

Population

This paper is part of a macro cohort study carried out in 11 Health Institutions of medium and high complexity level in the metropolitan area of Medellín, Colombia, since July 2005 to October 2006. CAP patients above 18 years of age who needed hospitalization were included consecutively and prospectively. This study was approved by the Ethics Committee of Universidad de Antioquia and the Internal Board of all the participant institutions. Patients with tuberculosis, who have had symptoms longer than 15 days, or suggestive radiological findings of chronic forms of that illness, were excluded from the study. All patients signed a consent form in accordance with the resolutions of the current legislation (Resolución 008430 del Ministerio de Salud, Colombia 1993).

In order to be part of this study, chest X-rays of all patients ought to show pulmonary infiltrates. All demographical, clinical, laboratory and microbiological data collected during hospitalization were considered in this analysis, including complications and mortality. Search of the pathogen responsible for pneumonia was investigated through routine cultures of conventional bacteria, paired serological testing for atypical bacteria (Mycoplasma pneumonia, Chlamydophila pneumonia, Legionella pneumophila, and Coxiella burnetii) and respiratory virus (Influenzavirus A/B; Parainfluenza 1,2,3; Respiratory Syncytial Virus and Adenovirus), and antigens detection in nasopharynx (respiratory viruses) and urine (Streptococcus pneumonia and L. pneumophila serogroup 1).

Radiological interpretation

The chest X-rays were read by two researchers, a pulmonologist (HO) and a radiologist (TS), both with more than 10 years of experience. All radiographs that were not in digital format, or radiographs which quality was considered inappropriate or poor by the researchers were excluded, in order to avoid bias in the interpretation. Separately, each observer made a blind reading without any clinical information of the patient. Two formats were used for this readings: I) conventional, in which the presence, localization and type of infiltrates were evaluated, discriminating between alveolar infiltrates (with or without air bronchogram), and interstitial infiltrates (of nodular, reticular or mixed types); unilobar or multilobar infiltrates, and pleural effusion; and II) quantitative scale, which assigns a score according to the extension of the pulmonary injury, determined by the number of affected intercostal spaces.⁹

All final images available were used in order to assess the interobserver agreement. Three months after the observers had performed their readings in both formats, a sample of 25 radiographs was selected among those with the best quality. Previously, this sample was recoded for a new reading, made again by the two researchers, in order to assess the intraobserver agreement. In addition, each observer was asked if they thought the chest X-rays suggested the presence of a particular respiratory pathogen. Finally, to be able to associate radiological findings with outcomes of CAP and the class of etiological agent involved, a joint reading by the two observers of all available radiographs was done. It should be noted that the severity of pneumonia as an outcome was not evaluated because the extension of pulmonary injury assessed through the chest X-rays is a severity criteria by itself.¹

Statistical analysis

The percentage of agreement and Cohens Kappa coefficient were used to calculate the intraobserver and interobserver concordance on the conventional reading format. For comparison purposes with other studies,^11-14 the mean kappa was calculated for the radiographic features evaluated in order to estimate the overall agreement in both cases. For the quantitative format, the intraobserver agreement was evaluated by calculating the intraclass correlation coefficient of mixed models, and for the interobserver agreement, the intraclass correlation coefficient of random models was used. The agreement for both formats was interpreted as poor when the calculated values were between 0 and 0.4, moderate between 0.4 and 0.6, good between 0.6 and 0.8 and excellent > 0.8. Negative values were interpreted as equal to 0.0.

To evaluate the association between radiological findings and clinical outcomes (need for ICU admission and death), a multiple logistic regression analysis was done using data from the joint reading done by both observers. Variables with p-value < 0.25 entered the model, using the stepwise method for selection of variables. A p-value < 0.05 was considered significant. Finally, the power of the chest X-ray as a predictor of the etiologic agent of CAP was explored, evaluating the agreement between the findings in the joint reading and the pathogen microbiologically identified. Towards this end, the etiologic agents were grouped into pyogenic bacteria, atypical bacteria, respiratory virus, tuberculosis, mixed etiology and without germ. The data analysis was performed using the statistical package PASW Statistics_® version 18.0 (SPSS Inc., Chicago, Il, USA).

RESULTS

A total of 211 patients with CAP who had chest X-rays available in a digital format were evaluated; 74 of those were excluded because the readers considered the chest X-rays of poor quality for this study. At the end, the analysis was done with 137 X-rays. Table 1 describes the main demographic, clinical and etiological characteristics of these patients. In general, they were middle age individuals, predominantly men, two out of five were smokers, most had underlying diseases, about 50% met the criteria for severe pneumonia,⁸ and 10.9% died during hospitalization. In a third of the patients an etiologic agent could not be identified. Pyogenic bacteria, atypical bacteria and respiratory viruses were, in this order, the pathogens most frequently involved in the genesis of pneumonia. In a quarter of cases it was considered that the etiology was mixed, and five cases of acute pneumonia by tuberculosis were documented.

The joint reading of chest X-rays allowed to define the main findings on the images evaluated. As shown in Table 2, the vast majority of the pulmonary infiltrates were considered as alveolar type, two-thirds had air bronchogram, one in three patients had pleural effusion, usually on one side, and one in four multilobar disease. The lower lobes were the most affected, the right one more than the left one, and according to the quantitative reading format, approximately one third of patients had scores of > 6.

In assessing the conventional reading of chest X-rays, the overall interobserver agreement was moderate. However, it was observed that although the percentages of agreement in most of the evaluated variables were greater than 80%, the agreement was poor for the type of infiltrates in almost all cases, and moderate to good when identifying the presence of parenchymal or pleural disease and their location. As shown in Table 3, the best Kappa coefficient was observed for the variables pleural effusion and location of the infiltrates in the right upper lobe and both lower lobes. The agreement in the quantitative reading between both observers was good (0.72, 95% CI 0.42 to 0.84).

The general intraobserver agreement in the conventional reading was considered good for reader 1 and moderate for reader 2. When the first of them judged the presence of interstitial infiltrates, bilateral pneumonia, pleural effusion and the location of infiltrates, it was considered between good and excellent. For the second reader, on the other hand, the variables absence of air bronchogram, presence of pleural effusion, and infiltrates in the upper and lower right lobes, had the best Kappa scores (Table 4). When we evaluated the intraobserver agreement for the quantitative reading format with both readers, it was 0.85 (95% CI 0.59-0.93) and 0.61 (95% CI 0.16-0.82), respectively for the first and second readers.

According to multivariate analysis, no association was found between the radiological findings from the joint reading and the estimated clinical outcomes (need for ICU admission and death), or with a specific group of agents (pyogenic bacteria, atypical bacteria, respiratory virus, tuberculosis, mixed etiology and without germs) by either of the two formats evaluated.

DISCUSSION

This study supports the notion that in adult patients with community-acquired pneumonia, the qualitative reading of the chest X-ray is highly dependent on the observer, and that there is no association between radiological findings and the etiologic agent or clinical outcomes.

Our results are clear. The interobserver and intraobserver agreement was considered moderate to good only when the most evident images were evaluated, such as the presence of pleural effusion or location and extent of pulmonary injury. On the other hand, it was poor when evaluating the type of infiltrates, and the presence or absence of air bronchogram. By contrast, when evaluating agreement with the quantitative reading format results were considered good to excellent (interobserver 0.72, intraobserver 0.61 and 0.85 respectively).

Several previous studies from different latitudes, some considered classical and some very recent, have addressed this issue in patients with pulmonary infiltrates, either secondary to pneumonia or to other noninfectious causes. Most of them have assessed the agreement, or the percentage of agreement, between radiologists, pulmonologists, pediatricians, internists, emergency department specialists, residents and medicine students, in both children^3,4,11,13-17 and adults,^5-8,12,18-25 and the association with the etiologic agent,^8,15-17 clinical presentation and outcomes.^20,23 Several of these studies have also estimated sensitivity, specificity and predictive values of chest X-rays for the diagnostic of pneumonia, its etiology and mortality.^5,6,15,16,19

When the above mentioned studies evaluated the interobserver agreement in order to define the presence of pneumonia in both children and adults, the results were poor to moderate (< 0.4 to 0.59),^3-6,11-14,24 while the percentage of agreement ranged between 41% and 87%.^18,21-23,25 These values, however, depended on the specialty, expertise and level of training of readers,^{3-6,11,14,16,18,19,21,24,25} and the radiological characteristics evaluated. In this regard, Albaum et al.⁷ and Boersma et al.⁸ encountered very similar findings, including the poor interobserver agreement for the type of infiltrates (< 0.3) and presence or absence of air bronchogram (< 0.31), poor to moderate agreement for pleural effusion (< 0.46), and moderate to good for multilobar pneumonia (< 0.67) and location of infiltrates (< 0.77). Also, Sarria et al.¹¹ and Venera et al.³ found poor to moderate agreement for the variables mentioned in pediatric populations.

The variations reported in chest X-rays interpretation highlight how the described findings, especially when it comes to fine details, depend significantly on the observer. Such variations have also been reported when evaluating images for noninfectious diseases, such as those of cardiovascular origin. In this regard, Young et al.²⁵ pointed out that patchy opacities are the cause of major disagreements, and that some findings such as air bronchograms, atelectasis and chronic obstructive lung disease are usually not recognized by the nonradiologists, while Herman et al.²² observed that false positive findings in X-ray readings are mainly due to vascular redistribution and other densities associated with congestive heart failure. To further emphasize the challenges of interpreting chest X-rays, Syrjala et al.²⁰ found that 30.8% of pneumonia identified by high resolution computed tomography was not identified in chest X-rays.

Regarding the intraobserver agreement, this was evaluated only in five of the above mentioned studies.^11-15 It ranged from moderate to excellent (0.54 to 0.93), very close figures to those found in our study as well as in others that assessed it, as reported recently by Johnson et al.¹⁴ However, only one of these papers has assessed intraobserver agreement of radiological characteristics evaluated in 20 healthy individuals and 20 with tuberculosis.¹² As said before with the interobserver agreement, Esquivel et al.¹² observed the lowest agreement when evaluating the finest details, such as the presence of complications or type of infiltrates, and higher values for more evident findings, such as the location of the infiltrates or the presence of pulmonary cavitations.

Other studies have evaluated the association between radiological findings and the etiologic agent. Several of them have found that X-rays can be useful in distinguishing between bacterial and viral pneumonia using different tools such as the clinical presentation, response to treatment, microbiological studies and scales of scores based on radiological findings.^15,16 By contrast, other authors such as Bettanay²⁶ and Virkki,²⁷ as well as us, did not find such association. In fact, both suggest that although the presence of consolidation is reliable for diagnosing pneumonia, it should not be used to assume the existence of a bacterial infection. Also, Swingler¹⁷ concludes, based on a systematic review of the literature that the chest X-ray is not useful to differentiate between bacterial and viral etiology.

Finally, previous studies have also evaluated the association between the extent of pulmonary disease, the presence of pleural effusion and mortality.^7,10,28-30 At least two of them^7,28 found an association between the presence of pleural effusion and 30-day mortality. Additionally, Albaum et al.⁷ found an association with multilobar pneumonia, and Hasley²⁸ suggests that the presence of bilateral pleural effusions may be an indicator of congestive heart failure or severe pneumonia. At the same time, Daley et al.,²⁹ Marrie et al.³⁰ and Fujiki et al.¹⁰ found that the extent of pulmonary disease, given by the number of lobes involved, is directly related to increased severity and mortality. It should be noted that the absence of such association in our study could be explained by the low frequency of events in each of the outcomes tested.

Our study has several limitations. Firstly, only two researchers evaluated the chest X-rays. However, their expertise and the similarity of our results when compared with those of previously published articles, in which the number of researchers varies considerably (between 2 and 72), suggests that this factor does not have an effect on the outcome of the agreement. Secondly, the chest X-rays were digital photographs of the original films, which may have distorted the radiological image quality. Moreover, the lateral projection was not available for all the patients. Nonetheless, the fact that the researchers themselves selected the films that ended up included in the study suggests that the results were not affected by their quality.

CONCLUSION

Our findings confirm that in patients with pneumonia, the chest X-ray interpretation depends on the observer - mainly the interpretation of the finest details - and therefore it should be read with knowledge of the patients clinical data. Also, since X-rays are not an adequate media to predict the etiologic agent, pertinent microbiological tests are required to identify the pathogen responsible for the patient's pneumonia.

ACKNOWLEDGEMENT

We give an acknowledgement to Luis Carlos Orozco-Vargas, M.D., MSc, associated professor, Santander's Industrial University (UIS) Nursing School by his data analysis support.

Submitted on: 03/28/2011

Approved on: 04/18/2011

We declare no conflict of interest.

Financial Support: Colciencias (Colombian Administrative Department of Science, Technology and Innovation, contract number 326-2004), and Universidad de Antioquia (code of Medical Research Institute 2495).

• 1. Mandell Lionel A, Wunderink Richard G, Anzueto A, et al. Infectious diseases society of america/american thoracic society consensus guidelines on the management of community-acquired pneumonia in adults. Clin Infect Dis. 2007;44:S27-S72.
• ²
  Situación de salud en las Américas. Indicadores básicos 2008. Organización Panamericana de la Salud. Vigilancia de la salud, prevención y control de las enfermedades. Información de la salud y análisis. Washington D.C: Organización Panamericana de la Salud; 2008 [cited 2009 Sept 24]; Available from: http://www.bvs-vspcol.bvsalud.org/php/level.php?lang=es&component=22&item=131
• 3. Venera AD, Rincón DA, Torres LI, Arango M. Concordancia interobservador de hallazgos en radiografía de tórax pediátrica. Rev Fac Med Univ Nac Colomb. 2004;52:192-98.
• 4. Limón AE, Moreno L, Valenzuela A, Carreón J, Medina JC, Sicilia E. Concordancia en la interpretación de radiografías de tórax, en niños. Rev Mex Pediatr. 1995;62:219-23.
• 5. Gatt ME, Spectre G, Paltiel O, Hiller N, Stalnikowicz R. Chest radiographs in the emergency department: is the radiologist really necessary? Postgrad Med J. 2003;79:214-17.
• 6. Rico FG, Múgica JJ, Fernández CG, González A. Concordancia diagnóstica en la interpretación de radiografía de tórax entre residentes de tercer año de radiología y neumología. Rev Inst Nal Enf Resp Mex. 1998;11:288-91.
• 7. Albaum MN, Hill LC, Murphy M, et al. Interobserver reability of the chest radiograph in community-acquired pneumonia. Chest. 1996;110:343-50.
• 8. Boersma WG, Daniel JM, Lowenberg A, Boeve W-J, Van D. Reliability of radiographic findings and the relation to etiologic agents in community-acquired pneumonia. Respir Med. 2006;100:926-32.
• 9. Saito A, Miki F, Oizumi K, et al. Clinical evaluation methods for new antimicrobial agents to treat respiratory infections: report of the committee for the respiratory system, Japan Society of Chemotherapy. J Infect Chemother. 1999;5:110-23.
• 10. Fujiki R, Kawayama T, Ueyama T, Ichiki M, Aizawa H. The risk factors for mortality of community-acquired pneumonia in Japan. J Infect Chemother. 2007;13:157-65.
• 11. Sarria E, Lima JA, Fisher GB, Menna SS, Flores JA, Sukiennik R. Interobserver agreement in the radiological diagnosis of lower respiratory tract infections in children. J Pediatr. 2003;79:497-503.
• 12. Esquivel CG, Aguirre A, Macías A, Velasco VM, Martinez VA. Concordancia del diagnóstico radiológico en neumología. Med Int Mex. 2003;19:131-36.
• 13. Herreros ML, Barja J, González A, et al. Concordancia en la interpretación de las radiografías de tórax pediátricas. An Pediatr (Barc). 2008;68:1-378.
• 14. Johnson J, Kline J. Intraobserver and interobserver agreement of the interpretation of pediatric chest radiographs. Emerg Radiol. 2010;17:285-90.
• 15. Moreno L, Bujedo E, Robledo H, et al. Validez de la radiografía de tórax para diferenciar etiología bacteriana de viral en niños hospitalizados con neumonía. Arch Argent Pediatr. 2006;104:109-13.
• 16. Ferrero F, Torres F, Noguerol E, et al. Evaluación de dos métodos estandarizados de interpretación de radiografías de tórax en niños con neumonía. Arch Argent Pediatr. 2008;106:510-14.
• 17. Swingler G. Observer variation in chest radiography of acute lower respiratory infections in children: a systematic review. BMC Medical Imaging [serial on the Internet]. 2001; 1(1): Available from: http://www.biomedcentral.com/1471-2342/1/1
• 18. Al aseri Z. Accuracy of chest radiograph interpretation by emergency physicians. Emerg Radiol. 2009;16:111-4.
• 19. Eng J, Mysko WK, Weller GE, et al. Interpretation of Emergency Department radiographs: a comparison of emergency medicine physicians with radiologists, residents with faculty, and film with digital display. AJR Am J Roentgenol. 2000;175:1233-8.
• 20. Syrjälä H, Broas M, Suramo I, Ojala A, Lähde S. High-resolution computed tomography for the diagnosis of communityacquired pneumonia. Clin Infect Dis. 1998;27:358-63.
• 21. Campbell SG, Murray DD, Hawass A, et al. Agreement between emergency physician diagnosis and radiologist reports in patients discharged from an emergency department with community-acquired pneumonia. Emerg Radiol. 2005;11:242-6.
• 22. Herman PG, Gerson DE, Hessel SJ, et al. Disagreements in chest roentgen interpretation. Chest. 1975;68:278-82.
• 23. Krüger PS, Muche RK, Welte MR, et al. Interobserver agreement in the assessment of pulmonary infiltrates on chest radiography in community-acquired pneumonia. Rofo. 2007;179:1152-6.
• 24. Melbye H, Dale J. Interobserver variability inthe radiographic diagnosis of adult outpatient pneumonia. Acta Radiol. 1992;33:79-81.
• 25. Young M, Marrie TJ. Interobserver variability in the interpretation of chest roentgenograms of patients with possible pneumonia. Arch Intern Med. 1994;154:2729-32.
• 26. Bettenay FA, Campo JFd, McCrossin DB. Differentiating bacterial from viral pneumonias in children. Pediatr Radiol. 1988;18:453-4.
• 27. Virkki R, Juven T, Rikalainen H, Svedström E, Mertsola J, Ruuskanen O. Differentiation of bacterial and viral pneumonia in children. Thorax. 2002;57:438-41.
• 28. Hasley PB, Albaum MN, Li Y-H, et al. Do pulmonary radiographic findings at presentation predict mortality in patients with community-acquired pneumonia? Arch Intern Med. 1996;156:2206-12.
• 29. Daley J, Jencks S, Draper D, Lenhart G, Thomas N, Walker J. Predicting hospital-associated mortality for medicare patients: a method for patients with stroke, pneumonia, acute myocardial infarction, and congestive heart failure. JAMA. 1988;260:3617-24.
• 30. Marrie TJ, Durant H, Yates L. Community-acquired pneumonia requiring hospitalization: 5-year prospective study. Rev Infect Dis. 1989;11:586-99.

Publication Dates

• Publication in this collection
  04 Jan 2012
• Date of issue
  Dec 2011

History

• Received
  28 Mar 2011
• Accepted
  18 Apr 2011