Interexaminer agreement in caries radiographic diagnosis by conventional and digital radiographs

Avaliação interexaminadores no diagnóstico de cáries por meio de radiografias convencionais e digitais

ORIGINAL ARTICLES

Interexaminer agreement in caries radiographic diagnosis by conventional and digital radiographs

Avaliação interexaminadores no diagnóstico de cáries por meio de radiografias convencionais e digitais

Anna Sílvia Penteado Setti da Rocha^I; Solange Maria de Almeida^II; Frab Norberto Bóscolo^II; Francisco Haiter Neto^II

^IDDS, MSc, Oral Radiology, Department of Physics, Federal Technologic University of Parana, Brazil

^IIDDS, PhD, Oral Radiology, Department of Oral Diagnosis, Piracicaba Dental School, State University of Campinas, Brazil

^{Corresponding address}

ABSTRACT

The purpose of this research was to compare two digital storage phosphor systems and conventional film, as well as verify the interexaminer agreement in radiographic diagnosis on occlusal and proximal cavities. Two digital systems and conventional radiographic film were used to evaluate 144 tooth surfaces with and without cavities; the radiographs were analyzed and scored in a scale of 4 points. Thirteen undergraduates and an oral radiologist participated as examiners. A light microscopy analysis was accomplished in order to validate the research. The results showed that there was no significant difference between the radiographic systems, however all of them had significant differences when compared to light microscopy, except for the Digora system on the proximal surface when the radiologist was the examiner. In the interexaminer evaluation, a moderate agreement level was obtained, and a fair to moderate level was obtained between the students and the oral radiologist. Regarding the validation (sensitivity, specificity, positive predict and negative predict) the values were similar between students and the oral radiologist, except for the sensitivity value on the occlusal surface. The students had the highest number of false-positive results and the oral radiologist the highest number of false-negative results.

Uniterms: Dental caries; Radiographic image enhancement; Radiographic, bitewing.

RESUMO

O objetivo desta pesquisa foi comparar dois sistemas digitais de armazenamento de fósforo e filme convencional, assim como verificar a concordância interexaminadores em diagnóstico radiográfico de cáries oclusais e proximais. Dois sistemas radiográficos digitais e filme convencional foram usados para avaliar 144 superfícies dentais com e sem cavitação; as radiografias foram analisadas e classificadas em uma escala de 4 pontos. Participaram como examinadores 13 alunos de graduação e um radiologista odontológico. Para validar esta pesquisa foram realizadas análises em microscopia ótica. Os resultados mostraram não haver diferença significante entre os sistemas radiográficos, entretanto, todos tiveram diferenças significantes quando comparados a microscopia óptica, com exceção do sistema Digora® nas superfícies proximais quando o radiologista foi o examinador. Na avaliação interexaminadores, foi obtido concordância moderada, e entre os alunos e o radiologista obteve-se nível sofrível a moderado. Com relação a validade (sensibilidade, especificidade, preditivos positivos e preditivos negativos), os valores foram semelhantes entre os alunos e o radiologista, com exceção dos valores de sensibilidade nas superfícies oclusais. Os estudantes obtiveram um maior numero de resultados falso-positivos e o radiologista odontológico obteve um maior numero de resultados falso-negativos.

Unitermos: Cárie dentária; Radiografia digital; Radiografia interproximal.

INTRODUCTION

Radiographic examination is a necessary complementary means for oral diagnosis. Tooth cavities may be visible and easy to access, which may lead to fast diagnosis, but commonly lesions located at proximal surfaces are not very often detected by clinical examination, requiring radiographic examination to confirm the diagnosis. Occlusal cavities, due to their positioning, may be hidden or just present microcavities, and that also would difficult a correct diagnosis⁹. Tooth cavities exposed to radiographic examination present a radiolucent image showing a loss of continuity of the dental structure. The intraoral radiographic technique is by far the most indicated, allowing finding of two times more cavities at proximal surfaces on posterior teeth than clinical inspection⁶. The intraoral digital radiographic systems came with the advance of dental radiology technology, initially the CCD (Charge Couple Device) and later the phosphorus storage systems in which the sensor is an optical plate coated with phosphorous salt that, when exposed to radiation stores a certain amount of energy on its surface, creating a latent image, which can be seen in a monitor later when receiving a discharge of laser, by conversion of analogical signals into digital. Studies^{12, 3, 8} report that, in order to evaluate a radiographic method and verify the image quality, the capacity of the examiner, professional formation and clinical experience should also be considered. Straub, et al.¹² (1991) state that many research works have been directed towards the improvement of imagery on systems, while the accuracy of examiners has been neglected. The great variation existing between examiners, mainly at radiographic diagnosis of cavities, brings up the interest to research the agreement between examiners of higher or lower experience, by using digital systems and conventional films, evaluating also which of the systems is the most appropriate for diagnosis of proximal and occlusal cavities.

MATERIALS AND METHODS

Radiographic exposure was carried out utilizing the General Electric X-ray machine, model GE-1000, operating with 65 kVp, 10 mA, with exposure time of 0.3 second for the conventional film and 0.25 second for the digital sensors, and a focus-film distance of 20cm. In order to obtain the digital images, the sensors were placed into an opaque cover, for protection from environmental light. After exposure of the sensor to X-ray, the protecting cover was taken off and each sensor was put in its respective laser scanner in order to perform the image reading. The digital images were saved in TIFF format for later interpretation. For radiographic interpretation the images obtained from the three systems were evaluated by 13 senior undergraduate dental students of Piracicaba Dental School/Unicamp and also by a radiologist with more than 10 years of experience. The examiners were trained and advised for method classification of cavity lesions using scores from 0 to 3 where: 0 = no caries, 1 = enamel cavity, 2 = cavity on the enamel-dentin junction, 3 = dentin cavity. The interpretation of conventional radiography was carried out individually in a darkened room, having as light source the light of a viewbox, with 2x magnification lens. The examiners classified the 12 interproximal radiographs according to the established scores, verifying the occlusal and proximal surfaces of the 48 teeth. The digital images of the Digora® and DenOptix® were saved in TIFF format and evaluated in a darkened room using Digora® software for Windows 3.11 and Vix Win 2000. The examiners could only use the controls of brightness and contrast in order to improve the image quality and zoom, allowing magnification of once or twice the original size. Evaluation was accomplished individually by analyzing firstly the radiographic images from the Digora® system and after three days the students evaluated the radiographic images from the DenOptix® system. Later on, the teeth were removed from the mannequin and sectioned through their long axes in mesial-distal direction. The sections were assembled on blades and examined on a light microscope with 32x magnification. For the statistical analyses, the interexaminer agreement was verified by the Kappa test.

The level of agreement between examiners was represented in accordance with Landis and Koch⁷ (1997), where Kappa results below 0 mean a bad level of agreement; from 0 to 0.20 weak level; from 0.21 to 0.40 sufferable; from 0.41- 0.60 regular level; from 0.61 to 0.80 good level and from 0.81 to 0.99 optimum level. In order to check the validity, the certainties (True – positive and negative) and errors (False – positive and negative) were verified by calculating the sensitivity, specificity and the predictive values, positive and negative.

RESULTS

Results show little variation between students, with levels of 0.43 to 0.55. The level of agreement was regular in relation to the three systems evaluated. When Kappa test was carried out between the students and the radiologist, there was a great variation, with levels of 0.28 to 0.54 (Table 1).

The results of the values of sensitivity, specificity and predicted values, positive and negative, for classification of the occlusal and proximal surfaces carried out by the students and by the radiologist can be observed on Tables 2 and 3, respectively.

Table II presents the results of evaluation of the students. For the proximal and occlusal surfaces, the values of specificity and positive predictive were high, varying from 0.96 to 1 and from 0.97 to 1, respectively. As for sensitivity, the students had greater values for the proximal surfaces, with values varying from 0.64 to 0.71, against 0.21 to 0.23 for the occlusal surfaces. The same occurred for the negative predictive values: for the proximal surfaces there was variation from 0.54 to 0.59 and for the occlusal surfaces from 0.05 to 0.08. The higher the sensitivity values, the higher the detection of surfaces with cavities. Therefore, the students detected more cavities on proximal surfaces than on occlusal aspects. About the specificity, the higher the value, the higher will be the detection intact surfaces. In this case, the students had 100% of correct answers, with the exception to the Digora® system, in which for the proximal surfaces there was 96% of correct answers.

Concerning the positive predictive value, it can be interpreted as the proportion of surfaces with cavities among those considered with cavities, and its high values found in the evaluation carried out by the students means that they had 100% of correct answers, with the exception, once again, of the Digora® system for which the proximal surfaces certainty found was 97%, with no false positive diagnosis for the detection of cavities. The negative predictive value is the proportion of intact surfaces among those considered without cavities, in this item the students had more true negative diagnosis on the proximal surfaces than on the occlusal. The variation of negative predictive values among proximal and occlusal surfaces shows that the student, during radiographic evaluation, when in doubt, preferred to classify the surfaces as intact, and that there was a higher frequency on the occlusal surfaces, where the false-negative diagnosis was more frequent. It can be seen on Table III that for the radiologist, the sensitivity values are similar to those of the students when the diagnoses of proximal surfaces is concerned with values varying from 0.56 to 0.71. However, for occlusal surfaces there was a great difference, with variation from 0.46 to 0.64. This greater sensitivity found shows that, on proximal surfaces, the radiologist was able to detect more cavities than the students. For the specificity, there was also 100% of correct answers for the occlusal surfaces, excluding the DenOptixÒ digital system, which detected only 66% of the intact surfaces. For the proximal surfaces the specificity was slightly lower than that found by the students, varying from 86 to 93%, even so these values are not considered high. As for the positive predictive values, these varied from 0.95 to 1 and from 0.90 to 0.96 for the occlusal and proximal surfaces, respectively. The negative predictive values were similar to those of the students, varying from 0.46 to 0.58 for the proximal surfaces. For the occlusal surfaces, the radiologist also obtained a great number of false-negative values varying from 0.07 to 0.15.

DISCUSSION

Straub, et al.¹² (1991) state that the radiographic examination depends on physical and psychological factors, and therefore, in evaluating a determined radiographic system, one should take in consideration, besides the image quality, the visual accuracy of the observer. The importance of the variability of diagnosis was discussed in the works by Stanley, et al.¹¹ (1993) and Sewell, et al.¹⁰ (1997) which led us to verify the agreement between examiners, the sensitivity, specificity and positive and negative predictive values among the undergraduate students and one experienced radiologist.

As for the Kappa test, our results differ from the ones presented by Tosoni¹⁴ (1998) where the author found the level of agreement among examiners varying from regular to good for the radiographic diagnoses of proximal and occlusal cavities. However, it should be highlighted that in the referred work the two research evaluators were professionals with the same level of experience and in this present work the comparison was carried out in a group of inexperienced students and a professional with renowned experience.

When comparing our results of specificity and sensitivity for the classification of radiologist with the results by other authors, it was noticed that when proximal surfaces were analyzed by means of conventional radiographs, the works by Tosoni¹⁴ (1998); Hintze, et al.⁴ (1998) and Svanes, et al.¹³ (2000), report values of sensitivity varying from 0.48 to 0.80 and the values for specificity from 0.84 to 1. Our research found values of 0.71 for the sensitivity and 0.93 for the specificity, which are within the range presented by the referred authors. Analyzing the occlusal surfaces by means of conventional radiographs, the works by Nytum, et al.⁹ (1992); Ketley and Holt⁵ (1993); Espelid, et al.² (1994); Cayley and Holt¹ (1997) and Tosoni¹⁴ (1998) found that the values of sensibility and sensibility varied from 0.50 to 1 and from 0.48 to 1, respectively.

As it was shown, there is a great variation in the sensitivity values. However, it is noticeable that the values found in this present work are very similar to those found by Tosoni¹⁴ (1998), once the sensitivity value in our work is 0.57 and in the work by Tosoni¹⁴ (1998) it was 0.50. As for the specificity, both the present and the study of Tosoni¹⁴ (1998) found a value of 1. Analyzing the sensitivity values found in this research for the analysis carried out by the radiologist, but utilizing the Digora® digital system, it can be said that authors like Tosoni¹⁴ (1998) and Svanaes, et al.¹³ (2000), utilizing the same system, found sensitivity values smaller than the specificity values, varying from 0.86 to 1. This research found values of 0.71 for sensitivity and 0.93 for specificity. When our results of occlusal surface diagnosis are analyzed by means of the Digora® digital system, having as examiner the radiologist, our results are not very similar to those of Tosoni¹⁴ (1998), which utilizing the same digital system, and also experienced examiners, found values of 0.50 for sensitivity and 1 for specificity. An interesting fact found was that the undergraduate students had in general slightly higher specificity values than the radiologist, and the opposite occurred for the sensitivity values.

Once more these results have shown that when there was doubt in the radiographic interpretation, the students preferred to suggest intact surfaces, while the radiologist preferred to suggest the presence of cavities. Perhaps this shows that the students, by the lack of experience and confidence, prefer not to suggest pathology and consequently more false-negative diagnoses appear. The professional, on the contrary, for being more in contact with examination and in a way induced to find something different from abnormality, prefers to make more false-positive diagnoses.

CONCLUSION

Based on the methodology employed in this work and on the results evaluated and discussed, it is possible to conclude that the evaluation among the examiners (students) obtained a regular level of agreement and between the students and the radiologist the level was sufferable to regular. With relation to the validity, the values of sensitivity, specificity, positive predictive and negative predictive values were similar between the students and the radiologist, with the exception of sensitivity values for the occlusal surfaces. There was no statistically significant difference between the Digora® and DenOptix® systems and the conventional film in the diagnosis of occlusal and proximal cavities; when the systems are compared to light microscopy, there was a significant difference, with the exception of the Digora® system for the evaluation of proximal surfaces carried out by the radiologist.

Received: March 16, 2004

Modification: December 09, 2004

Accepted: March 11, 2005

• 1- Cayley AS, Holt RD. The influence of audit on the diagnosis of occlusal caries. Caries Res. 1997;31(2):97-102.
• 2- Espelid I, Tveit AB, Fjelltveit A. Variations among dentists in radiographic detection of occlusal caries. Caries Res. 1994;28(3):169-75.
• 3- Hintze H, Christoffersen L, Wenzel A. In vitro comparison of Kodak ultra-speed, ektaspeed, and ektaspeed plus, and Agfa M2 comfort dental x-ray films for the detection of caries. Oral Surg Oral Med Oral Pathol Oral Radiol Endod. 1996;81(2):240-4.
• 4- Hintze H, Wenzel A, Danielsen B, Nyvad B. Reability of visual examination, fibre-optic transillumination, and bite-wing radiography, and reproducibility of direct visual examination following tooth separation for the identification of cavitated carious lesions in contacting approximal surfaces. Caries Res. 1998;32(5):204-9.
• 5- Ketley CE, Holt RD. Visual and radiographic diagnosis of occlusal caries in first permanent molars and in second primary molars. Br Dent J. 1993;174(10):364-70.
• 6- Kidd EAM, Pitt NB. A reappraisal of the value of the bitewing radiograph in the diagnosis of posterior approximal caries. Br Dent J. 1990;169(7):195-200.
• 7- Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1997;33(1):159-74.
• 8- Naitoh M, Yuasa H, Toyama M, Shiojima M, Nakamura M, Ushida M, et al. Observer agreement in detection of proximal caries with direct digital intraoral radiography. Oral Surg Oral Med Oral Pathol Oral Radiol Endod. 1998;85(1):107-12.
• 9- Nytum RB, Raadal M, Espelid I. Diagnosis of dentin involvement in occlusal caries based on visual and radiographic examination of the teeth. Scand J Dent Res. 1992;100(3):144-8.
• 10- Sewell CMD, Pereira MF, Varoli OJ. Princípios de produção de imagem digitalizada. RPG. 1997;4(1):55-8.
• 11- Stanley MD, Kantor ML. Digital radiology facts and fictions. J Am Dent Assoc. 1993;124:39-47.
• 12- Straub WH, Gur D, Good BC. Visual acuity testing of radiologists. Is it time? Amer J Roentgenol. 1991;156(5):1107-8.
• 13- Svanaes DB, Moystad A, Larheim TA. Approximal caries depth assessment with storage phosphor versus film radiography. Evaluation of the caries-specific oslo enhancement procedure. Caries Res. 2000;34(6):448-53.
• 14- Tosoni GM. Qualidade diagnóstica das imagens radiográficas convencionais e digitais da cárie dentária. Bauru; 1998. 175p. Tese (Doutorado)- Faculdade de Odontologia de Bauru, Universidade de São Paulo.

Publication Dates

• Publication in this collection
  20 Jan 2006
• Date of issue
  Dec 2005

History

• Accepted
  11 Mar 2005
• Reviewed
  09 Dec 2004
• Received
  16 Mar 2004