Reliability and reproducibility of three-dimensional cephalometric landmarks using CBCT: a systematic review

Abstract

Objective

: The aim of this study was to review the reliability and reproducibility of 3D-CBCT (cone beam computed tomography) cephalometric landmark identification.

Methods

: Electronic databases (Pubmed, Scopus, Web of Science) were searched for papers published from 1998 to October 2014. Specific strategies were developed for each database, with the guidance of a librarian. Two reviewers independently analyzed the titles and abstracts for inclusion. The articles that met the inclusion and exclusion criteria were selected for full-text reading, and the selected articles went through methodological quality evaluation. After the exclusion of repeated articles, the titles of the remaining ones were read and 1,328 of them were excluded. The abstracts of 173 articles were read, of which 43 were selected, read in full and submitted to the inclusion and exclusion criteria. Fourteen articles or studies with reliable methodology and reproducibility remained. The data were collected, organized into figures and analyzed for determination of the reliability and reproducibility of the three-dimensional cephalometric landmarks.

Results

: Overall, the landmarks on the median sagittal line and dental landmarks had the highest reliability, while the landmarks on the condyle, porion and the orbitale presented lower levels of reliability. Point S must be marked in the multiplanar views associated with visualization in 3D reconstruction. Further studies are necessary for evaluating soft tissue landmarks.

Cone-beam computed tomography; Reproducibility of results; Cephalometry; Three-dimensional imaging

INTRODUCTION

Traditional two-dimensional (2D) cephalograms are widely used. However, this method has been questioned due to its limitations, such as the generation of 2D images from three-dimensional (3D) structures, the superimposition of the right and left sides on the median sagittal plane, image enlargement and distortion, and the influence of head position during image acquisition²³.

Cone-beam computed tomography (CBCT) has been described as the 3D method of choice for maxillofacial imaging due to the lower cost of the equipment and test, the reduction in radiation level in comparison with multi-slice computed tomography, high resolution for bones and teeth, and the possibility of obtaining the whole set of traditional orthodontic images in only one exposure¹⁴.

To maximize the advantages of the diagnostic power offered by CBCT, the development of 3D cephalometric analyses requires adequate operational definitions of reference points in the three planes, and reliable reproducibility of their identification^2,18,23. Thus, it would be possible to structure a 3D cephalometric analysis based on reliable and reproducible landmarks.

Nevertheless, the literature presents studies with different methodologies, using variable anatomic landmark definitions, and images generated and manipulated by various types of equipment and software. Therefore, the aim of this study was to systematically review the reliability and reproducibility of 3D-CBCT cephalometric landmark identification and to determine the most frequently used anatomic references.

METHODS

The method for this systematic review was based on the PRISMA guidelines (www.prisma-statement.org)¹⁷. The period for this search was limited from 1998 to October 2014. Specific strategies have been developed for each database with a librarian (D.M.T.P.), irrespective of language. A detailed search was conducted in the following electronic databases: Medline/Pubmed, Scopus and Web of Science. Electronic searches were conducted to identify the methodological reliable and reproducible studies assessing three-dimensional cephalometric landmarks identification using CBCT.

The search strategy included appropriate changes in the keywords and followed the syntax rules of each database (Figure 1).Figure 2 outlines the population, exposure, comparisons, and outcomes (PECO format^6,24) and the null hypothesis used for systematic reviews.

Figure 1
Relationship between the databases and search strategies used and the total number of articles found

Figure 2
PECO Format and null hypothesis

Two reviewers (C.O.L. and A.T.S.M.) independently analyzed the list of titles and abstracts for inclusion. All of the articles that were shown to be compatible and related to the research question (Figure 2) were reviewed. Disagreements between the reviewers were solved by means of a consensus meeting and, if appropriate, consultation with a third reviewer (A.F.J.M).

After reading the abstracts, the articles that met the following inclusion and exclusion criteria were read in full. The inclusion criteria were: methodological reliability and reproducibility studies, studies including three-dimensional images, studies including points comprising the maxillo-mandibular complex, studies using CBCT, and studies in humans or dry skulls without markers. The following studies were excluded: case reports, epidemiologic studies, narrative reviews, author opinion articles, studies that used only conventional radiographs or 2D images generated from CBCT, and studies in animals, synthetic samples and dried skulls with markers.

After the evaluation by the inclusion and exclusion criteria, a new selection was made by means of a methodological quality scoring process (Figure 3). The list applied to this study was adapted from methodological evaluations developed in previous study²⁶.

Figure 3
Methodological quality scoring protocol (maximum score: 18 points)

The articles that satisfactorily fulfilled the methodological criteria received a maximum score. When there were two requirements for the maximum score, half the score was attributed to articles that partially fulfilled the criterion, while zero score was awarded for articles that did not meet the criterion.

Before evaluating the studies, the reviewers discussed all of the criteria for analyzing the articles so that there would be consensus about the contents. Each study was scored by the reviewers, and disagreements were resolved by discussions.

The methodological criteria were applied to the studies that were classified with the following scores: high (≥17), moderate (≥11 and ≤16) and low (≤10) methodological quality (Figure 4).

Figure 4
Methodological quality scores of the selected articles

RESULTS

Figure 5 illustrates the PRISMA flow diagram of the search strategy performed and the number of studies selected at each stage.

Figure 5
PRISMA flow diagram of the search results from the databases

The established search strategy identified 1,621 articles. After excluding the repetitions, 1,501 articles remained. After reading the titles, 1,328 articles were excluded, as they were unrelated to the research. Many articles evaluated the height and thickness of the alveolar ridge, root resorption, airways or osseointegrated implants.

The abstracts of 173 articles were read, of which 43 were selected and read in full. The inclusion and exclusion criteria were applied to the 43 selected articles. The excluded articles did not identify landmarks comprising the entire face, were performed in animals, evaluated cephalometric measurements, used synthetic/artificial samples or only marked points in 2D.

Fourteen articles^{1,7-12,15,16,20-22,25,28} fulfilled the inclusion and exclusion criteria and were evaluated for eligibility, according to the methodological quality evaluation (Figure 4). A summary of the information in the selected articles is illustrated in Figure 6.

Figure 6
Description of the included studies

A meta-analysis using the present results was attempted with the Review Manager (Revman, Cochrane Collaboration) software. However, meta-analysis was not possible due to the limitations related to the methodology of the studies and software features.

DISCUSSION

In spite of the rising use of CBCT in 3D cephalometry, there is still no established cephalometric analysis, especially based on cephalometric landmarks with a high level of reliability and reproducibility. Lou, et al.¹³ (2007) conducted a review similar to our study and concluded that there was a scarcity of studies and that further studies were necessary to determine the precision and reproducibility of the identification of the three-dimensional landmarks in the craniomaxillofacial region using cone-beam computed tomography.

In the present systematic review, a search was made for articles from 1998, the year in which CBCT was developed¹⁹; however, all of the articles selected were published in the last six years. It was possible to identify a satisfactory number of articles to determine the three-dimensional cephalometric landmarks with a greater degree of reproducibility and reliability.

Only four of the included articles^8,9,11,20 did not perform calibration of the evaluators. There are constant errors in cephalometry, and to minimize these errors, the evaluators must be experienced and calibrated. According to Oliveira, et al.²² (2009), the selection of the best location of the landmarks in each coordinate requires time, calibration and careful evaluation.

Six of the articles^{1,11,12,20-22} included in this systematic review used the Intraclass Correlation Coefficient (ICC) as the statistical test. Ludlow, et al.¹⁵ (2009) used the paired t-test; however, according to a study⁴, the correlation coefficient t-value is inadequate for evaluating reliability and reproducibility because it compares the means of the results and not the values individually. According to recent studies^4,5, the Bland-Altman statistical test is the most appropriate test for the comparisons of intra- and inter-observer reliability. Katkar, et al.¹⁰ (2013) used the Euclidean distance, as the image was taken by two distinct tomographs and there was no standardization of the images in their study, which might lead to artificially high results.

The tomographs used in the articles included in this review were the NewTom 3G, i-CAT, Galileos, Accuitomo 3D, 3D eXam, Pax Zenith 3D, Pax Reve 3D, Alphard VEGA and the Hitachi CB MercuRay. In a systematic review on the use of CBCT in the oral and maxillofacial regions, De Vos, Casselman and Swennen³ (2009) found a higher number of articles using the NewTom 9000. Based on the search and inclusion criteria of the present study, we did not find any article using the Newtom 9000, but Meldenik, et al.¹⁶ (2011) compared the Accuitomo 3D with other equipments. Both Newtom 9000 and Accuitomo 3D have a reduced field of view and are not ideal for the analysis of the whole craniomaxillofacial complex.

Katkar, et al.¹⁰ (2013) evaluated the reproducibility and reliability of cephalometric landmarks using the Galileos and i-CAT^® tomographs, and concluded that excellent intra- and inter-observer reliability could be obtained using either equipment. There may be differences in the images obtained using different equipments, such as differences in contrast and resolution, which may influence the visualization and location of landmarks by the operator.

The Dolphin 3D software program was used in six articles^{1,9,10,15,22,25} included in this review. A study²⁷ evaluated the precision in measurements of the airways using different softwares, and therefore was not included in this review. High reliability was found in all of the programs tested. However, Dolphin 3D software was shown to be more reliable than the other software. There is a scarcity of studies that compare the identification of three-dimensional cephalometric landmarks using different software.

Despite the fact that all of the studies examined the entire maxillomandibular complex and not only isolated regions, a great variability of landmarks was observed in the selected experiments. However, the majority of traditional 2D cephalometry reference points were evaluated, highlighting the possibility of three-dimensional evaluation of the left and right sides.

The classification of reliability based on the ICC values varied among the articles included. We did not find any fixed determination for ICC intervals; however, it was consistently stated that values closer to 1 had excellent reliability and reproducibility.

The sella point was evaluated in ten of the included articles^{7,9-12,15,21,22,25,28} with variable reliability. Neiva, et al.²¹ (2014) showed acceptable reliability in 3D reconstruction and poor reliability in the MPR views. Studies^9,25,28 showed that the identification of this point presented a high level of reliability when MPR imaging associated with 3D image reconstruction was used. Because it is a point without defined surface anatomic orientation, it is not a simple procedure to transfer this point to a 3D image. Therefore, we suggest that the sella point be identified in 3D images associated with MPR, whereas its identification in 3D or MPR reconstructions alone should be avoided.

Landmarks located on the midline presented a high level of reliability in some of the studies included^{1,8,9,11,21,22,25,28} in this systematic review and may be safely used in 3D cephalometry. In the study by Zamora, et al.²⁸ (2012) of the nasion, point A, anterior nasal spine, pogonion, gnathion and menton points showed the greatest degrees of reliability. According to Schlicher, et al.²⁵(2012), structures on the median sagittal line are easily identified, suggesting a similarity between the sagittal slice window in the 3D module and the traditional 2D lateral view. Moreover, these structures are identified in sequence, with little change to the position of the image.

Nevertheless, in the study by Meldenik, et al.¹⁶ (2011), the pogonion and gnathion showed greater standard deviation in the identification of the x and y coordinates (axial and coronal). According to the authors, this is because they are landmarks of reference located on a prominence or curvature, and thus identification may be difficult.

The orbitale was evaluated in ten articles^{7,9-12,15,21,22,26,28} included in this study. Chien, et al.¹ (2009) found intra- and inter-observer ICC values close to 1, whereas Ludlow, et al.¹⁶ (2009) found significant inter-observer differences in the identification of this landmark. According to Oliveira, et al.²² (2009) because the orbitale is found on a curved surface, it could be better identified by the three-dimensional reconstruction image.

There was an overall low reliability in the identification of condyle landmarks. Oliveira, et al.²² (2009) evaluated landmark identification in MPR images and found poor reliability for both intra- and inter-observer values in the sagittal plane of the right condyle. In the study by Chien, et al.¹ (2009), the identification of this landmark was also imprecise in the axial plane.

These findings can be explained by the deficient definition criteria of these particular landmarks, and by their location along anatomic areas that are not of maximum curvature²⁸. According to Chien, et al.¹ (2009), identification in the transverse direction may depend on the observer’s ability to visualize the most superior and posterior points along the surface of the condyle in different slices. When these points are viewed on a sagittal slice towards the medial region, they may be identified in a more anterior position than the correct position.

Hassan, et al.⁹ (2013) evaluated the precision and time required to perform a cephalometric analysis in CBCT, using two types of images: 3D image only or 3D image associated with MPR imaging. There was an increase in the precision of identification of 15 of the 22 points when images from MPR associated with 3D models were used, however the results were statistically significant in only 6 points (the left orbitale, sella, nasion, posterior nasal spine, menton and the left mandibular molar) and in average twice the time was required. The landmark with the highest level of imprecision in that study⁹ was the right porion. This point also had a low level of reliability in other studies included in this systematic review^.

Ludlow, et al.¹⁵ (2009) found great variability in the identification of the porion point in the visualization of MPR views. The porion presented a higher level of variation (x-axis and y-axis). According to these authors, the curvature of the external acoustic meatus and the proximity to the temporal bone surface makes the porion point difficult to identify in MPR views and 3D models. Schlicher, et al.²⁵ (2012) found significant errors in identifying the right and left porion on the transversal plane.

These authors suggested that the porion point could be used for analyses in the vertical and sagittal directions, but it must be avoided in transverse analyses. However, the study by Lagravère, et al.¹¹ (2009) found a higher level of inter-evaluator error in identifying the right and left porion on the Z coordinate (sagittal). Due to errors in identifying the porion, we suggest that its use be avoided in three-dimensional cephalometry.

Overall, dental landmarks presented a high level of reliability. In the study by Hassan, et al.⁹ (2013), the maxillary central incisor presented the highest level of precision in identification, similar to the results of Zamora, et al.²⁸ (2012), who found a high level of reliability for the maxillary central incisor and the right and left maxillary molars. Fuyamada, et al.⁸ (2014) showed favorable reproducibility of the upper-1, lower-1, left lower-6 and especially left upper-6. However, in the study by Katkar, et al.¹⁰ (2013), all of the dental landmarks showed significant differences in identification, leading the authors to suggest that skeletal points are more reproducible than dental points.

Among the articles included in this systematic review, only Ludlow, et al.¹⁵ (2009) evaluated soft tissue landmarks, and significant inter-observer variations were found in the identification of points A and B in soft tissue. Further studies are necessary to determine the reliability and reproducibility of soft-tissue cephalometric landmarks identification.

CONCLUSIONS

In spite of good reliability in the identification of 3D-CBCT cephalometric landmarks being shown in most of the studies, important variations were observed in the experimental methods regarding the parameters of image acquisition, software, types of visualization, and in the anatomic references marked.

Overall, the landmarks on the median sagittal line and dental landmarks had higher reliability, while the landmarks on the condyle, porion and the orbitale presented a low level of reliability. Point S must be marked in the multiplanar views associated with visualization in 3D reconstruction. Further studies are necessary for evaluating soft tissue landmarks.

REFERENCES

• ¹
  Chien PC, Parks ET, Eraso F, Hartsfield JK, Roberts WE, Ofner S. Comparison of reliability in anatomical landmark identification using two-dimensional digital cephalometrics and three-dimensional cone beam computed tomography in vivo Dentomaxillofac Radiol. 2009;38:262-73.
• ²
  Couceiro CP, Vasconcellos Vilella O. 2D/3D Cone-Beam CT images or conventional radiography: which is more reliable? Dental Press J Orthod. 2010;15:e.40-8.
• ³
  De Vos W, Casselman J, Swennen GR. Cone-beam computerized tomography (CBCT) imaging of the oral and maxillofacial region: a systematic review of the literature. Int J Oral Maxillofac Surg. 2009;38:609-25.
• ⁴
  Donatelli RE, Lee SJ. How to report reliability in orthodontic research: Part 1. Am J Orthod Dentofacial Orthop. 2013;144:156-61.
• ⁵
  Donatelli RE, Lee SJ. How to report reliability in orthodontic research: Part 2. Am J Orthod Dentofacial Orthop. 2013;144:315-8.
• ⁶
  Forrest JL, Miller SA. Evidence-based decision making in action: Part 1 - Finding the best clinical evidence. J Contemp Dent Pract. 2002;3:10-26.
• ⁷
  Frongia G, Piancino MG, Bracco AA, Crincoli V, Debernardi CL, Bracco P. Assessment of the reliability and repeatability of landmarks using 3-D cephalometric software. Cranio. 2012;30:255-63.
• ⁸
  Fuyamada M, Shibata M, Nawa H, Yoshida K, Kise Y, Katsumata A, et al. Reproducibility of maxillofacial landmark identification on three-dimensional cone-beam computed tomography images of patients with mandibular prognathism: comparative study of a tentative method and traditional cephalometric analysis. Angle Orthod. 2014;84:966-73.
• ⁹
  Hassan B, Nijkamp P, Verheij H, Tairie J, Vink C, van der Stelt P, et al. Precision of identifying cephalometric landmarks with cone beam computed tomography in vivo Eur J Orthod. 2013;35:38-44.
• ¹⁰
  Katkar RA, Kummet C, Dawson D, Moreno Uribe L, Allareddy V, Finkelstein M, et al. Comparison of observer reliability of three-dimensional cephalometric landmark identification on subject images from Galileos and i-CAT cone beam CT. Dentomaxillofac Radiol. 2013;42:20130059.
• ¹¹
  Lagravère MO, Gordon JM, Guedes IH, Flores-Mir C, Carey JP, Heo G, et al. Reliability of traditional cephalometric landmarks as seen in three-dimensional analysis in maxillary expansion treatments. Angle Orthod. 2009;79:1047-56.
• ¹²
  Lagravère MO, Low C, Flores-Mir C, Chung R, Carey JP, Heo G, et al. Intraexaminer and interexaminer reliabilities of landmark identification on digitized lateral cephalograms and formatted 3-dimensional cone-beam computerized tomography images. Am J Orthod Dentofacial Orthop. 2010;137:598-604.
• ¹³
  Lou L, Lagravère MO, Compton S, Major PW, Flores-Mir C. Accuracy of measurements and reliability of landmark identification with computed tomography (CT) techniques in the maxillofacial area: a systematic review. Oral Surg Oral Med Oral Pathol Oral Radiol Endod. 2007;104:402-11.
• ¹⁴
  Ludlow JB, Davies-Ludlow LE, Brooks SL. Dosimetry of two extraoral direct digital imaging devices: NewTom cone beam CT and Orthophos Plus DS panoramic unit. Dentomaxillofac Radiol. 2003;32:229-34.
• ¹⁵
  Ludlow JB, Gubler M, Cevidanes L, Mol A. Precision of cephalometric landmark identification: cone-beam computed tomography vs conventional cephalometric views. Am J Orthod Dentofacial Orthop. 2009;136:312.e31-10.
• ¹⁶
  Medelnik J, Hertrich K, Steinhäuser-Andresen S, Hirschfelder U, Hofmann E. Accuracy of anatomical landmark identification using different CBCT- and MSCT-based 3D images. An in vitro study. J Orofac Orthop. 2011;72:261-78.
• ¹⁷
  Moher D, Liberati A, Tetzlaff J, Altman DG; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Int J Surg. 2010;8:336-41.
• ¹⁸
  Moshiri M, Scarfe WC, Hilgers ML, Scheetz JP, Silveira AM, Farman AG. Accuracy of linear measurements from imaging plate and lateral cephalometric images derived from cone-beam computed tomography. Am J Orthod Dentofacial Orthop. 2007;132:550-60.
• ¹⁹
  Mozzo P, Procacci C, Tacconi A, Martini PT, Andreis IA. A new volumetric CT machine for dental imaging based on the cone-beam technique: preliminary results. Eur Radiol. 1998;8:1558-64.
• ²⁰
  Naji P, Alsufyani NA, Lagravère MO. Reliability of anatomic structures as landmarks in three-dimensional cephalometric analysis using CBC. Angle Orthod. 2014;84:762-72.
• ²¹
  Neiva MB, Soares AC, Lisboa CO, Vilella OV, Motta AT. Evaluation of cephalometric landmark identification on CBCT multiplanar and 3D reconstructions. Angle Orthod. 2015;85(1):11-7.
• ²²
  Oliveira AE, Cevidanes LH, Phillips C, Motta A, Burke B, Tyndall D. Observer reliability of three-dimensional cephalometric landmark identification on cone-beam computerized tomography. Oral Surg Oral Med Oral Pathol Oral Radiol Endod. 2009;107:256-65.
• ²³
  Quintero JC, Trosien A, Hatcher D, Kapila S. Craniofacial imaging in orthodontics: historical perspective, current status, and future developments. Angle Orthod. 1999;69:491-506.
• ²⁴
  Schardt C, Adams MB, Owens T, Keitz S, Fontelo P. Utilization of the PICO framework to improve searching PubMed for clinical questions. BMC Med Inform Decis Mak. 2007;7:16.
• ²⁵
  Schlicher W, Nielsen I, Huang JC, Maki K, Hatcher DC, Miller AJ. Consistency and precision of landmark identification in three-dimensional cone beam computed tomography scans. Eur J Orthod. 2012;34:263-75.
• ²⁶
  Van Vlijmen OJ, Kuijpers MA, Bergé SJ, Schols JG, Maal TJ, Breuning H, et al. Evidence supporting the use of cone-beam computed tomography in orthodontics. J Am D Assoc. 2012;143:241-52.
• ²⁷
  Weissheimer A, Menezes LM, Sameshima GT, Enciso R, Pham J, Grauer D. Imaging software accuracy for 3-dimensional analysis of the upper airway. Am J Orthod Dentofacial Orthop. 2012;142:801-13.
• ²⁸
  Zamora N, Llamas JM, Cibrián R, Gandia JL, Paredes V. A study on the reproducibility of cephalometric landmarks when undertaking a three-dimensional (3D) cephalometric analysis. Med Oral Patol Oral Cir Bucal. 2012;17:678-88.

Publication Dates

• Publication in this collection
  Mar-Apr 2015

History

• Received
  10 Sept 2014
• Reviewed
  1 Dec 2014
• Accepted
  23 Dec 2014