Abstract
Objective:
Threedimensional (3D) angular measurements between craniofacial planes pose challenges to quantify maxillary and mandibular skeletal discrepancies in surgical treatment planning. This study aims to compare the reproducibility and reliability of two modules to measure angles between planes or lines in 3D virtual surface models.
Methodology:
Twenty oriented 3D virtual surface models deidentified and constructed from CBCT scans were randomly selected. Three observers placed landmarks and oriented planes to determine angular measurements of pitch, roll and yaw using (1) 3D preexisting planes, (2) 3D planes created from landmarks and (3) lines created from landmarks. Inter and intraobserver reproducibility and repeatability were examined using the IntraClass Correlation (ICC) test. One observer repeated the measurements with an interval of 15 days. ANOVA was applied to compare the 3 methods.
Results:
The three methods tested provided statistically similar, reproducible and reliable angular measurements of the facial structures. A strong ICC varying from 0.92 to 1.00 was found for the intraobserver agreement. The interobserver ICC varied from 0.84 to 1.00.
Conclusion:
Measurements of 3D angles between facial planes in a common coordinate system are reproducible and repeatable either using 3D preexisting planes, created based on landmarks or angles between lines created from landmarks.
Keywords:
Reproducibility of results; Threedimensional imaging; Computerassisted image processing; Conebeam computed tomography
Introduction
Quantification of facial characteristics is of extreme importance in diagnosis and different measurement techniques have evolved from direct measurements of skulls to indirect measurements based on imaging exams. The advent of radiographs markedly increased the number of studies that attempted to understand the development of growth and treatment results.^{1}1 Björk A. Cranial base development: a followup xray study of the individual variation in growth occurring between the ages of 12 and 20 years and its relation to brain case and face development. Am J Orthod. 1955;41:198225^{–}^{3}3 Varjanne I, Koski K. Cranial base, sagittal jaw relationship and occlusion. A radiologicalcraniometric appraisal. Proc Finn Dent Soc 1982;78(4):17983. Even though 2D images had provided important information for decades, nowadays 3D images are able to provide more accurate information, therefore improving the assessment of craniofacial anatomy and changes after treatment or surgery. It overcomes some inherent flaws of 2D images, such as patient's head position, superimposition of different anatomical structures, image magnification and distortions. It also enables volumetric measurements, allowing for a detailed assessment of maxillofacial structures in variable thickness of axial, coronal and sagittal slices, providing real measurements with no magnification.^{4}4 Gossett CB, Preston CB, Dunford R, Lampasso J. Prediction accuracy of computerassisted surgical visual treatment objectives as compared with conventional visual treatment objectives. J Oral Maxillofac Surg. 2005;63(5):60917.^{–}^{13}13 Swennen GR, Eulzer C, Schutyser F, Hüttmann C, Schliephake H. Assessment of the distraction regenerate using threedimensional quantitative computer tomography. Int J Oral Maxillofac Surg. 2005;34(1):6473.
Conebeam computed tomography (CBCT) has been the image of choice for diagnosis and treatment planning of reconstructive surgeries, dental implants, patients with asymmetry and/or craniofacial anomalies because it improves the visualization and understanding of the anatomy. However, caution should be taken when requesting these images due to its radiation dose. 3D cephalometric tools for clinical diagnosis as well as visualization of the 3D images have been available in numerous commercial software such as Mimics (Materialise, Leuven, Belgium), OsiriX (Pixmeo, Geneva, Switzerland), Dolphin3D (Dolphin Imaging & Management Solutions, Chatsworth, California, USA), InVivo Dental (Anatomage, San Jose, California, USA), and Ondemand3D (CyberMed, Seoul, Korea). The visualization tools allow for the assessment of CBCT images by not only showing axial, coronal and sagittal images, but also creating a 3D reformatted image. However, research purposes and surgery planning go beyond simple visualization, and therefore several 3D cephalometric tools have also been proposed to quantify linear and angular craniofacial measurements, transitioning from 2D to 3D analyses.^{14}14 Nanda RS, Merrill RM. Cephalometric assessment of sagittal relationship between maxilla and mandible. Am J Orthod Dentofacial Orthop. 1994;105(4):32844.^{–}^{20}20 Gupta A, Kharbanda OP, Sardana V, Balachandran R, Sardana HK. Accuracy of 3D cephalometric measurements based on an automatic knowledgebased landmark detection algorithm. Int J Comput Assist Radiol Surg. 2016;11(7):1297309. Most of these studies use CBCT to visualize a specific region, but still perform an overall overview of the patient using reformatted 2D images.
The use of 3D planes to quantify the craniofacial morphology proportions or measure angles between planes of anatomical structures pose mathematical challenges. It is important to understand that in 3D analysis a plane is defined by three points that may not lie at the same level; that the angle between two planes are determined by the normal vector of the planes; and clinicians need to become familiar with which angle to measure in the 3D space, since complementary angles are calculated. The main purpose is to present three new methods from an opensource software (SlicerCMF) to calculate anatomical angles in virtual surface models (constructed from CBCT images) in the three planes of space and assess the reproducibility of measuring the angles: (1) using 3D preexisting planes, (2) creating a 3D plane based on landmarks, and (3) using lines created from landmarks.
Methodology
This study was approved by the Institutional Review Board (HUM00066254). Sample size calculation was performed by three observers a minimum and expected ICC (IntraClass Correlation) of 0.75 and 0.90, respectively. As a result, a sample size of 19 models was needed for interobserver correlation, and 30 models for intraobserver correlation.
Threedimensional surface models of deidentified patients were randomly selected from the archives of the Orthodontic Imaging Lab from the University of Michigan. The only exclusion criterion was lack of welldefined anatomical structures in the 3D surface models. All models were preoriented using the midsagittal plane, the Frankfurt horizontal plane, and the transporionic line.^{21}21 Ruellas AC, Tonello C, Gomes LR, Yatabe MS, Macron L, Lopinto J, et al. Common 3dimensional coordinate system for assessment of directional changes. Am J Orthod Dentofacial Orthop. 2016;149(5):64556.
Utilizing three angular measurements commonly used in 2D cephalometry analysis, in either lateral, AP or submentovertex cephalograms, a list of clearly defined anatomical landmarks that were placed in the surface models can be found in Figure 1. Anatomical angles were then measured in different views of the 3D space, defined as follows:

FMA (Frankfurtmandibular plane): The Frankfurt Horizontal plane passes by the most superior portion of the “Porion” at both sides and by the most inferior portion of the “Orbitale” at both sides. In cases of asymmetry, the plane was positioned inbetween the most inferior portion of the left and right sides. When placing the landmarks, the software is able to create a midpoint between the right and leftmost inferior portion of the Orbitale. The Mandibular plane passes by the most inferior and posterior portion of the lower border of the mandibular corpus (right and left sides) and by the “Menton” point (Mandibular Plane). This angle was assessed in coronal (FMA roll) and sagittal (FMA pitch) views (Figure 2a).

Gonial Angle: The “Mandibular plane” passes by the most inferior and posterior portion of the right and left lower border of the mandibular corpus and by the “Menton” point. The “Posterior border of the ramus plane” passes by the most posterior points of right and left condyles and rami. In cases of asymmetry, the plane was positioned in between the most posterior portion of the left and right condyles. When placing the landmarks, the software is able to create a midpoint between the right and leftmost posterior portion of the Condyles. This angle was assessed in sagittal view (Figure 2b).

Condylar Angle: Each condylar long axis plane passes through the medial and lateral poles, and through the center of the superior surface of the condylar head at each side. The condylar angle was assessed in axial view (Figure 2c).
(A) Lateral view of the mandibular (yellow, inferior position) and Frankfurt (red, superior position) planes used to calculate FMA roll and pitch in the coronal and sagittal views, respectively; (B) Posterior border of the ramus plane (yellow, vertical) and Mandibular plane (red, inferiorly located) used to calculate the Gonial Angle pitch in the sagittal view. Note indication of main (α) and complementary (ρ) angles measured; (C) Left Condyle (red; left) and Right Condyle (yellow; right) planes used to calculate the Condylar Angle yaw in axial view
The intersection of lines and/or planes in 3D determines the Yaw, Pitch and Roll angles in the three spatial planes (axial, sagittal and coronal, respectively). Three different methods for calculating the anatomic angles between planes were tested:

3D preexisting planes: Using the Angle Planes module in SlicerCMF 3.1 (www.slicer.org), two preexisting planes (Axial, Coronal or Sagittal) were manually positioned tangent to each anatomic structure of interest in the 3D surface models in order to determine the angles according to different spatial views (Figure 3a).

Creating a 3D plane based on landmarks: Using the Angle Planes module in 3D SlicerCMF 3.1 (www.slicer.org), landmarks were placed at specific anatomical locations (Figure 1) in order to create each of the planes used to determine the angles of interest (Figure 3b).

Angle between lines from landmarks: Using the Q3DC module in SlicerCMF 3.1 (www.slicer.org), landmarks were placed at specific anatomical locations (Figure 1), in order to create lines for the representation of the planes. When assessing bilateral structures, a midpoint was used as representative of both sides (Figure 3c and 3d).
(A) The original coordinate planes in the SlicerCMF 3.1 (www.slicer.org) software were manually moved to intersect the anatomical structures; (B) Illustrating the blue planes created by placing landmarks in the model; (C) Illustrating anatomical landmarks placed over the 3D surface model. Lines created to represent the planes to measure pitch and (D) roll
Statistical analysis
To assess the reproducibility and reliability of the methods, three observers assessed the angles defined in the three spatial planes. To assess the repeatability of the method, one observer repeated the angular measurements with an interval of 15 days. Statistical analysis was performed using the Intraclass Correlation Coefficient (ICC) test.
To compare the three methods of measuring 3D angles while considering the normal distribution of the results, the ANOVA test was applied.
Results
Table 1 shows the intra and interrater correlations. For the measurements performed with 3D preexisting planes adjusting the tangent to surfaces by utilizing the Angle Planes module, the smallest intraobserver ICC was 0.93 and the smallest interrater ICC was 0.84, For the measurements with “angles between lines from landmarks”, utilizing the Q3DC module, the lowest intraobserver ICC was 0.92 and the lowest interrater ICC was 0.88. For the measurements performed with 3D planes created based on landmarks by utilizing the Angle Planes module landmarks option, the lowest intraobserver ICC was 0.94 and the lowest interrater ICC was 0.91.
Even though there were slight differences in the inter and intrarater correlations using the three methods for angular measurements, the ANOVA test showed no significant difference between the three methods (Table 2).
Discussion
In the transition between 2D and 3D assessments of craniofacial structures, 2D images have been rendered from CBCT scans and conventional cephalometric analysis has been applied in a number of recent studies. The advantages of cephalometric analysis in images rendered from CBCT scans is the lack of magnification of the image and, in asymmetrical cases, the possibility to measure the right and the left sides separately. Previous studies have shown that 2D digital visual treatment objectives are similar to conventional assessment for the maxilla, but less for the mandible due to surgical mandibular changes being more complex than maxillary changes.^{4}4 Gossett CB, Preston CB, Dunford R, Lampasso J. Prediction accuracy of computerassisted surgical visual treatment objectives as compared with conventional visual treatment objectives. J Oral Maxillofac Surg. 2005;63(5):60917. Orthognathic surgery planning has improved significantly with the advent of 3D image analysis. Using 3D virtual surface models instead of 2D rendered images has the advantage of planned and customized osteotomies by creating surgical resection guides,^{6}6 SawhMartinez R, Parsaei Y, Wu R, Lin A, Metzler P, DeSesa C, et al. Improved temporomandibular joint position after 3Dimensional planned mandibular reconstruction. J Oral Maxillofac Surg. 2017;75(1):197206. moving the models as needed to place the landmarks in the most accurate position possible, and to use planes to measure different angles in all three views, which helps improve diagnosis and treatment planning.
With the development of 3D cephalometric analysis, commercial software initially offered capabilities of measuring 3D angles between landmarks.^{10}10 Olszewski R, Tanesy O, Cosnard G, Zech F, Reychler H. Reproducibility of osseous landmarks used for computed tomography based threedimensional cephalometric analyses. J Craniomaxillofac Surg. 2010;38(3):21421.^{–}^{13}13 Swennen GR, Eulzer C, Schutyser F, Hüttmann C, Schliephake H. Assessment of the distraction regenerate using threedimensional quantitative computer tomography. Int J Oral Maxillofac Surg. 2005;34(1):6473.^{,}^{22}22 Oliveira AE, Cevidanes LH, Phillips C, Motta A, Burke B, Tyndall D. Observer reliability of threedimensional cephalometric landmark identification on conebeam computerized tomography. Oral Surg Oral Med Oral Pathol Oral Radiol Endod. 2009;107(2):25665. A limitation of the method that requires “placing the landmarks” is that even though these are digital landmarks, they are still placed manually, and therefore, are still open to human errors.^{5}5 Power G, Breckon J, Sherriff M, McDonald F. Dolphin Imaging Software: an analysis of the accuracy of cephalometric digitization and orthognathic prediction. Int J Oral Maxillofac Surg. 2005;34(6):61926. A difference of ±2 mm between landmarks placement is acceptable in orthognathic planning without significant impact on clinical decisionmaking.^{8}8 Miloro M, Borba AM, RibeiroJunior O, NaclerioHomem MG, Jungner M. Is there consistency in cephalometric landmark identification amongst oral and maxillofacial surgeons? Int J Oral Maxillofac Surg. 2014;43(4):44553. Our findings corroborate the literature, showing differences of approximately 2 mm between first and second measurements, as well as between observers (Table 1). However, those repeated differences occurred both when placing landmarks and when moving the planes towards that direction.
Most conventional angular measurements in 2D cephalometry analysis are performed between two lines that often represent 3D planes such as the Frankfurt Horizontal and Mandibular Plane. When measuring the angles formed by the planes or lines in any of the three methods tested in this study, two complementary angles can be formed between two planes or lines, always resulting in 180°. While current cephalometric analyses do not list a “complimentary” angle and only list the specific angles of interest for clinicians, the current version of the software described in this study delineates preset cephalometric analyses and rather gives the user the flexibility to determine whatever measurements they would like to implement. The new tools require clinicians to interpret and understand what they intend to measure in each of the three spatial views (sagittal, frontal and axial). Interestingly, the given angles between two planes may have clinical meaning in two different views of the 3D space. For example, for FMA, both frontal perspective measurements (FMA roll) and sagittal perspective (FMA pitch) evaluate different and helpful aspects of facial morphology.
This study focused on four angles previously frequently utilized in 2D cephalometry studies with lateral cephalograms (FMA pitch and Gonial angle), frontal cephalograms (FMA roll) and submentovertex xrays (condylar angle). Even though this study utilized measurements derived from known 2D cephalometry populational norms and standards, the 3D surface models constructed from CBCT images allow users to measure any other angles that may be helpful to evaluate complex skeletal discrepancies that were not previously possible to measure in 2D images.^{6}6 SawhMartinez R, Parsaei Y, Wu R, Lin A, Metzler P, DeSesa C, et al. Improved temporomandibular joint position after 3Dimensional planned mandibular reconstruction. J Oral Maxillofac Surg. 2017;75(1):197206.^{,}^{11}11 Olszewski R, Reychler H, Cosnard G, Denis JM, Vynckier S, Zech F. Accuracy of threedimensional (3D) craniofacial cephalometric landmarks on a lowdose 3D computed tomograph. Dentomaxillofac Radiol. 2008;37(5):2617.^{–}^{13}13 Swennen GR, Eulzer C, Schutyser F, Hüttmann C, Schliephake H. Assessment of the distraction regenerate using threedimensional quantitative computer tomography. Int J Oral Maxillofac Surg. 2005;34(1):6473. The two methods using landmarks were similar, and no significant challenges were noticed. The method of managing preexisting planes, however, demanded more practice adjusting the planes towards the correct position during the calibration period. ANOVA results showed statistical similarity between the methods (Table 2). The high intraobserver correlation found suggests that all methods are repeatable options for angular measurements of 3D surface models (Table 1). The high interobserver correlations suggest that all methods are also reproducible (Table 1). Therefore, users may use any tool they feel more comfortable with.
The greatest challenge in transitioning from 2D to 3D craniofacial measurements is how to interpret the data findings in a clinically meaningful way in order to provide improved diagnosis and assessment of treatment outcomes. While single angular measurements were performed and easily interpreted in a 2D projection of the skull, when 3D angles are measured, three different angles can be determined: pitch, roll and yaw (Figure 4); and it is up to the observer to identify which angles are relevant. Additionally, when comparing between different timepoints and different patients, it is important to standardize the head position to consistently assess angular relationships to craniofacial structures. Knowing that 3D measurements are reliable and reproducible, further research should compare them against conventional 2D cephalometry, currently still considered the goldstandard method in research and clinical practice.
Conclusion
Based on the results from this study, measurements of 3D angles in a common coordinate system are reproducible and repeatable either using 3D preexisting planes, or creating a 3D plane based on landmarks or angles between lines created from landmarks.
Acknowledgement
Funding: This study was funded by NIH – NIDCR and NIBIB (R01 DE024450 and R21 DE025306).
References

^{1} Björk A. Cranial base development: a followup xray study of the individual variation in growth occurring between the ages of 12 and 20 years and its relation to brain case and face development. Am J Orthod. 1955;41:198225

^{2} Riolo ML. An atlas of craniofacial growth: cephalometric standards from the University school growth study, the University of Michigan. Ann Arbor: Center for Human Growth and Development, University of Michigan; 1974. 379 p.

^{3} Varjanne I, Koski K. Cranial base, sagittal jaw relationship and occlusion. A radiologicalcraniometric appraisal. Proc Finn Dent Soc 1982;78(4):17983.

^{4} Gossett CB, Preston CB, Dunford R, Lampasso J. Prediction accuracy of computerassisted surgical visual treatment objectives as compared with conventional visual treatment objectives. J Oral Maxillofac Surg. 2005;63(5):60917.

^{5} Power G, Breckon J, Sherriff M, McDonald F. Dolphin Imaging Software: an analysis of the accuracy of cephalometric digitization and orthognathic prediction. Int J Oral Maxillofac Surg. 2005;34(6):61926.

^{6} SawhMartinez R, Parsaei Y, Wu R, Lin A, Metzler P, DeSesa C, et al. Improved temporomandibular joint position after 3Dimensional planned mandibular reconstruction. J Oral Maxillofac Surg. 2017;75(1):197206.

^{7} Santos RM, De Martino JM, Haiter F Neto, Passeri LA. Influence of different setups of the Frankfort horizontal plane on 3dimensional cephalometric measurements. Am J Orthod Dentofacial Orthop. 2017;152(2):2429.

^{8} Miloro M, Borba AM, RibeiroJunior O, NaclerioHomem MG, Jungner M. Is there consistency in cephalometric landmark identification amongst oral and maxillofacial surgeons? Int J Oral Maxillofac Surg. 2014;43(4):44553.

^{9} Ritto FG, Schmitt AR, Pimentel T, Canellas JV, Medeiros PJ. Comparison of the accuracy of maxillary position between conventional model surgery and virtual surgical planning. Int J Oral Maxillofac Surg. 2018;47(2):1606.

^{10} Olszewski R, Tanesy O, Cosnard G, Zech F, Reychler H. Reproducibility of osseous landmarks used for computed tomography based threedimensional cephalometric analyses. J Craniomaxillofac Surg. 2010;38(3):21421.

^{11} Olszewski R, Reychler H, Cosnard G, Denis JM, Vynckier S, Zech F. Accuracy of threedimensional (3D) craniofacial cephalometric landmarks on a lowdose 3D computed tomograph. Dentomaxillofac Radiol. 2008;37(5):2617.

^{12} Olszewski R, Frison L, Wisniewski M, Denis JM, Vynckier S, Cosnard G, et al. Reproducibility of threedimensional cephalometric landmarks in conebeam and lowdose computed tomography. Clin Oral Investig. 2013;17(1):28592.

^{13} Swennen GR, Eulzer C, Schutyser F, Hüttmann C, Schliephake H. Assessment of the distraction regenerate using threedimensional quantitative computer tomography. Int J Oral Maxillofac Surg. 2005;34(1):6473.

^{14} Nanda RS, Merrill RM. Cephalometric assessment of sagittal relationship between maxilla and mandible. Am J Orthod Dentofacial Orthop. 1994;105(4):32844.

^{15} Ferrario VF, Serrao G, Ciusa V, Morini M, Sforza C. Cephalometric and in vivo measurements of maxillomandibular anteroposterior discrepancies: a preliminary regression study. Angle Orthod. 2002;72(6):57984.

^{16} Gribel BF, Gribel MN, Manzi FR, Brooks SL, McNamara JA Jr. From 2D to 3D: an algorithm to derive normal values for 3dimensional computerized assessment. Angle Orthod. 2011;81(1):310.

^{17} Titiz I, Laubinger M, Keller T, Hertrich K, Hirschfelder U. Repeatability and reproducibility of landmarks  a threedimensional computed tomography study. Eur J Orthod. 2012;34(3):27686.

^{18} Pittayapat P, LimchaichanaBolstad N, Willems G, Jacobs R. Threedimensional cephalometric analysis in orthodontics: a systematic review. Orthod Craniofac Res. 2014;17(2):6991.

^{19} Lee M, Kanavakis G, Miner RM. Newly defined landmarks for a threedimensionally based cephalometric analysis: a retrospective conebeam computed tomography scan review. Angle Orthod. 2015;85(1):310.

^{20} Gupta A, Kharbanda OP, Sardana V, Balachandran R, Sardana HK. Accuracy of 3D cephalometric measurements based on an automatic knowledgebased landmark detection algorithm. Int J Comput Assist Radiol Surg. 2016;11(7):1297309.

^{21} Ruellas AC, Tonello C, Gomes LR, Yatabe MS, Macron L, Lopinto J, et al. Common 3dimensional coordinate system for assessment of directional changes. Am J Orthod Dentofacial Orthop. 2016;149(5):64556.

^{22} Oliveira AE, Cevidanes LH, Phillips C, Motta A, Burke B, Tyndall D. Observer reliability of threedimensional cephalometric landmark identification on conebeam computerized tomography. Oral Surg Oral Med Oral Pathol Oral Radiol Endod. 2009;107(2):25665.
Publication Dates

Publication in this collection
03 June 2019 
Date of issue
2019
History

Received
03 July 2018 
Reviewed
07 Nov 2018 
Accepted
18 Nov 2018