Longitudinal study for dental caries calibration of dentists unexperienced in epidemiological surveys

Abstract

This study aimed to make a longitudinal analysis of interexaminer calibration reproducibility in diagnosing dental caries in posterior teeth, by examiners without previous experience in epidemiological studies. A group of 11 inexperienced examiners underwent theoretical-practical training and calibration assessments, assisted by a standard examiner. An examiner who did not participate directly in the research selected 5-year-old children with and without caries. The D3 diagnostic threshold was used to evaluate dental caries, based on the World Health Organization (WHO) criteria. The initial calibration (baseline) was performed after the theoretical-practical training session, and consisted of examining 20 children; the second calibration occurred three months later, and involved evaluating another 18 children. The interexaminer agreement was obtained by kappa statistics, and by overall percentage agreement. The paired t-test was applied to compare the values for kappa means and overall percentage agreement between the time points studied. At baseline, the values for kappa (> 0.81) and overall percentage agreement (> 95.63%) were considered high. At the 3-month calibration assessment, all the examiners showed some decrease in both kappa (p < 0.0001) and overall percentage agreement (p = 0.0102). The calibration process currently proposed by the WHO is effective. However, reproducibility was not maintained over time for inexperienced examiners evaluating the posterior teeth of 5-year-old children, under epidemiological conditions.

Dental Caries; Epidemiology

Introduction

Despite the decline observed in recent years, dental caries is still considered the main public health problem worldwide in the context of oral health.¹-⁶ Accordingly, large-scale epidemiological surveys involving several examiners are performed with the purpose of determining the epidemiological profile of this disease in different regions.⁷-⁹

The training and calibration of these examiners is an essential step needed to achieve the strict quality control of dental research, and to ensure the reliability of the results collected.¹⁰-¹² Reproducibility is considered the ability to achieve agreement among the results of a given instrument when it is applied several times, and can be evaluated using different methods, such as the overall percentage agreement, the Dice index and the kappa statistic.¹¹,¹³-¹⁵When the prevalence of disease in a given sample is low, kappa statistics are recommended, because they ensure greater precision, and take into account chance agreement.¹¹,¹⁶-¹⁸ Accordingly, the decline in the prevalence of caries disease in some regions of the world in recent years has made kappa statistics the most appropriate choice for assessing reproducibility.⁴

Five-year-old preschool children represent a significant challenge in epidemiological surveys, mainly because they may display more unpredictable behavior, be reactive to the environment, and move around frequently during the exam, making it difficult for the examiner to assess their oral health.¹⁵¹⁹-²⁰ In order to cope with these difficulties, examiners must be calibrated to evaluate this age group, since the most significant number of caries lesions in dentin are found in children under six years of age.²¹,²²

In addition, it is difficult to evaluate posterior teeth reliably in the context of an epidemiological investigation. The prevalence of cavitated caries lesions is higher on the occlusal surfaces of posterior teeth than on the smooth surfaces of anterior teeth, and performing a diagnosis can be a challenge on occlusal surfaces, considering their greater anatomical complexity.¹³,²³ Therefore, studies that report high values of agreement among examiners may not be portraying the actual difficulties involved in examining all the anterior and posterior tooth surfaces, and in diagnosing all the carious lesions that might be found. This difficulty may lead to an overestimation of the reported reproducibility values, hence to an erroneously reported low prevalence of the disease.

The literature is scare regarding the calibration process, when assessing how well examiners without previous experience in epidemiological surveys can diagnose caries, since most studies evaluate experienced examiners.¹³,²²-²⁵ This imbalance restricts how effectively the calibration process can be determined, because the same yardstick being used to measure the agreement level of experienced examiners should be able to determine how well the calibration methodology can measure high levels of agreement for new examiners.

In this context, the aim of this study was to conduct a longitudinal analysis of the interexaminer reproducibility of examiners without previous experience in epidemiological surveys, taking into account the difficulties in detecting lesions in the posterior teeth of children with primary dentition, under epidemiological conditions.

Methodology

Ethical aspects

This study was conducted in accordance with the Declaration of Helsinki, and was approved by the Research Ethics Committee of the School of Dentistry of Piracicaba (CAAE 06263219.4.3001.5418).

Study design

Characterization of the examiners

A group of 11 dental students of a postgraduate course were randomly selected. They had previous experience in performing the dental examinations of children under 6 years old, in an indoor setting, but not under epidemiological conditions. The group was assisted by a standard examiner, with extensive experience as an examiner, and coordinator of epidemiological surveys.²⁶

Characterization of the children examined

Five-year-old children from a public school, previously authorized by their parents or guardians were included in the study. They did not use any orthodontic appliance, or have any physical or intellectual disability that could preclude their participation. Children both with and without dental caries were selected, although the caries were at different stages of progression. The two groups exhibited low caries prevalence (both at baseline and after 3 months) (Table 1).

Table 1
Sample characterization.

Variables	Baseline		Three months
	All teeth	Molars	All teeth	Molars
Mean	1.60	1.60	0.67	0.56
Caries prevalence (%)	8.00	16.25	0.92	6.94

Characterization of epidemiological examinations

Examinations were carried out at school, in an outdoor setting, with the examiner using the tactile visual method, with a flat # 5 mouth mirror, WHO 621 / ball point probes, and with prior tooth brushing. During the assessment, the children remained seated in front of the examiner. Previously trained note takers recorded dental caries on WHO forms, using the D3 diagnostic threshold²⁷, frequently used in epidemiological surveys.⁷,⁹,²⁷,²⁸ It should be pointed out that only deciduous molars were evaluated.

Calibration process

The calibration process of the examiners was conducted in accordance with the WHO criteria.²⁶ Initially, a theoretical discussion lasting 4 hours was held to establish the standardization of the 11 examiners in regard to the index used for caries evaluation. The standard examiner gave a precise theoretical explanation of the index codes and criteria, after which a theoretical expository exercise was conducted, by exhibiting photos of each clinical situation that could be found in the exams.

In the practical training stage, the standard examiner performed a clinical demonstration of how the exams should be carried out, using 2 children, illustrating how the materials should be positioned, and showing how the forms should be organized. The examiner explained the ergonomic aspects of a dentist’s working postures, together with an oral health assistant. Three four-hour training sessions were carried out under the same circumstances and conditions as would be done in an outdoor setting, as described above, to resemble the reality of epidemiological surveys as closely as possible. At this stage, discussions were allowed between the dentists and the standard examiner regarding the clinical findings, diagnostic criteria, coding and registration errors, with the purpose of reaching an acceptable level of agreement.

Calibration evaluation

Calibration (baseline) was performed when the standard examiner judged that the examiners were qualified, and when an overall percentage agreement level of over 85%, and the kappa statistics (>0.80) had been obtained. At this stage, 20 children were evaluated in a 4-hour period under the same exam conditions, and in the same environment as the practical training session.

Three months after the baseline calibration, a new calibration assessment was made. At this time, the D3 codes and criteria were adopted, but the conditions for holding the dental examinations were not established, since the aim of this stage was to assess whether the baseline calibration levels had been maintained after three months. The examiners evaluated another group of 18 children, following the same criteria as those proposed in the initial stage, during one 4-hour period. It should be pointed out that no discussions were allowed between the examiners and the standard examiner at this stage.

Statistical analysis

The data were dichotomized for analysis purposes into with and without the disease in the posterior teeth; that is, the codes related to the present and past history of the caries experience, and those related to the absence and presence of the disease were grouped into two distinct blocks. The interexaminer agreement values at baseline and after three months were obtained using the kappa statistic, and overall percentage agreement. The paired t-test was applied to compare the values for kappa means and overall percentage agreement between the time points studied. The analyses were performed using the SAS statistical program (SAS Institute 2011 version 9.4, Cary, USA).

Results

Table 1 describes the sample characterization in relation to prevalence of caries at baseline and after three months. The prevalence of caries in the two groups differed, with a higher prevalence at baseline than after three months. The prevalence of caries was also higher when only the group of posterior teeth was considered.

Table 2 shows the values for kappa and overall percentage agreement at baseline and after three months. At baseline, the interexaminer agreement for both kappa (> 0.81) and overall percentage agreement (> 95.63%) was high. After three months, a decrease was observed in the values for kappa and overall percentage agreement. Five examiners had kappa values below 0.61 (moderate agreement), and the other six had values ranging between 0.61 and 0.80 (substantial agreement), respectively.¹⁶ The values for overall percentage agreement ranged from 89.58 to 97.22.

Table 2
Kappa values and overall percentage agreement for interexaminer reproducibility at baseline and after three months.

Variables	Baseline		Three months
	Kappa (95%CI)	Overall percentage agreement	Kappa (95%CI)	Overall percentage agreement
Examiner 1	0.91 (0.81–0.99)	97.51	0.69 (0.43–0.94)	96.53
Examiner 2	0.98 (0.93–1.00)	99.38	0.50 (0.29–0.71)	89.58
Examiner 3	0.95 (0.89–1.00)	98.76	0.43 (0.20–0.66)	89.58
Examiner 4	0.89 (0.79–0.98)	96.88	0.53 (0.24–0.81)	94.44
Examiner 5	0.91 (0.82–1.00)	97.51	0.74 (0.53–0.96)	96.54
Examiner 6	0.92 (0.84–1.00)	98.13	0.53 (0.24–0.81)	94.44
Examiner 7	0.91 (0.82–1.00)	97.51	0.76 (0.54–0.99)	97.22
Examiner 8	0.88 (0.78–0.98)	96.88	0.46 (0.18–0.74)	93.05
Examiner 9	0.84 (0.72–0.95)	95.63	0.61 (0.36–0.85)	94.44
Examiner 10	0.95 (0.89–1.00)	98.76	0.76 (0.54–0.99)	97.22
Examiner 11	0.90 (0.81–1.00)	97.51	0.64 (0.41–0.87)	94.45

• CI: Confidence interval.

Table 3 compares the values for mean kappa and overall percentage agreement for interexaminer reproducibility according to the time periods studied. There was a significant decrease in both the mean kappa (p < 0.0001) and the overall percentage agreement (p = 0.0102) values, three months after the initial assessment (baseline).

Table 3
Mean and standard deviation of kappa values and overall percentage agreement for interexaminer reproducibility in evaluating examiner calibration at baseline and after three months.

Variations	Mean (standard deviation)	p-value
Kappa values
Baseline	0.91 (0.04)	< 0.0001
Three months	0.60 (0.12)
Overall percentage agreement
Baseline	97.09 (1.13)	0.0102
Three months	93.91 (2.77)

Discussion

The representative samples of the studied population with the highest prevalence of the disease were selected based on the calibration exercises of the examiners determining dental caries, so that reliable values of reproducibility could be achieved.¹¹,²⁶ However, the current global context of lower and polarized caries prevalence should be taken into account,¹,²,²¹,²⁹ since this selection cannot be made in practice, when an epidemiological survey is performed.

Diagnosis using the WHO criteria for healthy anterior teeth is relatively easy,²⁷ since visualization is facilitated even under epidemiological conditions. However, the low prevalence of the disease in the posterior teeth can lead to high reproducibility values, and consequent imbalances or even an overestimation of these values.¹³,³⁰-³² Studies have shown that the kappa values for posterior teeth are considerably lower than those of anterior teeth.¹³,³⁰ Under these circumstances, therefore, it is essential to focus the calibration exercise on the group of teeth with the highest prevalence of the disease.

Although there are scientific studies that evaluate the calibration process of examiners in epidemiological caries surveys, most have evaluated examiners with previous experience.¹³,²²-²⁵ Thus, there is a gap in the literature regarding the calibration of caries. When examiners with no experience in the practice of oral health surveys are evaluated, it may not be possible to assess how effective the proposed methodology is for training new examiners.

In this study, the interexaminer kappa values were shown to be high at baseline. This outcome corroborated the results of other studies,¹³,²⁴,²⁵and reinforced the finding that the stages of theoretical-practical training are efficient in assessments conducted in the short term. At this time point, even inexperienced examiners had no difficulty in assessing the presence/absence of caries. This could be attributed to the more recent performance of the theoretical and practical calibration stages; therefore, the codes and classification criteria were also more recent in the examiners’ memory.

In contrast, there was a significant decrease in the mean kappa values for the calibration assessment after three months. Other longitudinal studies have shown that the kappa statistic remained high and stable after 12 months post theoretical-practical training stage.¹³,²⁴ This could be attributed to the studies’ having been conducted with examiners who were experienced in caries surveys, and who were used to the environment, the data collection, the calibration scheme, and the WHO criteria. In addition, the children evaluated were in the seven-year-old age group, and were selected with different levels of disease prevalence.²⁴

Surveys for collecting epidemiological oral health information on the population is an essential step in the organization, management and planning of public dental services.⁶,²¹,³³ Failure to diagnose cavitated lesions has costly economic implications, since the cost of dental treatment is much higher in these cases.⁶,³⁴

In the context of public oral health services, the calibration of new examiners is a practice required to train dentists who are not familiar with the indexes used, and the conditions of the outdoor settings where oral health surveys are often conducted. In these circumstances, the longitudinal design of this study was essential, because it demonstrated the failure to maintain the data reproducibility results over time. This failure may make the strategy used for calibrating the examiners in the present study inadequate for meeting future treatment demands in this age group. Moreover, it may lead to training unprepared examiners.

In our study, even within the context of the low prevalence of the disease, and the decrease in the number of children with caries lesions assessed after three months, the kappa value and the overall percentage of agreement dropped dramatically. In principle, this point could constitute a limiting factor of this study. However, it should be pointed out that although dentin caries prevalence has decreased worldwide, the number of initial enamel lesions may be increasing. This epidemiological change may have led to an increase in the disagreements in diagnosing conditions related to healthy and decayed teeth (enamel/dentin lesions). Furthermore, with time, examiners may not remember all the nuances regarding the WHO coding, and this may also give rise to disagreements among them.

In this respect, the methodology currently proposed by the WHO,²⁶ and followed in this study to evaluate caries in posterior teeth in the 5-year age group, proved not effective over time in maintaining the calibration status of examiners inexperienced in calibrating epidemiological surveys. This ineffectiveness reinforces the clinical and social relevance of the present study. The only way inexperienced examiners can become better prepared is if more time is dedicated to theoretical-practical training than the time proposed at present. Moreover, additional periods of theoretical and practical classes for examining survey participants must be provided under epidemiological conditions. In view of the foregoing, a possible solution would be to recalibrate inexperienced examiners periodically, until they reach a minimum level of experience, so that reliable reproducibility values can be maintained over time.

Conclusion

The values for mean interexaminer kappa and overall percentage agreement of examiners with no previous experience in epidemiological caries surveys decreased when they were assessed three months after baseline. This outcome demonstrates that the calibration process currently proposed by the WHO does not maintain the initial calibration level of inexperienced examiners over time. The calibration process was based on the assessment by inexperienced examiners of the posterior teeth of groups of 5-year-old children, under epidemiological conditions. Therefore, a longer time of theoretical-practical training and periodic recalibration assessments is required until reliable levels of reproducibility can be maintained.

References

• ¹
  Kraljevic I, Filippi C, Filippi A. Risk indicators of early childhood caries (ECC) in children with high treatment needs. Swiss Dent J. 2017 May;127(5):398-410.
• ²
  Henry JA, Muthu MS, Swaminathan K, Kirubakaran R. Do oral health educational programmes for expectant mothers prevent early childhood caries? Systematic review. Oral Health Prev Dent. 2017;15(3):215-21. https://doi.org/10.3290/j.ohpd.a38522
  » https://doi.org/10.3290/j.ohpd.a38522
• ³
  Mathur VP, Dhillon JK. Dental caries: a disease which needs Attention. Indian J Pediatr. 2018 Mar;85(3):202-6. https://doi.org/10.1007/s12098-017-2381-6
  » https://doi.org/10.1007/s12098-017-2381-6
• ⁴
  Taqi M, Razak IA, Ab-Murat N. Comparing dental caries status using Modified International Caries Detection and Assessment System (ICDAS) and World Health Organization (WHO) indices among school children of Bhakkar, Pakistan. J Pak Med Assoc. 2019 Jul;69(7):950-4.
• ⁵
  Neves ÉT, Dutra LD, Gomes MC, Paiva SM, Abreu MH, Ferreira FM, et al. The impact of oral health literacy and family cohesion on dental caries in early adolescence. Community Dent Oral Epidemiol. 2020 Jun;48(3):232-9. https://doi.org/10.1111/cdoe.12520
  » https://doi.org/10.1111/cdoe.12520
• ⁶
  Medina-Solís CE, Ávila-Burgos L, Borges-Yañez SA, Irigoyen-Camacho ME, Sánchez-Pérez L, Zepeda-Zepeda MA, et al. Ecological study on needs and cost of treatment for dental caries in schoolchildren aged 6, 12, and 15 years: data from a national survey in Mexico. Medicine (Baltimore). 2020 Feb;99(7):e19092. https://doi.org/10.1097/MD.0000000000019092
  » https://doi.org/10.1097/MD.0000000000019092
• ⁷
  Corrêa-Faria P, Paixão-Gonçalves S, Paiva SM, Pordeus IA. Incidence of dental caries in primary dentition and risk factors: a longitudinal study. Braz Oral Res. 2016 May;30(1):S1806-83242016000100254. https://doi.org/10.1590/1807-3107BOR-2016.vol30.0059
  » https://doi.org/10.1590/1807-3107BOR-2016.vol30.0059
• ⁸
  Lawal F, Alade O. Dental caries experience and treatment needs of an adult female population in Nigeria. Afr Health Sci. 2017 Sep;17(3):905-11. https://doi.org/10.4314/ahs.v17i3.34
  » https://doi.org/10.4314/ahs.v17i3.34
• ⁹
  Musinguzi N, Kemoli A, Okullo I. Prevalence and treatment needs for early childhood caries among 3-5-year-old children from a rural community in Uganda. Front Public Health. 2019 Sep;7:259. https://doi.org/10.3389/fpubh.2019.00259
  » https://doi.org/10.3389/fpubh.2019.00259
• ¹⁰
  Banting DW, Amaechi BT, Bader JD, Blanchard P, Gilbert GH, Gullion CM, et al. Examiner training and reliability in two randomized clinical trials of adult dental caries. J Public Health Dent. 2011;71(4):335-44. https://doi.org/10.1111/j.1752-7325.2011.00278.x
  » https://doi.org/10.1111/j.1752-7325.2011.00278.x
• ¹¹
  Tonello AS, Silva RP, Assaf AV, Ambrosano GM, Peres SH, Pereira AC, et al. Interexaminer agreement dental caries epidemiological surveys: the importance of disease prevalence in the sample. Rev Bras Epidemiol. 2016 Apr-Jun;19(2):272-9. https://doi.org/10.1590/1980-5497201600020005
  » https://doi.org/10.1590/1980-5497201600020005
• ¹²
  Rechmann P, Jue B, Santo W, Rechmann BM, Featherstone JD. Calibration of dentists for Caries Management by Risk Assessment Research in a Practice Based Research Network - CAMBRA PBRN. BMC Oral Health. 2018 Jan;18(1):2. https://doi.org/10.1186/s12903-017-0457-3
  » https://doi.org/10.1186/s12903-017-0457-3
• ¹³
  Assaf AV, Tagliaferro EP, Meneghim MC, Tengan C, Pereira AC, Ambrosano GM, et al. A new approach for interexaminer reliability data analysis on dental caries calibration. J Appl Oral Sci. 2007 Dec;15(6):480-5. https://doi.org/10.1590/S1678-77572007000600005
  » https://doi.org/10.1590/S1678-77572007000600005
• ¹⁴
  Koç Vural U, Kütük ZB, Ergin E, Yalçın Çakır F, Gürgan S. Comparison of two different methods of detecting residual caries. Restor Dent Endod. 2017 Feb;42(1):48-53. https://doi.org/10.5395/rde.2017.42.1.48
  » https://doi.org/10.5395/rde.2017.42.1.48
• ¹⁵
  Christian B, Amezdroz E, Calache H, Gussy M, Sore R, Waters E. Examiner calibration in caries detection for populations and settings where in vivo calibration is not practical. Community Dent Health. 2017 Dec;34(4):248-53. https://doi.org/10.1922/CDH_4102Christian06
  » https://doi.org/10.1922/CDH_4102Christian06
• ¹⁶
  Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977 Mar;33(1):159-74. https://doi.org/10.2307/2529310
  » https://doi.org/10.2307/2529310
• ¹⁷
  Assaf AV, Zanin L, Meneghim Mde C, Pereira AC, Ambrosano GM. [Comparison of reproducibility measurements for calibration of dental caries epidemiological surveys]. Cad Saude Publica. 2006(a) Sep;22(9):1901-7. Portuguese. https://doi.org/10.1590/S0102-311X2006000900021
  » https://doi.org/10.1590/S0102-311X2006000900021
• ¹⁸
  Agbaje JO, Mutsvari T, Lesaffre E, Declerck D. Measurement, analysis and interpretation of examiner reliability in caries experience surveys: some methodological thoughts. Clin Oral Investig. 2012(a) Feb;16(1):117-27. https://doi.org/10.1007/s00784-010-0475-x
  » https://doi.org/10.1007/s00784-010-0475-x
• ¹⁹
  Aminabadi NA, Puralibaba F, Erfanparast L, Najafpour E, Jamali Z, Adhami SE. Impact of temperament on child behavior in the dental setting. J Dent Res Dent Clin Dent Prospect. 2011;5(4):119-22. https://doi.org/10.5681/joddd.2011.027
  » https://doi.org/10.5681/joddd.2011.027
• ²⁰
  Warren JJ, Weber-Gasparoni K, Tinanoff N, Batliner TS, Jue B, Santo W, et al. Examination criteria and calibration procedures for prevention trials of the Early Childhood Caries Collaborating Centers. J Public Health Dent. 2015;75(4):317-26. https://doi.org/10.1111/jphd.12102
  » https://doi.org/10.1111/jphd.12102
• ²¹
  Silva RP, Assaf AV, Mialhe FL, Mendes KL, Meneghim MC, Pereira AC. Dental caries diagnostic thresholds: which one? Why? When? Int J Public Health. 2020 Apr;65(3):371-4. https://doi.org/10.1007/s00038-020-01332-3
  » https://doi.org/10.1007/s00038-020-01332-3
• ²²
  Agbaje JO, Mutsvari T, Lesaffre E, Declerck D. Examiner performance in calibration exercises compared with field conditions when scoring caries experience. Clin Oral Investig. 2012(b) Apr;16(2):481-8. https://doi.org/10.1007/s00784-011-0523-1
  » https://doi.org/10.1007/s00784-011-0523-1
• ²³
  Silva RP, Assaf AV, Pereira SM, Ambrosano GM, Mialhe FL, Pereira AC. Reproducibility of adjunct techniques for diagnosis of dental caries in an epidemiological situation. Oral Health Prev Dent. 2011;9(3):251-9.
• ²⁴
  Assaf AV, Meneghim MC, Zanin L, Tengan C, Pereira AC. Effect of different diagnostic thresholds on dental caries calibration: a 12 month evaluation. Community Dent Oral Epidemiol. 2006(b) Jun;34(3):213-9. https://doi.org/10.1111/j.1600-0528.2006.00278.x
  » https://doi.org/10.1111/j.1600-0528.2006.00278.x
• ²⁵
  Boye U, Pretty IA, Tickle M, Walsh T. Comparison of caries detection methods using varying numbers of intra-oral digital photographs with visual examination for epidemiology in children. BMC Oral Health. 2013 Jan;13(1):6. https://doi.org/10.1186/1472-6831-13-6
  » https://doi.org/10.1186/1472-6831-13-6
• ²⁶
  World Health Organization (WHO). Oral health surveys: basic methods. 5th ed. Geneva: WHO; 2013.
• ²⁷
  Alsadat FA, El-Housseiny AA, Alamoudi NM, Elderwi DA, Ainosa AM, Dardeer FM. Dental fear in primary school children and its relation to dental caries. Niger J Clin Pract. 2018 Nov;21(11):1454-60. https://doi.org/10.4103/njcp.njcp_160_18
  » https://doi.org/10.4103/njcp.njcp_160_18
• ²⁸
  Calache H, Christian B, Mamerto M, Kangutkar T, Hall M. An epidemiological study of dental caries and associated risk factors among primary school children in the Aileu Municipality, Timor-Leste. Rural Remote Health. 2019 Oct;19(4):5322. https://doi.org/10.22605/RRH5322
  » https://doi.org/10.22605/RRH5322
• ²⁹
  Cunha IP, Mialhe FL, Pereira AC, Vedovello SA, Bulgareli JV, Frias AC, et al. Self-perceived dental treatment need among adolescents: A hierarchical analysis. Community Dent Oral Epidemiol. 2020 Apr;48(2):130-6. https://doi.org/10.1111/cdoe.12510
  » https://doi.org/10.1111/cdoe.12510
• ³⁰
  Peres MA, Traebert J, Marcenes W. [Calibration of examiners for dental caries epidemiologic studies]. Cad Saude Publica. 2001 Jan-Feb;17(1):153-9. Portuguese. https://doi.org/10.1590/S0102-311X2001000100016
  » https://doi.org/10.1590/S0102-311X2001000100016
• ³¹
  Sim J, Wright CC. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther. 2005 Mar;85(3):257-68. https://doi.org/10.1093/ptj/85.3.257
  » https://doi.org/10.1093/ptj/85.3.257
• ³²
  Viera AJ, Garrett JM. Understanding interobserver agreement: the kappa statistic. Fam Med. 2005 May;37(5):360-3.
• ³³
  Al-Thani MH, Al-Thani AA, Al-Emadi AA, Al-Chetachi WF, Akram H, Poovelil BV. Oral health status of six-year-old children in Qatar: findings from the national oral health survey. Int J Dent Hyg. 2018 May;16(2):225-32. https://doi.org/10.1111/idh.12258
  » https://doi.org/10.1111/idh.12258
• ³⁴
  Ladewig NM, Camargo LB, Tedesco TK, Floriano I, Gimenez T, Imparato JC, et al. Management of dental caries among children: a look at the cost-effectiveness. Expert Rev Pharmacoecon Outcomes Res. 2018 Apr;18(2):127-34. https://doi.org/10.1080/14737167.2018.1414602
  » https://doi.org/10.1080/14737167.2018.1414602

Publication Dates

• Publication in this collection
  31 Mar 2023
• Date of issue
  2023

History

• Received
  17 Aug 2020
• Accepted
  24 Feb 2022
• Reviewed
  28 Sept 2022