Internal consistency and reproducibility of portuguese version of research diagnostic criteria for temporomandibular disorders (RDC/TMD-Axis II)

Abstracts

OBJECTIVE: To study the reliability of the Portuguese version of a questionnaire for psychologically and psychosocially diagnosing individuals with temporomandibular disorders (RDC/TMD). METHOD: Interviews were held with 109 individuals of both sexes who required care at the physical therapy clinic of the Araraquara University Center (UNIARA) from January to July 2006. The questionnaires were applied by a single examiner. Two weeks later, the same questionnaire was reapplied to 36 individuals. To evaluate the internal consistency of the method, Cronbach's alpha coefficient was used. The intra-examiner reproducibility was analyzed using the intraclass correlation coefficient (rho) for quantitative variables and Kappa (kappa) statistics for qualitative variables. RESULTS: The internal consistency for the dimensions of chronic pain intensity, disability, limitations relating to mandibular function, nonspecific physical symptoms with pain items included, and nonspecific physical symptoms with pain items and depression excluded, were 0.8479, 0.8971, 0.8673, 0.8080 and 0.9270 respectively, thus confirming the excellent internal validity of the method. The intra-observer agreement was found to be "excellent" for questions relating to pain duration and intensity and "good" for the question relating to current pain. The lowest kappa values were associated with items relating to physical symptoms and depression. The subjects' perception of clicking and creaking also had "satisfactory" agreement, as did the question on seeking professionals for pain treatment. The remaining questions showed "good" and "very good" reproducibility and most of them presented the maximum agreement. CONCLUSION: The Portuguese adaptation of the questionnaire was shown to be reliable for detecting psychological and psychosocial abnormalities relating to temporomandibular disorders.

Reliability; Internal consistency; Reproducibility of results; Temporomandibular joint dysfunction; Diagnosis

OBJETIVO: Estudar a confiabilidade, da versão em português, do questionário para o diagnóstico psicológico e psicossocial dos indivíduos com desordens temporomandibulares (RDC/TMD). MÉTODOS: Foram entrevistados 109 indivíduos, de ambos sexos, que demandaram atendimento junto à Clínica de Fisioterapia do Centro Universitário de Araraquara, de janeiro a julho de 2006. Os questionários foram aplicados por um único examinador. Após duas semanas, o mesmo foi reaplicado em 36 indivíduos. Para avaliação da consistência interna do método, utilizou-se o Coeficiente Alfa de Cronbach; para análise da reprodutibilidade intra-examinador, o Coeficiente de Correlação Intraclasse (ro) e a estatística Kappa (kapa), respectivamente às variáveis de natureza quantitativa e qualitativa. RESULTADOS: A consistência interna para as dimensões intensidade da dor crônica e incapacidade; limitação da função mandibular; sintomas físicos não-específicos, incluindo os itens de dor; sintomas físicos não-específicos, excluindo os itens de dor e depressão foi de 0,8479, 0,8971, 0,8673, 0,8080 e 0,9270 respectivamente, atestando ao método excelente validade interna. Obteve-se "excelente" concordância intra-examinador para as questões referentes ao tempo de presença da dor e sua gradação, e "boa" para a questão referente à dor presente. Os menores valores de kapa relacionaram-se aos itens de sintomas físicos e depressão. A percepção de estalos ou rangidos pelos indivíduos apresentou concordância "regular" bem como a questão referente à procura de profissional para tratamento da dor. As demais questões apresentaram reprodutibilidade "boa" e "ótima", sendo que a maioria dessas apresentou nível máximo de concordância. CONCLUSÃO: A versão adaptada para o português mostrou-se confiável para detecção das alterações psicológicas e psicossociais associadas às desordens temporomandibulares.

confiabilidade; consistência interna; reprodutibilidade de resultados; disfunção da articulação temporomandibular; diagnóstico

SCIENTIFIC ARTICLES

Internal consistency and reproducibility of portuguese version of research diagnostic criteria for temporomandibular disorders (RDC/TMD-AXIS II)

Campos JADB^I; Carrascosa AC^II; Loffredo LCM^I; Faria JB^II

^IDepartment of Social Dentistry, Faculty of Dentistry, Universidade Estadual Paulista - UNESP, Araraquara, SP - Brazil

^IIDepartment of Food and Nutrition, Faculdade de Ciências Farmacêuticas, UNESP

Correspondence to

ABSTRACT

OBJECTIVE: To study the reliability of the Portuguese version of a questionnaire for psychologically and psychosocially diagnosing individuals with temporomandibular disorders (RDC/TMD).

METHOD: Interviews were held with 109 individuals of both sexes who required care at the physical therapy clinic of the Araraquara University Center (UNIARA) from January to July 2006. The questionnaires were applied by a single examiner. Two weeks later, the same questionnaire was reapplied to 36 individuals. To evaluate the internal consistency of the method, Cronbach's alpha coefficient was used. The intra-examiner reproducibility was analyzed using the intraclass correlation coefficient (ρ) for quantitative variables and Kappa (κ) statistics for qualitative variables.

RESULTS: The internal consistency for the dimensions of chronic pain intensity, disability, limitations relating to mandibular function, nonspecific physical symptoms with pain items included, and nonspecific physical symptoms with pain items and depression excluded, were 0.8479, 0.8971, 0.8673, 0.8080 and 0.9270 respectively, thus confirming the excellent internal validity of the method. The intra-observer agreement was found to be "excellent" for questions relating to pain duration and intensity and "good" for the question relating to current pain. The lowest κ values were associated with items relating to physical symptoms and depression. The subjects' perception of clicking and creaking also had "satisfactory" agreement, as did the question on seeking professionals for pain treatment. The remaining questions showed "good" and "very good" reproducibility and most of them presented the maximum agreement.

CONCLUSION: The portuguese adaptation of the questionnaire was shown to be reliable for detecting psychological and psychosocial abnormalities relating to temporomandibular disorders.

Key words: Reliability; Internal consistency; Reproducibility of results; Temporomandibular joint dysfunction; Diagnosis.

INTRODUCTION

Temporomandibular disorders (TMDs) are a collective term that refers to a group of musculoskeletal and joint problems that affect the temporomandibular joint and its associated structures. Among these disorders, myofascial pain stands out for being documented as the most prevalent cause of pain in that area^1-3 and having the potential to affect the sufferer's daily routine to a certain extent⁴.

The etiology of temporomandibular disorder is associated with various factors, such as occlusal imbalance, parafunctional oral behaviors, postural imbalance, hormonal variations, besides psychosocial and behavioral alterations^4-7. Dworkin and Leresche⁵ point out that, in literature, there is no consensus on diagnostic criteria. Kosminsky et al.⁸ emphasize that the lack of standard criteria to evaluate the aspects related to this pathology often leads to discrepant results. In this sense, Svensson⁹ cites the need to use universally accepted and validated classification criteria.

In order to achieve this, the Research Diagnostic Criteria for Temporomandibular Disorders (RDC/TMD - Axis I and Axis II)⁵ instrument was proposed which reflects the complex interaction between the physical and psychological dimensions of chronic pain. This instrument evolved into a double axis system in an attempt to provide reliable measurement of signs and symptoms of temporomandibular disorder (Axis I), as well as its associated psychological and psychosocial factors (Axis II).

The development of RDC/TMD aimed to establish reliable and valid criteria¹⁰ for diagnosing and defining TMD subtypes because one of the biggest methodological problems is the lack of precision in defining populations. The RDC/TMD established a classification system for research that consists of a self-administered questionnaire with 31 questions and a clinical examination form with 10 items, as well as clinical examination specifications and diagnostic criteria that allow classification of each case according to the patient's physical (Axis I) and psychological (Axis II) conditions. Axis I classifies the individuals in three categories, that is, muscular TMD, joint disc displacement and other temporomandibular joint (TMJ) conditions, while Axis II divides them according to chronic pain intensity and disability, degree of depression, scale of non-specific physical symptoms and limitation of jaw function.

The RDC/TMD was initially proposed in the English language, and the questionnaire was later translated by Pereira et al.¹¹. To make the cultural adaptation of this questionnaire, Kominsky et al.⁸ assessed the English into Portuguese translation and submitted the translated version to an assessment by a multidisciplinary committee, followed by a pre-test. According to goes et al.¹², this process of cultural adaptation resulted in an easily understood and administered instrument. However, it would be useful to verify the internal consistency and reproducibility of this questionnaire in order to provide a reliable standard instrument that measures the psychological and psychosocial factors related to temporomandibular disorders, and additionally provide the means to compare results. Therefore, this study aims to verify the internal consistency and the reproducibility of the Brazilian Portuguese version of the diagnostic criteria for RDC/TMD - Axis II.

MATERIAL AND METHODS

Sample

We assessed 109 individuals of both genders who requested care at the physical Therapy Clinic of Centro Universitário de Araraquara-UNIARARA between January and July 2006, using the convenience sampling method. The inclusion criteria were: minimum age of 20, at least twenty functional teeth in the oral cavity and a history of musculoskeletal facial pain. The exclusion criteria were: the presence of total or removable partial prosthesis, occlusal splints and the use of analgesic drugs.

This study was conducted after the approval by the ethics in Research Committee of Centro Universitário de Araraquara - UNIARA (protocol 361/2005), and subject participation was voluntary and linked to acceptance and signing of the written informed consent.

Instrument

As a measuring instrument, we used the cross-cultural adaptation to Brazilian portuguese of the Research Diagnostic Criteria for Temporomandibular Disorders RDC/TMD - Axis II produced by Kominsky et al.⁸. This questionnaire comprises 31 questions, however it was decided that, for the study of internal consistency and reproducibility, the 9 sociodemographic questions would not be included, limiting the questionnaire to 22 questions, which allow answers according to categories in nominal, ordinal, and numeric scale.

Experimental procedure

The questionnaires were administered by a single rater who was previously trained and familiar with the procedure. The questionnaire was administered again to a sample of 36 individuals after two weeks, during which time the patient received no treatment for pain.

Statistical analysis

To assess internal consistency, we used the standardized Cronbach Alpha Coefficient^13,14 (α) for the first administration of the questionnaire, as recommended by studies^14-16 that name as one of the Coefficient's advantages the possibility of calculation with only one administration of the questionnaire. There was an assessment of the internal consistency of the questions composing each proposed domain (chronic pain intensity and disability, limitation of jaw function, scale of non-specific physical symptoms, and degree of depression). After that, the intrarater reproducibility of all questions was estimated. Thus for quantitative variables, the Intraclass Correlation Coefficient (ρ)¹⁷ was estimated; based on the value of ρ, the agreement level was classified according to criteria suggested by Fermanian¹⁸. For the qualitative data, the weighted Kappa statistic was applied (κ)¹⁹, and the agreement level was classified according to landis and Koch²⁰. To achieve this, we used the STATA software²¹ in a Windows environment^®.

RESULTS

At the end of the study, the sample consisted of 23 male individuals (21.10%) and 86 female individuals (78.90%), with mean age of 31.65 years, standard error of 0.92, with a minimum age of 20 and maximum age of 50 years. In terms of instruction level, 3 participants had only elementary education; 6 had junior high education; 65 had completed high school; and 35 were college graduates.

Table 1 shows the internal consistency analysis in each domain (chronic pain intensity and disability; limitation of jaw function; scale of non-specific physical symptoms [pain items included]; non-specific physical symptoms [pain item excluded] and depression) proposed by the RDC/TMD - Axis II.

There is excellent internal consistency in all proposed domains. The homogeneity analysis for chronic pain intensity and disability showed that the variation of the α coefficient when each question was excluded was greater than 0.03, indicating a great contribution of questions 7, 8, and 9 to this domain's internal consistency. The α coefficient rose only with the exclusion of question 10. The same was observed in the non-specific physical symptoms domain (pain items excluded) when items c, w, and x of question 20 were eliminated, which shows their important contribution to this domain.

Table 2 includes the findings for intrarater reproducibility for the questions related to the presence and intensity of pain. For questions 4a, 4b, 8, 9, 11, 12, and 13, there was excellent agreement, whereas for question 7, it was classified as good.

Table 3 presents the Kappa statistic values for the qualitative questions. The lowest Kappa values observed, typical of poor agreement, were related to a few items from question 20 that refer to distress caused by loss of interest, lack of energy, feelings of guilt, lower back pain, loneliness, sorrow, nausea, muscle soreness, weakness, heavy feelings in the arms and legs, and overeating.

For the subjects' perception of popping or grinding (questions 15a and 15b), there was "fair" agreement. This "fair" agreement (κ = 0.5894) also draws attention in the question related to seeking a health care professional to treat facial pain. The remaining questions presented "good" and "very good" reproducibility, most of them with maximum agreement level.

DISCUSSION

In epidemiological studies, the quality of collected data represents one of the most relevant aspects, as well as the validation of the method to be employed^{8,12,16,22-27}. The concept of internal consistency of methods based on questionnaires must be verified through the Cronbach α coefficient¹¹ and has been widely used^{12,14,16,28,29} to determine both the reliability of the questionnaire in general and the adequacy of each question inserted therein^30,31.

Internal consistency and interrater reproducibility analysis of the RDC/TMD - Axis I is available in current literature^24-27,32. However, when it comes to internal consistency³³ and intrarater reproducibility of the RDC/TMD - Axis II, studies are scarcer, especially for an instrument in the portuguese language.³⁴ Consequently, this study is considered a very important contribution to researchers in the field of TMDs in terms of the reliability of a culturally-adapted assessment tool in Brazilian Portuguese.

The internal consistency of the proposed domains proved to be excellent, and the high correlation among the items comprising each domain (Table 1) indicated that they are part of a single conceptual dimension, resulting in a reliable measuring scale. Regarding the homogeneity of the questions within each domain, it is recommended that the variation of the test's α value not exceed 0.03 when each individual question is excluded.

The smallest homogeneity recorded among the items of chronic pain intensity and disability and non-specific physical symptoms (pain items excluded) may have occurred because of their reduced number of questions¹³. However, this fact does not threaten their reliability because the correlation α coefficients proved to be adequate, i.e. above 0.20²².

In a validation study of the portuguese version of RDC/TMD - Axis II, Lucena et al.³⁴ found α= 0.73 for jaw function limitation, which is considered an adequate value, although lower than what was found in the present study (α= 0.8971). For the original version of the questionnaire (in English), Dworkin et al.³³ found α values of 0.91, 0.82, 0.78 and 0.71 for the domains of depression, non-specific physical symptoms (pain items included), non-specific physical symptoms (pain items excluded), and chronic pain intensity and disability respectively, similar to those shown in Table 1.

Stability has been determined by test-retest, also known as reproducibility, commonly used in studies on temporomandibular disorders. However, Braga¹⁵ points out that, depending on the type of assessment desired, this type of approach is not viable because it requires two assessments.

We believe that the verification of reproducibility is of utmost importance to the quality of the Brazilian Portuguese version of the questionnaire. However, attention must be given to the variability that can occur in symptomatology due to the interval between the two applications of the questionnaire. This fact can be observed in Table 2, which shows "good" reproducibility when facial pain is questioned during assessment, while reproducibility was classified as "excellent" in the other questions referring to the history of presence and intensity of pain. This may have occurred because this question refers to the exact moment the assessment was taking place and, because the individuals were questioned in two different occasions, there may have been alterations in symptomatology. These findings show that the questions were clearly worded, which contributes to their very reliable measures for the presence and intensity of pain.

Poor levels of reproducibility were verified for highly subjective questions (question 20 - Table 3), such as the perception of various symptoms, feelings of guilt, lack of interest, sorrow, loneliness, and heavy feeling in the arms or legs, which may have occurred because of the two-week interval between assessments, possibly leading to an alteration in the cited conditions. We recommend a shorter interval in future studies.

The question on seeking treatment for facial pain had fair agreement (κ = 0.5894), which can be explained by the fact that the individuals initially answered that they had not sought treatment, and later, this answer was altered, probably because some of the individuals saw the first application of the questionnaire as a treatment visit.

It is difficult to compare the findings of this study with others of the same nature because there are no studies in literature that investigated intrarater reproducibility of the RDC/TMD - Axis II. given the importance of this type of study for determining the reliability of a measuring instrument, we recommend that more studies be conducted. In light of that, it can be stated that the cross-cultural adaptation, test validation and pre-test of this version of the RDC/TMD - Axis II questionnaire, prepared by Kominsky et al.⁸, was a process that resulted in a clearly-worded and very reliable instrument.

CONCLUSION

Given the internal consistency and reproducibility results, we can conclude that the Brazilian portuguese version of the questionnaire (Research Diagnostic Criteria for Temporomandibular Disorders RDC/TMD - Axis II) proved to be a reliable instrument to detect psychological and psychosocial alterations associated with temporomandibular disorders.

REFERENCES

• 1. Fricton J, Kroening R, Haley D, Siegert R. Myofascial pain and dysfunction of the head and neck: a review of the clinical characteristics of 164 patients. Oral Surg Oral Med Oral Pathol. 1985;60:615-23.
• 2. Skootsky AS, Jaeger B, Oye RK. Prevalence of myofascial pain general internal medicine practice. West J Med. 1989;151: 157-60.
• 3. Fricton JR, Dubner R. Dor orofacial e desordens temporomandibulares. Santos: São Paulo; 2003.
• 4. Mcneill C. Temporomandibular disorders: guidelines for classification, assessment and management. 2Ş ed. Chicago: Quintessence; 1993.
• 5. Dworkin SF, Leresche L. Research diagnostic criteria for temporomandibular disorders: review, criteria, examinations and specifications, critique. J Craniomandib Disord. 1992;6:301-55.
• 6. Von Korff MR, Dworkin S, Fricton J, Orbach R. Research diagnostic criteria part II. J Craniomand Disord. 1992; 6(4):327-55.
• 7. Yap AUJ, Tan KBC, Chua EKC, Tan HH. Depression and somatization in patients with temporomandibular disorders. J Prosth Dent. 2002;88(5):479-84.
• 8. Kosminsky M, Lucena LBS, Siqueira JTT, Pereira Jr FJ, Góes PSA. Adaptação cultural do questionário research diagnostic criteria for temporomandibular disorders: Axis II para o português. J Bras Clin Odontol Integr. 2004;8(43):51-61.
• 9. Svensson P. Orofacial musculoskeletal pain. In: Giamberardino MA. Pain 2002 Na updated review. Refresher course syllabus. Seattle: IASP Press; 2002. p. 447-66.
• 10. Pehling J, Schiffman E, Look J, Shaefer J, Lenton P, Fricton J. Interexaminer reliability and clinical validity of the temporomandibular index: a new outcome measure for temporomandibular disorders. J Orafac Pain. 2002;16(4): 296-304.
• 11. Pereira F, Huggins KM, Dworkin SF. Critérios de diagnóstico para pesquisa das desordens temporomandibulares RDC/DTM [homepage na Internet]. Seattle: RDC/TMD; 2002 {atualizada em 20 Fev 2005; acesso em 05 Jun 2006]. Disponível em URL: http://www.rdc-tmdinternational.org/translations/frmtranslations.htm
• 12. Góes PSA, Fernandes LMA, Lucena LBS. Validação de instrumentos de coleta de dados. In: Antunes JLF, Peres MA. Fundamentos de odontologia Epidemiologia da saúde bucal. Rio de Janeiro: Guanabara Koogan; 2006. p. 390-7.
• 13. Cronbach LJ. Coefficient alpha and the internal structure of the tests. Psychology. 1951;16:297-334.
• 14. Cronbach LJ. My current thoughts on coefficient alpha and successor procedures. Educ Psychol Meãs. 2004;64(3): 391-418.
• 15. Braga W. Algumas considerações sobre testes e demais avaliações de alunos [monografia na internet]. Rio de Janeiro: PUC; 2004 [acesso em 23 junho 2006]. Disponível em: http://wwwusers.rdc.puc-rio.br/wbraga/transcal/pdf/Papers/cobem051.pdf
• 16. Yu CH. An introduction to computing and interpreting Cronbach coefficient alpha in SAS. Statistics, Data analysis and Data mining [periódico na internet]. 2000 [acesso em 02 junho 2006]; 1:[aproximadamente 6 p.]. Disponível em: http://www2.sas.com/proceedings/sugi26/p246-26.pdf
• 17. Fisher RA. Intraclass correlations and the analysis of variance. In: Fisher RA. Statistical methods for research workers. 14Ş ed. Edinburg: Oliver and Boyd; 1970. p. 213-49.
• 18. Fermanian J. Measure de l'accord entre deux juges: cas quantitatif. Rev Epidemiol Sante Publique. 1984;32:408-13.
• 19. Light RJ. Measures of response agreement for qualitative data: some generalizations and alternatives. Psychol Bull. 1971;76: 365-77.
• 20. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159-74.
• ²¹
  STATA. Stata Statistical Software. Release 8.0 [software em CD-ROM]. Texas (USA): Stata Corporation; 2003.
• 22. Streiner D, Noeman G. Health measurement scales. A practical guide to their development and use. Oxford: Oxford University Press; 1995.
• 23. Coleman GC, Nelson JF. Princípios de diagnóstico bucal. Rio de Janeiro: Guanabara Koogan; 1996.
• 24. John MT, Dworkin SF, Mancl LA. Reliability of clinical temporomandibular disorder diagnoses. Pain. 2005;118:61-9.
• 25. Schitter M, Ohlmann B, John MT, Hirsch C, Rammelsberg P. Research diagnostic criteria for temporomandibular disorders: a calibration and reliability study. Cranio. 2005;23(3):212-8.
• 26. Hassel AJ, Rammelsberg P, Schitter M. Inter-examiner reliability in the clinical examination of temporomandibular disorders: influence of age. Community Dent Oral Epidemiol. 2006;34: 41-6.
• 27. Nilsson I, List T, Drangsholt M. The reliability and validity of self-reportes temporomandibular disorder pain in adolescents. J Orofac Pain. 2006;20:138-44.
• 28. Fleck MPA, Louzada S, Xavier M, Chachamovich E, Vieira G, Santos L, et al. Aplicação da versão em português do instrumento abreviado de avaliação da qualidade de vida "WHOQOL-bref". Rev Saúde Pública. 2000;34(2):178-83.
• 29. Hill MM, Hill A. Investigação por questionário. Lisboa: Silabo; 2005.
• 30. Guyatt GH, Feeny DH, Patrick DL. Measuring health-related quality of life. Ann intern Med. 1993;118:622-9.
• 31. Schiffman E, Ohrbach R, Truelove E. Research diagnostic criteria: reliability and validity. Bethesda: National Institutes of Health; 2005.
• 32. Lobbezoo F, Selms MKA, John MT, Huggins K, Ohrbach R, Visscher CM, et al. Use of the research diagnostic criteria for temporomandibular disorders for multinational research: translation efforts and reliability assessments in the Netherlands. J Orofac Pain. 2005;19:301-8.
• 33. Dworkin SF, Sherman J, Mancl L, Ohrbach R, LeReshe L, Truelove E. Reliability, validity, and clinical utility of the research disgnostic criteria for temporomandibular disorders Axis II scales: depression, non-specific physical symptoms, and graded chronic pain. J Orofac Pain. 2002;16:207-20.
• 34. Lucena LBS, Kosminsky M, Costa LJ, Góes PSA. Validation of the Portuguese version of the RDC/TMD Axis II questionnaire. Braz Oral Res. 2006;20(4):312-7.

Publication Dates

• Publication in this collection
  11 Dec 2008
• Date of issue
  Dec 2007

History

• Accepted
  21 Aug 2007
• Reviewed
  29 Mar 2007
• Received
  29 Nov 2006