Cross-cultural adaptation of simplified version (s) of Behçet's Disease Current Activity Form (BDCAF) and comparison between two different instruments with Brazilian versions for evaluating Behçet's Disease Activity: BR-BDCAF and BR-BDCAF(s)

Abstracts

OBJETIVO: Um instrumento de avaliação da atividade da Doença de Behçet (DB) denominado Behçet's Disease Current Activity Form (BDCAF) foi originalmente elaborado em língua inglesa e adaptado para a língua portuguesa do Brasil na versão BR-BDCAF. Recentemente, um modelo simplificado desse protocolo (com 12 itens) foi apresentado pela Sociedade Internacional para a Doença de Behçet (International Society for Behçet's Disease, ISBD) para produzir um índice denominado BDAI (Behçet's Disease Activity Index). Por esse motivo, neste trabalho produzimos o modelo simplificado da versão brasileira, medimos sua confiabilidade e avaliamos a validade dos resultados de ambos os instrumentos, comparando seus desempenhos. MÉTODO: O modelo simplificado da versão brasileira BR-BDCAF, adaptada transculturalmente, foi denominado BR-BDCAF(s). Ambos os protocolos foram aplicados em 25 pacientes com DB. A confiabilidade do BR-BDCAF(s) foi avaliada através das reprodutibilidades intra e interobservadores pela estatística kappa. A validade e as propriedades diagnósticas sensibilidade(S), especificidade(E) e acurácia dos dois protocolos na definição de casos ativos da doença foram avaliadas pela comparação ao julgamento clínico de um reumatologista perito em DB, e os melhores pontos de corte foram estabelecidos para cada instrumento através da curva ROC (receive-operator characteristic). RESULTADOS E CONCLUSÕES: BR-BDCAF(s) apresentou boa confiabilidade nas questões sobre manifestações mucocutâneas e articulares da DB. Os melhores pontos de corte para a definição de casos ativos foram índices maiores que quatro no BR-BDCAF (S = 80,0%, E = 86,7%) e maiores que um no BR-BDCAF(s) (S = 70,0%, E = 86,7%). Suas acurácias foram semelhantes, sugerindo que ambos podem ser utilizados como instrumentos de medida de atividade da DB.

Doença de Behçet; tradução; Adaptação transcultural; índice de atividade de doença

ORIGINAL ARTICLE

^IRheumatologist, postgraduate student in Medical Sciences in Rheumatology at FMUSP

^IIResident in Rheumatology at HC-FMUSP

^IIIRheumatologist, Assistant at Rheumatology Department at HC-FMUSP

^IVAssistant Professor in Rheumatology at HC-FMUSP

Correspondence to

ABSTRACT

OBJECTIVE: Behçet's Disease Current Activity Form (BDCAF), an instrument for assessing activity of Behçet's Disease (BD), was originally produced in English. Its cross-cultural adaptation to Brazilian Portuguese language generated a version called BR-BDCAF. Recently, a simplified model of this protocol (with 12 items) was developed by the International Society for Behçet's Disease (ISBD) to generate an index called BDAI (Behçet's Disease Activity Index). For this reason, in this work we intended to create the simplified model of the Brazilian version, measure their reliability and evaluate the validity of the results of both instruments, comparing their performances.

METHODS: The simplified version of BR-BDCAF was called BR-BDCAF(s). Both protocols were applied in 25 BD patients. Reliability of BR-BDCAF(s) was evaluated through intra- and interobserver agreement according to kappa statistic. Sensitivity (Se), specificity (Sp) and accuracy of both protocols on defining BD active cases were evaluated by comparing the rheumatologist expert clinical judgement to each protocol, and the best cut-off points were established for each instrument by the ROC curve (receive-operator characteristic).

RESULTS AND CONCLUSIONS: BR-BDCAF(s) showed good reliability on items relating to mucocutaneous and articular manifestations of BD. The best cut-off points for the definition of active cases were: higher than four with BR-BDCAF (Se = 80.0%, Sp = 86.7%) and higher than one with BR-BDCAF( s) (Se = 70.0%, Sp = 86.7%). Accuracy of both protocols was similar, suggesting that both can be used as tools to assess activity of DB.

Keywords: Behçet's Disease, translation, Cross-cultural comparison, Disease activity index.

INTRODUCTION

Translation and validation of assessment instruments in rheumatology into Brazilian Portuguese language has been recently acknowledged in a late editorial in this journal.¹ Behçet Disease (BD), a multisystem inflammatory disorder characterized by ocular, cutaneous, genital, and oral vasculitic injuries, and several other less frequent manifestations,² is one of the rheumatologic entities that need standardized assessment instruments in Brazilian literature.

In 1999, Bhakta et al.³ defined a clinical instrument named Behçet's Disease Current Activity Form (BDCAF) to describe and assess BD activity. It is a form with several questions presented as a guideline for the clinical interview with BD patient, assessing different manifestations of active disease over the preceding four weeks. Most symptoms assessed by BDCAF are quantified from zero to four, according to the duration of the symptom in weeks. It is important to explain that it is not a patient self-assessment form, but a guide for the clinical interview. The clinician's judgment is necessary to fill it out by interpreting complaints presented by the patient, related or not to BD.

Since its initial presentation, BDCAF has been used in some clinical trials as an instrument to relate the presence or intensity of BD activity to laboratory or clinical outcomes.^4-6 We performed a crosscultural adaptation of the BDCAF to Brazilian Portuguese language and tested its reliability among Brazilian patients creating a version named BR-BDCAF (Fig. 1), published elsewhere.⁷ Adaptation process from original English version BDCAF was conducted according to the guidelines proposed by Beaton et al.^8,9 Concisely, two translations (T1 and T2) had been made by two native Brazilian bilingual physicians (FSN and JCBM); T1 and T2 had been combined by their translators together with a Professor of Portuguese Language (MBL) in a common version (T12). From T12, two back translations into English (BT1 and BT2) had been made by native-speaking English teachers, non doctors (LV and CB). During a final meeting, professionals involved in adaptation, solved occasional discrepancies and elaborated a final Portuguese version. Its performance has been considered satisfactory for common BD manifestations (general, mucocutaneous and articular), moderate for ocular manifestations and poor for visceral manifestations (gastrointestinal tract, central nervous system and major vessel).⁷ These results are similar to those found using the original English BDCAF⁴ and also to the Turkish version.¹⁰ Considering only the 8 most reliable questions (rated from zero to 4) about mucocutaneous, articular and general manifestations, BR-BDCAF is able to produce a score varying from zero to 32 associated with disease activity (Figure1).

Later on, the International Society for Behçet's Disease (ISBD) assessed the results of BDCAF in three countries (United Kingdom, Turkey and Korea) trying to create a unidimensional index linked to the degree of general BD activity.¹¹ Twelve original BDCAF items had been considered valid for this purpose: (1) headache, (2) oral ulcers, (3) genital ulcers, (4) erythema nodosum, (5) pustules, (6) arthralgia, (7) arthritis, (8) abdominal pain/nausea/vomit, (9) gastrointestinal bleeding, (10) ocular symptoms, (11) impairment of Central Nervous System (CNS), (12) impairment of major vessel. Some items grading (from zero to 4), as proposed in original BDCAF, had not been kept. In order to obtain activity index, all variables have been considered to be dichotomous (like 'yes/no'). So another index is produced, using variables ranging from zero to 12, in order to associate with the disease activity, also including the different visceral injuries of BD, additionally to articular and mucocutaneous manifestations. From these data, these 12 items from the Brazilian BR-BDCAF version (previously created) have been selected. These items had constituted a simplified instrument of disease assessment. Reliability of the new index was assessed in a population of Brazilian BD patients. Validation of both instruments was assessed considering gold standard the opinion of a BD rheumatologist expert about disease's activity and, finally, accuracy of both instruments in BD activity assessment had been compared.

MATERIAL AND METHODS

Cross-cultural adaptation. From the Brazilian BR-BDCAF version, previously cross-culturally adapted to Portuguese language⁷ from the English original BDCAF,⁴ a simplified version, with only 12 dichotomic items, have been created by the same authors (FSN and JCBM). This version was applied to a pre-test population of 14 BD patients. The items considered more difficult to interpret by patients during protocol application were additionally explained using laical terms. Final simplified version was named BR-BDCAF(s).

Interviews and patients. Two second-year Rheumatology residents with similar experience (CAMC and DMM), were, respectively: Observer 1 and Observer 2. They interviewed, in an independent way, each patient, on the same day of their appointment at Behçet's Disease clinic at Rheumatology Department at Hospital das Clínicas of the Medical School of the University of São Paulo. The Brazilian version BR-BDCAF(s) was used, with an average time of 20 minutes for each interview. Patients were invited to come back after 24 to 48 hours to repeat interview. Twenty-five patients attended at the clinic between April and July of 2007, were included in this study. An interview using complete BR-BDCAF protocol was also carried out with each patient, as part of routine care of BD patients, almost 30 minutes before BR-BDCAF(s) interviews on the first evaluation day, by the resident doctor responsible for this appointment. Every patient met diagnosis criteria of International Study Group for Behçet's Disease (ISGBD)¹² and were more than 18 years old at the moment of interview. Local ethics committee approved this study.

Reliability, validity, accuracy and statistic analysis. Reliability of the results obtained with version BR-BDCAF(s) was evaluated according to two criteria: Reproducibility (concordance) and bias. The level of intraobserver and interobservers agreement was assessed by calculating the kappa statistics.

This index ranges between zero and one, with values close to one indicating high agreement. In this study, values of kappa were interpreted according to recommendations by Landis & Koch (between 0.21 and 0.40, fair agreement; 0.41 and 0.60, moderate; 0.61 and 0.80, substancial; 0.81 and 1.00, almost perfect).¹³ possibility of bias (systemic evaluation deviation) interobservers and intraobservers had been evaluated using Student's t-test.

BR-BDCAF(s) validity to identify active BD cases had been assessed comparing it's average scores obtained in interviews held by resident doctors (observers 1 and 2) with the therapeutic intervention of the BD rheumatologist expert (CRG). He assessed personally each patient included in this study, with no knowledge of the BR-BDCAF or BR-BDCAF(s) score obtained during interviews held by resident doctors and his assessment had been considered golden standard when defining active cases of disease. This physician procedure was assessed by retrospective review of patient's record, by an observer (FSN) also blind to BR-BDCAF or BR-BDCAF(s) index. Cases were considered active when the therapeutic intervention proposed by the expert physician included introduction or increment of medicine doses that control disease manifestations. BR-BDCAF(s) sensitivity and specificity in different values of activity index to identify active cases have been established, and the best cut-off value for the index established by inflection point location of ROC curve.

In order to compare both instruments' performance, BR-BDCAF protocol's validity had been assessed in the same population of patients. Accuracy of both instruments to identify active cases was then compared analyzing the area under ROC curves. For statistic analysis, software SPSS™ 15.0 and MedCalc™ 9.4-2.0 for Windows™ were used. Sample distribution had been assessed by Kolmogorov-Smirnov test, correction by Liliefors, and comparison of variables with normal distribution between two groups, had been performed by Student's t-test.

RESULTS

Cross-cultural adaptation process resulted in a Brazilian version named BR-BDCAF(s) (Figure 2).

This version was applied to 25 patients by Observers 1 and 2 on the first assessment day to evaluate interobserver agreement. There was 14 (56%) female patients and 11 (44%) male patients, age ranging from 42.2 ± 12.2 years old (average ± standard deviation) and disease duration ranging from 11.0 ± 8.2 years. Twelve patients came back for the second interview day, allowing us to assess intraobserver agreement. Significant results of agreement analysis are shown on Table 1 (values presented represent statistic measure kappa ± standard error). We observed that the only item with fair interobserver agreement was headache, which we assigned to assessment subjectivity of this symptom by the clinician (assigning this common symptom to BD activity or considering it only as an unrelated incident). There was no sufficient number of gastrointestinal bleeding, ocular symptoms, CNS or major vessel cases for statistic evaluation of such items. There was no significant interobserver or intraobserver bias.

Among 25 patients assessed with BR-BDCAF(s), 10 had been considered active from the rheumatologist expert point of view. In this group, average ± standard deviation of BR-BDCAF(s) index was 2.15 ± 1.0. Fifteen patients were considered inactive and, in this group, average ± standard deviation of index was 0.80 ± 0.70, p = 0.001. In the active group, average BR-BDCAF was 6.40 ± 2.95; in the inactive group, average BR-BDCAF was 2.07 ± 1.79, p < 0.001.

Considering the cut-off value to establish active cases, index equal or higher than 2 for BR-BDCAF(s), we established 70% of sensitivity (34.8% to 93%) and specificity of 86.7% (59.5% to 97.9%). For BR-BDCAF, considering the cut-off value to establish active cases, index equal or higher than 5, we found 80.0% of sensitivity (57.1% to 92.0%) and specificity of 86.7% (71.4% to 94.7%) - values in brackets represent reliability range of 95% of proportions. Cut-off values had been chosen using ROC curve analysis. There was no significant difference among areas under curves for BR-BDCAF(s) and BR-BDCAF (0.870 ± 0.070 and 0.937 ± 0.045, respectively) (Figure 3).

DISCUSSION

Previous work using BR-BDCAF version (Figure 1) assessed 40 BD patients⁷ and found that those items referring to visceral manifestations (gastrointestinal tract, CNS and major vessel) obtained fair interobserver agreement (kappa lower or equal 0.4), while ocular manifestations obtained moderate agreement (kappa ranging from 0.4 to 0.6). Poor performance of the instrument in these items have been assigned especially to the need of specialized ophthalmologic assessment to evaluate ocular activity and other complementary exams (such as digestive endoscopy, imaging tests), in addition to clinical assessment to evaluate visceral activity of disease.

The other eight BDCAF items (fatigue, headache, oral ulcers, genital ulcers, pustules, erythema nodosum, arthralgia and arthritis - corresponding to BD general, mucocutaneous and articular manifestations highlighted in Figure1) obtained ideal agreement levels, indicating that these variables associated with disease activity can be readily assessed in routine clinical practice.

Those questions with symptoms scored from zero to 4 allowing BDCAF instrument to create an index varying from 0 to 32, correlating with disease's activity. We consider this index particularly useful in patients without ocular inflammation or visceral impairment. Whether disease activity in ocular, gastrointestinal, major vessel, or CNS is suspected, specialist consultation and complementary tests are indicated for a detailed assessment of the specific organ system and may be separately added to mucocutanous-articular disease's activity score (for example, a patient with oral ulcers for 4 weeks, genital ulcers for 2 weeks and active uveitis should be described as "active Behçet's Disease patient, BR-BDCAF = 6 + Uveitis"). In BDCAF protocol, ocular or visceral manifestations are not graded from zero to four they are treated only as dichotomic variables ('yes' or 'no') - a way to add BD visceral manifestation intensity to BDCAF index has not been thought, creating a sole general index associated with general disease activity. That is why we consider the index reliable considering the items highlighted in Figure 1, representing the articular-mucocutaneous activity score of disease.

Reliability of simplified version BR-BDCAF(s) on items associated with mucocutaneous articular BD manifestations (Table 1) was similar to that found using BR-BDCAF protocol, and that is why we also considered its use in BD activity assessment. Different from BR-BDCAF, the simplified protocol is capable of producing a single index also covering the disease visceral manifestations. However, as in the full protocol, we believe that the assessment of disease activity in these systems is not reliable when performed only in clinical bases, and must be checked through additional tests.

Both instruments can identify and describe active BD cases, with similar accuracy (Figure 3). However, with different features: instrument BR-BDCAF seems more suitable to evaluate cases in which mucocutaneous and articular manifestation intensity is the most interesting point. An intervention producing reduction in oral ulcers frequency (for example, 4 weeks to 2 weeks during the last month) may be seen by a 2.0 reduction in BR-BDCAF index. On the other hand, the BR-BDCAF(s), in contrast, does not address the intensity of events within a specific organ system, and consequently does not have sufficient variability to detect this difference. Considering the same example above, BR-BDCAF(s) would assign a point for the presence of oral ulcers in both situations, and the effectiveness of intervention would not be perceived. On the other hand, by including different organic systems in a single index, BR-BDCAF(s) seems more useful in the assessment of cases involving multiple systems, including those with visceral manifestations, by allowing a rapid verification of BD activity score as a whole. However, we emphasize that, additionally to index, detailed description of involvement of any organ system, based on complementary tests, is always necessary for a correct evaluation of disease activity.

Brazilian versions of two international standardized tools were presented for evaluation of BD activity score. Both are valid and provide a good performance to identify active cases of disease in relation to articular and mucocutaneous manifestations. Each instrument has its own features that make them suitable for specific situations, especially in clinical trials involving BD in our setting .

Acknowledges: English teachers Lucille Vogel and Claudio Bizocchi and Portuguese teacher Maria Beatriz Leite, physicians at Behçet's Disease Clinic, Dr. Cláudia Goldenstein-Schainberg and Dr. Laís Verderame Lage.

REFERENCES

• ¹
  Lage LV, Levy RA, Ciconelli RM. Instrumentos de avaliação em Reumatologia: importância de sua tradução e validação em nosso idioma. Rev Bras Reumatol. 2006;46:237.
• ²
  Yurdakul S, Hamuryudan V, Yazici H. Behçet syndrome. Curr Opin Rheumatol. 2004;16:38-42.
• ³
  Bhakta BB, Brennan P, James TE, Chamberlain MA, Noble BA, Silman AJ. Behçet's disease: evaluation of a new instrument to mesure clinical activity. Rheumatology. 1999;38:728-33.
• ⁴
  Monastero R, Camarda C, Pipia C, Lopez G, Camarda LKC, Baiamonte V, et al Cognitive impairment in Behçet's disease without overt neurological involvement. J Neurol. Sci 2004;220:99-104.
• ⁵
  Bodur H, Borman P, Ozdemir Y, Atan C, Kural G. Quality of life and life satisfaction in patients with Behçet's disease: relationship with disease activity. Clin Rheumatol. 2006;25:329-33.
• ⁶
  Kwon SR, Lim MJ, Park SG, Moon YS, Park W. Decreased protein S activity is related to the disease activity of Behçet's disease. Rheumatol Int. 2006;27:39-43.
• ⁷
  Neves FS, Moraes JC, Kowalski SC, Goldenstein-Schainberg C, Lage LV, Gonçalves CR. Cross-cultural adaptation of the Behcet's Disease Current Activity Form (BDCAF) to Brazilian Portuguese language. Clin Rheumatol. 2007;26:1263-7.
• ⁸
  Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993;46:1417-32.
• ⁹
  Beaton DE, Bombardier C, Guillemin F, ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine. 2000;25:3186-91.
• ¹⁰
  Hamuryudan V, Fresko J. Evaluation of the Turkish translation of a disease activity form for Behçet's syndrome. Rheumatology. 1999;38:734-6.
• ¹¹
  Lawton G, Bhakta BB, Chamberlain MA, Tennant A. The Behçet's disease activity index. Rheumatology. 2004;43:73-8.
• ¹²
  International Study Group for Behçet's Disease. Criteria for diagnosis of Behçet's Disease. Lancet. 1990;150:1078-80.
• ¹³
  Landis JR, Koch G. The measurement of observer agreement for categorical data. Biometrics 1977;33:159-174.

Cross-cultural adaptation of simplified version (s) of Behçet's Disease Current Activity Form (BDCAF) and comparison between two different instruments with Brazilian versions for evaluating Behçet's Disease Activity: BR-BDCAF and BR-BDCAF(s)

Fabrício de Souza Neves^I; Cezar Augusto Muniz Caldas^II; Danielle Martins de Medeiros^II; Júlio César Bertacini de Moraes^III; Célio Roberto Gonçalves^IV

Publication Dates

• Publication in this collection
  03 Mar 2009
• Date of issue
  Feb 2009

History

• Received
  16 Feb 2008
• Accepted
  24 June 2008