Validity and limitations of the Brazilian version of the Composite International Diagnostic Interview (CIDI 2.1)

Validade e limitações da versão brasileira do Composite International Diagnostic Interview (CIDI 2.1)

ORIGINAL ARTICLE

Validity and limitations of the Brazilian version of the Composite International Diagnostic Interview (CIDI 2.1)

Validade e limitações da versão brasileira do Composite International Diagnostic Interview (CIDI 2.1)

Maria Inês Quintana^I; Fábio Leite Gastal^II; Miguel Roberto Jorge^I; Cláudio Torres Miranda^III; Sérgio Baxter Andreoli^I,IV

^IUniversidade Federal de São Paulo (UNIFESP), São Paulo (SP), Brazil

^IINational Hospital Accreditation Organization of Brazil, Brasília (DF), Brazil

^IIIPan American Health Organization, Division of Mental Health, Washington, DC, USA

^IVUniversidade Católica de Santos (UNISANTOS), Santos (SP), Brazil

^{Correspondence}

ABSTRACT

OBJECTIVE: To study the concurrent validity of the Brazilian Composite International Diagnostic Interview 2.1 using as gold standard the clinical diagnoses based on the ICD-10 criteria and the Longitudinal, Expert, All Data (LEAD) procedure.

METHOD: The sample was composed of 185 subjects selected at psychiatric hospitals, psychiatric outpatient units, the community, and primary care services. These individuals were intentionally selected according to 9 diagnostic groups. Instruments: Composite International Diagnostic Interview (CIDI-core) version 2.1 (paper-and-pencil) administered by 16 trained interviewers. Analysis: concurrent validity of diagnoses of the Composite International Diagnostic Interview 12-month.

RESULTS: Values found for sensitivity and specificity in each diagnosis were: alcohol-related disorders (79.5%/97.2%); psychoactive substance-related disorders (77.3%/100%); schizophrenia and other psychotic disorders (28.6%/93.9%); manic episode and bipolar affective disorder (38.9%/96.4%); depressive disorder (82.5%/ 93.8%); phobic-anxiety disorder (80.6%/93.5%); obsessive-compulsive disorder (18.2%/98.9%); somatoform disorder (41.7%/90.8%); eating disorder (45.5%/100.0%).

CONCLUSION: The Composite International Diagnostic Interview proved to be valid for diagnoses of alcohol-related disorders, psychoactive substance-related disorders, depressive disorder and phobic-anxiety disorder. The probable explanations for the poor performance for the other diagnoses were: necessity of some clinical judgement by the lay interviewer; difficulty to use the Probe Flow Chart; interviewees' difficulty of understanding; and lack of mechanisms to certify the veracity of the information.

Descriptors: Diagnosis; Psychiatric status rating scales; Validity of tests; Interview, psychological

RESUMO

OBJETIVO: Validação concorrente da versão brasileira do Composite International Diagnostic Interview 2.1, utilizando como padrão ouro o diagnóstico médico baseado nos critérios diagnósticos da CID-10 e critérios Longitudinal, Experts Clinicians, All Data (LEAD).

MÉTODO: Amostra composta por 185 indivíduos procedentes de hospitais psiquiátricos, ambulatórios de especialidades psiquiátricas, serviços comunitários e atenção primária à saúde, selecionados intencionalmente segundo nove grupos diagnósticos. Instrumentos: CIDI 2.1 (lápis e papel), versão para diagnósticos ao longo da vida, aplicado por 16 entrevistadores treinados. Análise: validade concorrente dos diagnósticos do Composite International Diagnostic Interview no último ano.

RESULTADOS: Os valores encontrados de sensibilidade e especificidade foram: transtornos decorrentes do uso de álcool (79,5%/97,2%); transtornos decorrentes do uso de substâncias psicoativas (77,3%/100%); esquizofrenia e outros transtornos psicóticos (28,6%/93,9%); episódio maníaco e transtorno afetivo bipolar (38,9%/96,4%); transtorno depressivo (82,5%/93,8%); transtorno fóbico-ansioso (80,6%/93,5%); transtorno obsessivo-compulsivo (18,2%/98,9%); transtorno somatoforme (41,7%/90,8%); e transtorno alimentar (45,5%/100,0%).

CONCLUSÃO: O Composite International Diagnostic Interview mostrou-se válido para os diagnósticos de transtornos decorrentes do uso de álcool e substâncias psicoativas, transtorno depressivo e transtorno fóbico-ansioso. As prováveis explicações para o pior desempenho nos demais diagnósticos foram: necessidade de algum julgamento clínico do entrevistador leigo; dificuldade no manuseio do Diagrama de Especificação de Resposta; dificuldade de compreensão dos entrevistados; e falta de mecanismos para atestar a veracidade das informações.

Descritores: Diagnóstico; Escala de graduação psiquiátrica; Validade dos testes; Entrevista, psicológica

Introduction

The Composite International Diagnostic Interview (CIDI) is a fully standardized, structured interview that provides a psychiatric diagnosis through computerized algorithms¹ according to the International Classification of Diseases, 10^th edition (ICD-10),² and the Diagnostic and Statistical Manual of the American Psychiatric Association, 4^th edition (DSM-IV).³ The CIDI was developed in 1980 by the World Health Organization (WHO) in collaboration with the former US Alcohol, Drug Abuse and Mental Health Administration (ADMHA) as a Joint Project for the Diagnosis and Classification of Mental Disorders, and Alcohol- and Drug-Related Problems. Its greatest appeal is that it was designed to be administered by trained lay interviewers in epidemiological studies, clinical trials and research centers.^4-8 It can be administered to individuals aged above 18 years, regardless their social, economic and cultural status, and does not depend on the patients' literacy.^7-9

The CIDI is available in paper-and-pencil and computer-administered forms (self-administered) with a diagnostic coverage for both lifetime (Core version) and 12-month assessment. The average time to administer the questionnaire is 75 minutes.⁸ The questions are explicit and positive answers are further explored by a specified probing system, the Probe Flow Chart (PFC). The PFC leads the interviewer to a standardized decision tree (algorithm) that will determine if the symptom was present but was not important enough for the individual to seek assistance (code 2); if the symptom occurred but was due to the use of medication, drug, alcohol, or caused by trauma or a physical disorder (code 3 and 4) or, finally, if it is a psychiatric symptom (code 5). The CIDI comprises 288 symptom questions distributed throughout 14 sections, 10 of which are for diagnostic purposes and 4 are non-diagnostic (Table 1). Training for the use of the CIDI 2.1 should follow the norms and regulations established by the WHO.⁸

The objective of this study is to assess the concurrent validity and the limitations of the CIDI 2.1 Core version for 12-month diagnosis in mental health services in Brazil, using as gold standard the clinical diagnoses based on the ICD-10 criteria.

Method

This study is part of the project to assess the "Performance of the Composite International Diagnostic Interview (CIDI 2.1) in Brazilian mental health services," whose first stage was the assessment of the instrument's reliability.¹⁰ At this stage its validation will be presented. The gold standard used was the clinical diagnoses formulated by experienced psychiatrists, using the ICD-10 criteria and based on the Spitzer's LEAD procedure: "Longitudinal" (longitudinal assessment of symptoms), "Expert" (diagnosis formulated by specialists), "All Data" (all available information).¹¹

1. Subjects

The sample was composed of 185 subjects allocated to each of the nine diagnostic sections of the CIDI to be validated (except for the tobacco section). Subjects were intentionally selected from different mental health services: psychiatric hospitals (Clínica Olivé Leite, Pelotas-RS; Hospital Espírita, Pelotas-RS; and UNIFESP, São Paulo-SP; n = 81), mental health outpatient units (Clínica Olivé Leite, UFPel, Pelotas-RS; and UNIFESP, São Paulo-SP; n = 54), community services (NGO, Clínica Olivé Leite, UFPel; n = 6) and a primary care center (UFPel; n = 40). Patients with severe intellectual impairment, incapacitating organic psychoses and without definitive diagnosis (diagnostic doubts) were excluded from the study.

The sample was selected by convenience so it would cover all the CIDI diagnoses since some of them present a low prevalence.

2. Instruments

1) CIDI 2.1

The CIDI 2.1 was translated into Portuguese by a bilingual psychiatrist, according to WHO guidelines. There was no backtranslation, although the sections were presented to experienced psychiatrists in order to evaluate translation aspects such as psychopathological phenomena, terminology and cultural adaptation according to Rubio-Stipec's recommendations.¹⁴ Remarks and suggestions of the specialists were incorporated into the final version of the questionnaire. Interviewers were submitted to the standard training of the CIDI 2.1 according to WHO guidelines.⁸ The interviewer's team was composed of 13 first year medical students, with no training in psychiatry or psychology, and 3 mental health researchers (2 psychologists and 1 social worker). The interview with the CIDI 2.1 occurred on the day that the psychiatrist was consulted about the subject's diagnosis. The interviewers, however, were blind to the clinical diagnosis and administered all the instrument's sections.

3. Procedures

The data collection had different characteristics in each type of collection site. In psychiatric hospitals and specialized outpatient units the inclusion criteria were the main diagnosis and the comorbidities provided by the subjects' psychiatrists, based upon LEAD standard. In community services, the subjects were examined by an experienced psychiatrist in order to formulate the main diagnosis and the comorbidities. In the primary care centers, the inclusion criteria were modified in order to include individuals without psychiatric diagnosis. In this setting, before being examined by an experienced psychiatrist, patients or their accompanying person answered the Self Report Questionnaire (SRQ-30),^12-13 and then were classified according to the cut-off point of 7/8.

All interviewers where blind for the medical diagnostic and the SRQ, and there was no previous contact between the interviewer and the respondent.

The study was approved by the Research Ethics Committee of the Universidade Federal de São Paulo.

4. Analysis

The concurrent validation was studied for 12-month diagnoses, and indicators of sensitivity, specificity and incorrect classification rate were calculated. Confidence interval of 95% was calculated to sensitivity and specificity by the following formula: sensibility ± 1.96and especificity ± 1.96 . Inconsistent results of CI 95%, such as less than 0 (zero) or more than 1, were approximated to 0 and 100% receptiveness. The Kappa coefficient was calculated to allow comparisons with other studies.

In all individuals presenting false negative result, the diagnoses were inspected to determinate de possible error causes.

Results

The sample was composed of 185 subjects, of which 53% (n = 98) were women with a mean age of 37 years (16-73 years; sd: 12.9), and a mean of 1.7 children per subject (0-11; sd: 1.8). Regarding the marital status, 39.5% (n = 73) had never been married, 36.8% (n = 68) were married, 19.5% (n = 38) were separated or divorced, and 3.2% (n = 6) were widowers. Mean schooling was 7 years (0-20 years; sd: 3.8), but 86.5% (n = 160) were not studying and 68.1% (n = 126) were unemployed.

The average duration of the interview was 2 hrs 30 min, varying from 50 minutes (subject with no psychiatric diagnosis) to 3 hrs 40 min (subject with Eating Disorder diagnosis). Most interviews (80%) where completed in one session.

Clinicians formulated in average 1.3 diagnoses per subject, the CIDI generated 1.2 diagnoses per subject and when the exclusion criteria of the ICD-10 was not taken into account, this mean increased to 1.5 diagnoses per subject.

The validity indicators of the specific sections are shown in Table 2. For 12-month diagnoses, the CIDI had sensitivity above 70% for alcohol and psychoactive substance abuse and dependence, depressive and phobic-anxiety disorders and below 50% for psychotic, manic episode and bipolar affective disorder, obsessive-compulsive disorder, somatoform and eating disorders.

Discussion

The interview with the CIDI at mental health services was well accepted¹⁰ and there were no refusals to participate. Psychiatrists formulated more diagnoses per patient than the CIDI, which is in disagreement with the literature.¹⁵ This was probably due to the instructions and the discussion with psychiatrists to include all diagnoses.

Our results are in agreement with those reported in the literature, i.e., sensitivity above 70% for diagnoses of alcohol dependence syndrome,¹⁶ depressive disorder,^16,18-19 phobic-anxiety disorder,^17-18 and sensitivity below 70% for diagnoses of schizophrenia and other psychotic disorders,^17,20 manic episode and bipolar affective disorder,¹⁷ somatoform disorder²¹ and eating disorder.^17,22 Nevertheless, some results are different from previous studies. Komiti et al. found lower sensitivity rates for generalized anxiety (12%) and social phobia (63%).¹⁵ Andrews et al. found higher sensitivity rates for the diagnosis of obsessive-compulsive disorder (85.5%).¹⁸

We found advantages in the CIDI, such as standardization of data collection, widening and deepening the investigation field of the physician,^15,23-24 standardization of the communication between different professionals, and utilization of lay interviewers as a way to reduce costs in population studies.

Some of the limitations of the instrument were: the rigidity of its rules; the inflexibility of the diagnostic algorithms; the need of some clinical judgment, with the consequent hampering of the administration by totally lay interviewers.

The rigidity of the rules and the inflexibility of the diagnostic algorithms caused the detection of only the number of occurred symptoms and not their severity or hierarchical importance. For instance, the subject should present six depressive symptoms for a diagnosis of depression, regardless of the symptoms. Therefore, sleeping problems have the same weight in formulating the diagnostic as suicidal thoughts or attempts.

The need of clinical judgment occurs in some questions that involve cultural aspects. This was observed particularly in the schizophrenia section, which investigates symptoms that are very subtly different from the normal. These limitations can by minimized with strong training focusing these difficulties.

Other limitations are: the interviewees' difficulty of understanding, limitation of information sources for diagnostic formulation, lack of mechanisms to verify the veracity of the information (intentional or unintentional denial of symptoms and recall of the symptom), and long duration of the interview.

The probable explanations for the low sensitivity rate of some sections are different for each diagnosis, but are related to the above-mentioned limitations. Thus, for schizophrenia and other psychotic disorders, the worse performance was due to the need of clinical judgment by the interviewer and the limited capability of the patient to provide information. For manic episode and bipolar affective disorder, the lack of mechanisms to verify the veracity of the information; for somatoform disorder, the need of clinical judgment and the difficult management of the Probe for Chart (PFC); for obsessive-compulsive disorder and eating disorder, the lack of mechanisms to verify the veracity of the information and the limited capability of the patient to provide information.

The CIDI indirectly requires some knowledge or judgment capacity to identify organic disorders and mental symptoms. The most common difficulties were: 1) the interviewer should understand the interviewees' answers and decide in which category they fit and 2) at some moments, the questionnaire has rules through which the interviewer should judge and codify a symptom mentioned by the interviewee. Questions such as: "has the interviewee felt worthless/guilty only due to depression?," require a minimum level of clinical experience to decide about the presence of the symptom. Besides, different cultural conceptions, religious ideas and sociocultural level may influence this judgment.²⁵

Other complication generated by the lack of clinical expertise is the information provided by the interviewees about their clinical diseases, which are unknown to the interviewers. In these cases, in addition to the interviewer's clinical judgment, the patient should understand the diagnosis he/she had received.²¹

The patient's phase of the disease is an important factor: subjects in an acute phase showed much difficulty to concentrate in the interview, understand the questions, and answer them accurately.^20,26-28 The CIDI diagnostic accuracy decreases in the acute phase of schizophrenia.²⁹ As to chronic phases, the fear of stigmatization makes individuals deny or minimize their experience with psychotic symptoms.^25,27,30 These difficulties associated with psychosis have frequently generated opinions against the use of structured instruments for more chronic diseases such as schizophrenia.³¹

The training of the professionals who will administer the CIDI in the clinical setting is, as described above, a limiting factor. The experience of WHO Collaborating Center in Brazil and of other training centers for CIDI shows that specialists (psychiatrists) are not good interviewers of structured questionnaires, especially when they are very experienced, as they have much difficulty with rigid rules and closed questions that do not allow a clinical judgment, feeling diminished in their knowledge and underestimated.³² On the other hand, lay interviewers showed better skills to administer the CIDI in this study, despite the lack of the clinical knowledge, which is required in some situations. These considerations lead to a paradox which may be taken into account when choosing the instrument. However, training of lay interviewers with more attention to the symptoms, and questions that need clinical reasoning and culturally determined interpretations, may minimize its limitations.

Limitation

The validity of some specific disorders could not be studied because it was not possible to identify subjects, for instance, patients with post-traumatic stress disorder. Other disorders were affected by the small size of the sample, such as, obsessive-compulsive disorder, somatoform disorder, and eating disorder. Among them, obsessive-compulsive disorder was the only disorder that had a result in disagreement with the literature, thus we recommend a new study with a larger sample.

Conclusion

The CIDI showed good indicators of concurrent validity for diagnoses of alcohol- and drug-related disorders, phobic-anxiety disorder and depressive disorder, when administered in the clinical setting by lay interviewers. Validity values were poor for diagnoses of manic episode and bipolar affective disorder, somatoform disorder, obsessive-compulsive disorder and eating disorder. The use of lay interviewers cause limitations that may be minimized by training focused on the symptoms and on the questions that need clinical reasoning and culturally determined interpretations.

References

1. Wittchen HU, Lachner G, Wunderlich U, Pfister H. Test-retest reliability of the computerized DSM-IV version of the Munich-Composite International Diagnostic Interview (M-CIDI). Soc Psychiatry Psychiatr Epidemiol. 1998;33(11):568-78.

2. World Health Organization (WHO). The ICD-10 Classification of Mental and Behavioral Disorders: Diagnostic Criteria for Research. Geneva: World Health Organization; 1993.

3. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (DSM-IV). 4th Edition. Washington, DC: American Psychiatric Association; 1994.

4. Cottler LB, Robins LN, Grant BF, Blaine J, Towle LH, Wittchen HU, Sartorius N. The CIDI-core substance abuse and dependence questions: cross-cultural and nosological issues. The WHO/ADAMHA Field Trial. Br J Psychiatry. 1991;159:653-8.

5. Tiemens BG, VonKorff M, Lin EH. Diagnosis of depression by primary care physicians versus a structured diagnostic interview. Understanding discordance. Gen Hos Psychiatry. 1999;21(2):87-96.

6. Wittchen HU, Essau CA. Comorbidity and mixed anxiety-depressive disorders: is there epidemiologic evidence? J Clin Psychiatry. 1993;54(Suppl 1):9-15.

7. Wittchen H-U, Robins LN, Cottler LB, Sartorius N, Burke JD, Regier D. Cross-cultural feasibility, reliability and sources of variance of the Composite International Diagnostic Interview (CIDI). The Multicentre WHO/ADAMHA Field Trials. Br J Psychiatry. 1991;159:645-53, 658.

8. World Health Organization (WHO). The Composite International Diagnostic Interview (CIDI). Geneva, Switzerland: World Health Organization; 1997.

9. Robins LN, Helzer JE, Croughan J, Ratcliff KS. National Institute for Mental Health Diagnostic Interview Schedule. It's history, characteristics and validity. Arch Gen Psychiatry. 1981;38(4):381-9.

10. Quintana MI, Andreoli SB, Jorge MR, Gastal FL, Miranda CT. The reliability of the Brazilian version of the Composite International Diagnostic Interview (CIDI 2.1). Braz J Med Biol Res. 2004;37(11):1739-45.

11. Spitzer RL. Psychiatric diagnosis: are clinicians still necessary? Compr Psychiatry. 1983;24(5):399-411.

12. Mari JJ, Williams P. A comparison of the validity of two psychiatric screening questionnaire (GHQ-12 and SRQ-20) in Brazil, using Relative Operating Characteristic (ROC) analysis. Psychol Med. 1985;15(3):651-9.

13. World Health Organization (WHO). A user's guide to the Self Reporting Questionnaire. Geneva, Switzerland: World Health Organization; 1994.

14. Rubio-Stipec M, Bravo M, Canino G. The Composite International Diagnostic Interview (CIDI): an epidemiologic instrument suitable for using in conjunction with different diagnostic systems in different cultures. Acta Psiquiatr Psicol Am Lat. 1991;37(3):191-204.

15. Komiti AA, Jackson HJ, Judd FK, Cockram AM, Kyrios M, Yeatman R, Murray G, Hordern C, Wainwright K, Allen N, Singh B. A comparison of the Composite International Diagnostic Interview (CIDI-Auto) with clinical assessment in diagnosing mood and anxiety disorders. Aust N Z J Psychiatry. 2001;35(2):224-30.

16. Janca A, Robins LN, Cottler LB, Early TS. Clinical observation of assessment using the Composite International Diagnostic Interview (CIDI). An analysis of the CIDI Field Trials-Wave II at the St Louis site. Br J Psychiatry. 1992;160:815-18.

17. Rosenman SJ, Korten AE, Levings CT. Computerized diagnosis in acute psychiatry: validity of CIDI-Auto against routine clinical diagnosis. J Psychiatry Res. 1997;31(5):581-92.

18. Andrews G, Peters L, Guzman AM, Bird K. A comparison of two structured diagnostic interviews: CIDI and SCAN. Aust N Z J Psychiatry. 1995;29(1):124-32.

19. Booth BM, Kirchner JE, Hamilton G, Harrell R, Smith GR. Diagnosing depression in the medically ill: validity of a lay-administered structured diagnostic interview. J Psychiatry Res. 1998;32(6):353-60.

20. Cooper L, Peters L, Andrews G. Validity of the Composite International Diagnostic Interview (CIDI) psychosis module in a psychiatric setting. J Psychiatry Res. 1998;32(6):361-8.

21. Villasenor Y, Waitzkin H. Limitations of a structured psychiatric diagnostic instrument in assessing somatization among Latino patients in primary care. Med Care. 1999;37(7):637-46.

22. Thornton C, Russell J, Hudson J. Does the Composite International Diagnostic Interview underdiagnose the eating disorders? Int J Eat Disord. 1998;23(3):341-5.

23. Peters L, Andrews G. Procedural validity of the computerized version of the Composite International Diagnostic Interview (CIDI-Auto) in the anxiety disorders. Psychol Med. 1995;25(6):1269-80.

24. Saghir MT. A comparison of some aspects of structured and unstructured psychiatric interviews. Am J Psychiatry. 1971;128(2):180-4.

25. Anthony JC, Folstein M, Romanoski AJ, Von Korff MR, Nestadt GR, Chahal R, Merchant A, Brown CH, Shapiro S, Kramer M. Comparison of the lay Diagnostic Interview Schedule and a standardized psychiatric diagnosis. Experience in eastern Baltimore. Arch Gen Psychiatry. 1985;42(7):667-75.

26. McGorry PD, Copolov DL, Singh BS. Royal Park Multidiagnostic Instrument for Psychosis: part I. Rational and review. Schizophr Bull. 1990;16(3):501-15.

27. Pulver AE, Carpenter WT, Adler L, McGrath J. Accuracy of the diagnoses of affective disorders and schizophrenia in public hospitals. Am J Psychiatry. 1988;145(2):218-20.

28. Farmer AE, Katz R, Mcguffin P, Bebbington P. A comparison between the Present State Exam and the Composite International Diagnostic Interview. Arch Gen Psychiatry. 1987;44(12):1064-8.

29. Erdman HP, Klein MH, Greist JH, Bass SM, Bires JK, Machtinger PE. A comparison of the Diagnostic Interview Schedule and clinical diagnosis. Am J Psychiatry. 1985;144(11):1477-80.

30. Semler G, Von Cranach M, Wittchen HU. Comparison Between the Composite International Diagnostic Interview and the Present State Examination: Report to the WHO/ADAMHA Task Force on Instrument Development. Geneva, Switzerland: World Health Organization; 1987.

31. Spengler PA, Wittchen HU. Procedural validity of standardized symptom questions for the assessment of psychotic symptoms: a comparison of the DIS with two clinical methods. Compr Psychiatry. 1988;29(3):309-22.

32. Rubinson EP, Asnis GM. Use of structure interviews for diagnosis. In: Wetzler S, editor. Measuring Mental Illness: psychometric assessment for clinicians. Washington, DC: Washington Psychiatric Press, Inc; 1999. p. 45-66.

Financing: FIDEPS, Clínica Olivé Leite and Fundação de Apoio à Pesquisa do Estado de São Paulo (FAPESP, 98/10253-1).

Conflict of interests: None

Submitted: September 25, 2005

Accepted: September 18, 2006

Universidade Federal de São Paulo (UNIFESP), São Paulo (SP), Brazil.

Publication Dates

• Publication in this collection
  27 Nov 2006
• Date of issue
  Mar 2007

History

• Accepted
  18 Sept 2006
• Received
  25 Sept 2005