Predictive validity of the Brazilian version of the Expected Treatment Outcome Scale in cocaine-dependent outpatients at a drug treatment referral center

Validade preditiva da versão em português da Escala do Desfecho Esperado do Tratamento em pacientes ambulatoriais dependentes de cocaína em um centro especializado

BRIEF REPORT

Predictive validity of the Brazilian version of the Expected Treatment Outcome Scale in cocaine-dependent outpatients at a drug treatment referral center

Validade preditiva da versão em português da Escala do Desfecho Esperado do Tratamento em pacientes ambulatoriais dependentes de cocaína em um centro especializado

Marcelle Maria Lobo Dinis^I; Sonia Regina Lambert Passos^II; Luiz Antonio Bastos Camacho^III

^IPsychiatry Service, Hospital Universitário Pedro Ernesto, Universidade Estadual do Rio de Janeiro (UERJ), Rio de Janeiro (RJ), Brazil

^IIDepartment of Epidemiology, Instituto de Pesquisa Clínica Evandro Chagas, Fundação Oswaldo Cruz (Fiocruz), Rio de Janeiro (RJ), Brazil

^IIIDepartment of Epidemiology and Quantitative Health Methods, Escola Nacional de Saúde Pública (ENSP), Fundação Oswaldo Cruz (Fiocruz), Rio de Janeiro (RJ), Brazil

^{Correspondence}

ABSTRACT

BACKGROUND: High dropout rates among patients under treatment for cocaine dependence have stimulated research into predictors of treatment outcome.

OBJECTIVE: To assess the predictive value of the Brazilian version of the Expected Treatment Outcome Scale.

METHODS: The original English version of the scale was translated and back-translated. A total of 210 subjects participating in a 10-week randomized double-blind clinical trial (nefazodone versus placebo) completed the questionnaire at their first appointment. Mean Expected Treatment Outcome Scale scores were compared with treatment outcomes.

RESULTS: There were ten subjects (5%) who failed to complete at least six items, and 37 (17.5%) failed to complete 1 to 3 items. The most frequently unanswered questions involved time estimates (treatment time and abstinence) and third-party judgments. The mean score was 34.4 (9.3) (median, 33.9). There were no differences in mean scores between subjects evaluated in the first to the fifth appointment 35.2 (9.3) or in the sixth to the eleventh appointment 35.2 (9.3) (p = 0.13); completing the treatment 33.8 (10.3) or not 34.6 (9.1) (p = 0.64); remaining abstinent for three weeks 34 (9.3) or not 34.8 (9.4) (p = 0.58), and medication compliance 33.9 (8.8) or noncompliance 35.3 (10.3) (p = 0.34). The ROC curve of Expected Treatment Outcome Scale scores, when dropout was defined as not appearing for all 11 appointments, was linear, with an area under the curve of .54 (range, .44-.64), suggesting that the scale is ineffective in discriminating between cases and noncases.

CONCLUSION: In this study, the Brazilian version of the Expected Treatment Outcome Scale was found to have no predictive value for treatment adherence and abstinence in cocaine-dependent subjects subjected to a standardized treatment protocol.

Keywords: Cocaine-related disorders/psychology; Cocaine-related disorders/rehabilitation; ROC curve; Patient dropouts; Substance abuse treatment centers; Predictive value of tests; Questionnaires; Randomized controlled trial [Publication type]

RESUMO

INTRODUÇÃO: Altas taxas de abandono de tratamento por dependentes de drogas têm intensificado a pesquisa sobre fatores preditivos.

OBJETIVO: Estudar a validade preditiva da Escala do Desfecho Esperado (EDET).

MÉTODOS: Tradução e back-translation. Auto-aplicação da Escala do Desfecho Esperado na primeira consulta de 210 dependentes de cocaína alocados em grupos iguais (nefazodone ou placebo), aleatoriamente, em um ensaio clínico duplo-cego ambulatorial, com 10 semanas de duração. São descritos os escores médios da Escala do Desfecho Esperado segundo os desfechos do ensaio.

RESULTADOS: Dos 210 questionários, 10 (5%) tinham mais do que 6 e 37 (17,5%) tinham 1 a 3 questões não preenchidas. As questões mais freqüentemente não compreendidas envolviam cálculos (tempo de tratamento e de abstinência) ou julgamento de terceiros. O escore médio foi 34,4 (s.d 9,3) e a mediana 33,9. Não há diferença entre os escores médios para 1 a 5 consultas 35,2 (9,3)e 6 a 11 consultas 35,2 (9,3), p = 0,13; completar o tratamento 33,8 (10,3) ou não 34,6 (9,1), p = 0,64; permanecer três semanas abstinente 34 (9,3) ou não 34,8 (9,4), p = 0,58; e aderir à prescrição 33,9 (8,8) ou não 35,3 (10,3), p = 0,34. A curva ROC dos escores da Escala do Desfecho Esperado, assumindo o não comparecimento a todas as 11 consultas como caso de abandono, é linear com uma área sob a curva de 0,54 (0,44-0,64), revelando uma má performance da escala como preditora de abandono.

CONCLUSÃO: Neste estudo, a Escala do Desfecho Esperado não evidenciou validade preditiva para adesão ao tratamento e abstinência em dependentes de cocaína submetidos a um protocolo de tratamento padronizado.

Descritores: Transtornos relacionados ao uso de cocaína/psicologia; Transtornos relacionados ao uso de cocaína/reabilitação; Curva ROC; Desistência do paciente; Centros de tratamento de abuso de substâncias; Valor preditivo; Questionários; Ensaio controlado aleatório [tipo de publicação]

Introduction

Noncompliance with treatment among drug addicts has been correlated with increased morbidity and mortality¹ and has given rise to studies that have lead to the development of scales designed to determine prognostic factors of noncompliance,² internal motivation^2-3 and external^4-5 motivation.

The controversy surrounding predictive variables of treatment noncompliance have been attributed to the heterogeneity of case definitions concerning the type of drug and administration route, the failure to characterize noncompliance in terms of number of appointments, and the different statistical methods used for data analysis.⁶

Among the scales proposed to assess expected treatment outcome, we include the Maudsley Addiction Profile,⁴ the Cocaine Selective Severity Assessment,² the University of Rhode Island Change Assessment scale,^3,7 the Addiction Severity Index (ASI)⁵ and the Escala de Desfecho Esperado do Tratamento (EDET, Expected Treatment Outcome Scale).⁸

The creation of such scales presupposes that they will be validated for use in different cultural contexts. The predictive validity of the EDET for use in Brazil has not been assessed.⁸ The original, English language, version has face value and is highly reliable. The choice of the EDET is justified because: 1) it is in the public domain, published in full, with a scoring system and lends itself to psychometric assessment; 2) there is no other instrument that presents predictive value for treatment adherence among cocaine-dependent subjects; 3) other scales, such as the ASI, that characterize patients are long and involve subjective assessment; and 4) it is possible to apply the EDET in the context of a clinical trial using standardized procedures and randomizing the clientele to program resources.⁹

Methods

This was a longitudinal study of the predictive diagnostic validity of the EDET, the Brazilian version (in Portuguese) of the Expected Treatment Outcome Scale, which is a multidimensional, self-administered instrument with 48 closed questions regarding: age; pregnancy; chronic disease; motivation for treatment; consequences of drug use; taboos; social and physical environment; total number of arrests of three close friends; total number of arrests in the previous year; lifestyle; family support; employment; support from the partner; number of dependents; number of weeks they have worked regularly (even as a day worker) in the previous year; variety of drugs used; frequency of drug use; duration of addiction; history of treatment for psychoactive substance dependence; time (in months) of abstinence and sobriety since the first treatment; intensity of feelings of rage; previous history of psychiatric treatment; depression; history of physical or sexual abuse; and current physically or sexually abusive relationships.

The specific objectives were to compare the average EDET scores as to: 1) completion of outpatient treatment (11 visits); 2) adherence to the treatment plan based of self-reporting; 3) abstinence (self-reported) for three consecutive weeks. An additional objective was to determine the cut-off point of highest sensitivity, specificity and predictive values, using adherence to treatment as the standard.

The EDET scoring procedure was based on the multiattribute value model, reflecting the opinions of specialists.⁸ The score ranges from 0 to 100 points, higher scores being associated with worse outcomes.

The scale was translated into Portuguese by a translator specializing in the medical field and back-translated into English by a psychiatrist. The back translation was then compared to the original version. The structure and order of the questions were not intentionally altered. This validation study was conducted during a double-blind, placebo-controlled clinical trial involving 210 cocaine dependent subjects in weekly outpatient treatment for 11 weeks.⁹ All patients included in the clinical trial completed the EDET themselves.

A sample calculation for detecting 90% sensitivity and 90% specificity, allowing a 95% confidence interval (alpha error of 0.05) and an estimated accuracy of approximately 5%, indicated that a minimum of 138 individuals would be necessary. Inclusion criteria were being from 18 to 65 years of age, and meeting the criteria for cocaine dependence according to the CID-10, with or without dependence on other drugs. We used the following exclusion criteria: 1) diagnosed with schizophrenia, schizophreniform disorder, schizoaffective disorder, brief reactive psychosis, mental retardation or organic mental disorder; 2) being a parolee; 3) presenting health conditions that might contraindicate the use of nefazodone hydrochloride (being a woman of childbearing age not using any contraceptive methods, being pregnant, having kidney and liver diseases, having a history of hypersensitivity to other phenylpiperazine antidepressants); 4) using terfenadine or astemizole; 5) presenting suicidal ideation; 6) being epileptic; 7) using monoamine oxidase inhibitors or other psychotropics in the 15 days prior to the initial interview; 8) using crack or injected cocaine.^10-11

The study was approved by the Ethics in Research Committee of the Universidade do Estado do Rio de Janeiro Hospital Universitário Pedro Ernesto. The EDET scores, after editing, were entered into SPSS Win 11.0. The score means were compared using the Student's t-test. The validation parameters cut-off point for best sensitivity and cut-off point for best specificity were calculated and confirmed using Receiver Operating Characteristic (ROC) curve graphs.

Results

Of the 210 questionnaires, 10 (5%) presented more than 6 unanswered questions and were therefore excluded from the analysis, and 37 (17.5%) presented 1 to 3 unanswered questions. The 20 patients who did not agree to participate in the study did not differ in gender or age from those who were included. The sample profile was male, single, white, age 30 and employed with an income of less than four times the minimum wage. A little less than half of the individuals had previously been treated for drug dependence. The mean duration of treatment prior to the study outset was 13.2 weeks (Nefazodone) and 7.2 weeks (placebo). The majority - 67.6% of those in the nefazodone group and 62.9% of those in the placebo group - had less than nine years of schooling. In addition, the majority - 70.5% of those receiving nefazodone and 86.6% of those receiving the placebo - were employed. The group receiving nefazodone had twice the number of unemployed individuals.

The most frequently unanswered items involved calculations (time of treatment and time of abstinence) or judgment by a third party. The scale scores in this sample assumed an approximately normal (Gaussian) distribution.

The mean EDET score was 34.4 ± 9.3 (median, 33.9). The mean score did not differ between the two treatment groups (nefazodone: 34.8 ± 8.9; placebo: 33.9 ± 9.8, p = 0.53) Table 1 shows that the differences in mean score between the subgroups based on the number of visits, completion of the treatment and adherence to the treatment regimen, are small and have no statistical significance. The ROC curve (figure available from the authors) generated by the EDET scores, when dropout was defined as not attending at all 11 appointments, was linear, with an area under the curve of 0.54 (range, 0.44-0.64), suggesting that the scale is inappropriate for predicting outcomes or for distinguishing between cases and noncases.

Discussion

The validation of scales is fundamental to enable inferences to be made from the scores. The EDET was translated without changes in content, items or adaptation to the local culture. The questions most frequently misunderstood, probably due to a lack of semantic clarity or to sociocultural factors, were those involving estimates (how often the individual was approached by the police for drug possession, how often three close friends were arrested, how long the individual worked during the previous year, time during which the individual remained under outpatient treatment or hospitalization, time of sobriety and of drug or alcohol use); those related to judgments made by third parties (the judge or the social worker said you would lose custody of your child, friends said you should change your life, etc.); those dealing with difficulties in distinguishing preexisting clinical conditions from those related to the use of drugs; and those pertaining to employment, especially for individuals who were unregistered workers. These questions can partially explain why it was not possible to use the EDET to predict treatment outcomes in cocaine-dependent outpatients. Since the accuracy of the original instrument in English was not assessed, it was not possible to compare the results of this study with those of other studies. Neither can we know to what extent the limitations of the scale were related to its translation into Portuguese.

The EDET emphasizes external motivation, as well as social, criminal and emotional components related to the consumption of psychoactive substances. It includes a question regarding the degree of patient willingness to seek assistance inspired by being "tired" of the problems related to the use of drugs. Cahill et al found that the scores for internal motivation, seeking assistance and confidence in the treatment did not drop after the initiation of treatment, whereas those for external motivation, which indicate the search for treatment, decreased during the treatment.¹² The poor performance of the EDET, expressed by the small area under the ROC curve, could be partially explained by the fact that components related to internal motivation were underrepresented.

These results cannot be extrapolated to patients who primarily use drugs other than cocaine. Furthermore, it cannot be said that fewer appointments are predictive of a worse prognosis. Reiber et al reported that the completion of the treatment has no predictive value regarding success in achieving partial or complete abstinence ^13.

Financing: Faperj (grant number: E-26/170.697/2000), Fiocruz and Bristol Myers-Squibb

Conflicts of interests: None

Submitted: 4 June 2004

Accepted: 27 October 2004

Study presented at the XXI Brazilian Congress of Psychiatry and VII Brazilian Congress of Collective Health.

• 1. McMahon RC, Kouzekanani K, Malow RM. A comparative study of cocaine-treatment completers and dropouts. J Subst Abuse Treat. 1999;16(1):17-22.
• 2. Kampman KM, Volpicelli JR, McGinnis DE, Alterman AI, Weinrieb RM, D'Angelo L, et al. Reliability and validity of the Cocaine Selective Severity Assessment. Addict Behav. 1998;23(4):449-61.
• 3. McConnaughy EN, Prochaska JO, Velicer WF. Stages of change in psychotherapy: Measurement and sample profiles. Psychother Theo Res Prac. 1983;20:368-75.
• 4. Marsden J, Gossop M, Stewart D, Best D, Farrell M, Lehmann P, et al. The Maudsley Addiction Profile (MAP): a brief instrument for assessing treatment outcome. Addiction. 1998;93(12):1857-67.
• 5. Castel S, Formigoni MLOS. Escalas para avaliação de tratamentos para dependência de álcool e outras drogas. In: Goreinstein C, Andrade LHG, Zuardi AW, orgs. Escalas de avaliação clínica em Psiquiatria e Psicofarmacologia. São Paulo: Lemos; 2000. p. 267-86.
• 6. Passos SR, Camacho LA. Factors associated with immediate dropout of outpatient treatment for drug abuse in Rio de Janeiro. Soc Psychiatry Psychiatr Epidemiol. 2000;35(11):513-7.
• 7. Oliveira Júnior HP, Malbergier A. Avaliação da motivação para tratamento em pacientes dependentes de álcool que procuram um serviço especializado. Rev Bras Psiquiatr. 2003;25(1):5-10.
• 8. Alemi F, Stephens RC, Llorens S, Orris B. A review of factors affecting treatment outcomes: Expected Treatment Outcome Scale. Am J Drug Alcohol Abuse. 1995;21(4):483-509.
• 9. Passos SRL, Camacho LAB, Lopes CS, dos Santos MAB. Nefazodone in out-patient treatment of inhaled cocaine dependence: a randomized double-blind placebo-controlled trial. Addiction. 2005;100(4):489-94.
• 10. Schatzberg AF, Cole JO. Antidepressivos. In: Schatzberg AF, Cole JO. Manual de psicofarmacologia clínica. Porto Alegre: Artes Médicas; 1989. p. 35-60.
• 11. Talbott J, Hales R, Yudofsky S. Tratado de Psiquiatria. Porto Alegre: Artes Médicas; 1996.
• 12. Cahill MA, Adinoff B, Hosig H, Muller K, Pulliam C. Motivation for treatment preceding and following a substance abuse program. Addict Behav. 2003;28(1):67-79.
• 13. Reiber C, Ramirez A, Parent D, Rawson RA. Predicting treatment success at multiple timepoints in diverse patient populations of cocaine-dependent individuals. Drug Alcohol Depend. 2002;68(1):35-48.

Publication Dates

• Publication in this collection
  21 Nov 2005
• Date of issue
  Sept 2005

History

• Accepted
  27 Oct 2004
• Received
  04 Mar 2004