The effectiveness of substance abuse treatment: development of a brief questionnaire

Abstract

Background

Practitioners need brief instruments to monitor outcomes in both treatment of drugs and alcohol addiction because they are useful to guide decision making in a short time.

Objectives

This study aims to develop a brief questionnaire, based on Client Evaluation of Self and Treatment, to evaluate the treatment effectiveness in drug and alcohol addiction treatment settings.

Methods

A cross-sectional study using a convenience sample (N = 608) recruited from Division for Intervention on Addictive Behaviours and Dependencies (DICAD – ARS North).

Results

The results show a new four-factor solution that accounted for 54.4% of the total variance and that provides the best fit to the data (c²/df = 1.72, CFI = .94, GFI = .91, RMSEA = .048 [.040-.057]; prmsea = .623). It also revealed a high internal consistency (a = .82). It was found a significant negative correlation (r = - .52, p < .01) between the final version of the instrument and a self report measure of psychopathology symptoms.

Discussion

This brief questionnaire, with good psychometric properties, can be useful to provide a viable and rapid feedback of treatment outcomes. Further studies should be performed to continue the evaluation of the reliability of this measure.

Addiction; treatment effectiveness; psychopathology; psychometric proprieties

Introduction

There have been significant changes in addiction treatment over the past three decades, where has been a significant movement from inpatient to outpatient programs. However, while treatment programs have changed over time, monitoring the effectiveness during the implementation of these programs has been little explored both by researchers and practitioners¹. There are two underlying main approaches to conceptualize the treatment effectiveness. One is the rehabilitation-oriented model is the traditional paradigm where the method used to evaluate the effectiveness of addiction treatment is based on post-treatment follow-up outcomes, assuming that benefits were attained during treatment, and positive changes have occurred by the end of the treatment, and it is expected sustained abstinence at the time of follow-up².

This method has been used in the case of residential treatment, where drugs are not available in the treatment setting, and the effectiveness outcome is evaluated for a substantial follow-up period of time. However, the limitation of this method is the excessive attention to the sustained and complete recovery from a substance use as criterion of treatment effectiveness. According to White³ the problem is that “groups like Narcotics Anonymous (NA) have defined recovery in terms of abstinence from drug use, but addiction scientists have generally defined recovery from illicit drug dependence in terms of problem resolution rather than absence of drug use”. Thus, this conceptualization “would allow measuring levels of outcomes over time and answer questions about the viability of particular problem-resolution strategies for particular populations”³. In addition, the pos-treatment follow-up method has other several limitations. The evaluations are usually conducted by an external researcher from the treatment team², who do not measured patient changes during treatment. When participants are not located or refuse to participate at follow-up evaluations, the internal and external validity of the collected data could be compromised due to the characteristics of those participants who drop out (e.g., involved in antisocial and criminal behaviors)⁴. Finally, many complex ethical issues arise from the follow-up evaluations, in part, due to the assertive methods required to generate high follow-up rates⁵.

Other approach that has been suggested by researchers is the alternative method of evaluating the treatment effectiveness based on the monitoring method instead of follow-up results. This new conceptualization was emerged from two changes, namely the change of the health-care delivery system that moved from residential care to outpatient setting, where nowadays practitioner’s need for more economical, rapid and clinically relevant information to guide decision making, and the transition of the recovery definition from medicalized term to a problem-solving process, and a psychosocial perspective²,³. The proposal called “concurrent recovery monitoring” (CRM)²could be described as a brief and repeated evaluations, relatively easy to collect, concurrent with treatment to monitor and assist patient change, toward clinical and social outcomes, and more relevant and in time information to guide the decision making in a more effective way². For instance, clinical decisions about which specific treatments are more suitable according to the different patterns of drug or alcohol use and the different characteristics of drug users⁶. Monitoring can also optimize the results of treatment⁷and monitor the need and the readiness for change in programs and organizational factors⁸and contribute to retention in treatment⁹. Further, the monitoring method can overcome the ethical problems of complex and large assessments, including the burden of time and efforts to generate high follow-up rates⁵. However, the practitioners need economical, rapid and clinical reliable methods to guide decision making in a short time period without compromising the patient attendance and participation in treatment. Complex or lengthy assessment measures may also lead to staff noncompliance¹⁰. On the other hand, the monitoring system can be burdensome for clinical staff in cases of large time-consuming intake, weekly and follow-up assessments, and can compromise the quality of data through a monitoring system¹¹. The solution appears to be the use of self-report questionnaires that took a short-time to be completed, not requiring the attendance of the staff member in training program, as in the case of interviews, and are well appropriate to systems of care¹⁰.

Machado et al.¹conducted a study with individuals undergoing dependence treatment, where they created a software that allows building a database for monitoring the intervention efficacy in clinical practice. For that, the authors used the following instruments: (1) Drug Abuse Screening Test – Portuguese version (DAST) – a dichotomous 20-item scale to assess the severity level of consumption. It evaluates the consequences related to consumption during the last 12 months such as physical and psychological symptoms, social and relational aspects, among others; (2) Outcome Questionnaire – Portuguese version (Q-45), a 45-item questionnaire that provides a reliable assessment of various aspects of the adjustment level and psychosocial disturbance of individuals; (3) The Inventory of Drug-Taking Situations – Portuguese version (IDTS) – an instrument consisting of 50 items that describes potential drug use situations over the past year; it allows to recognize problematic situations and anticipate risk situations; (4) Consumer Satisfaction Questionnaire – Portuguese version (CSQ-8), an 8-item instrument to explore the degree of patient satisfaction regarding to treatments and services received and their impact on the patient’s life. This study brings a clear progress in terms of research in this area in Portugal.

However, the administration of this wide array of assessment tools leads to a final database of about 123 items, making it difficult to apply on a regular basis in the clinical setting. On the other hand, if we choose not to apply the same instruments in all stages of evaluation/monitoring we are not able to compare results of the various moments of administration, compromising the effective therapy evolution of individual consumers during their treatment.

There are other instruments available to monitor the effectiveness of treatment programs. Marsden et al.¹² validated, for the European context, the Treatment Perceptions Questionnaire (TPQ). They also validated the Maudsley Addiction Profile (MAP), which adds some of the most relevant performance indicators in the evaluation of treatment outcomes. Notwithstanding, other important indicators of effectiveness (including the therapeutic relationship, the involvement or active participation in treatment, the social support network⁷,¹³-¹⁵, psychopathological symptoms, negative affects, or treatment orientation¹⁶-¹⁹ are missing in this questionnaire.

Recently, studies conducted by Joe et al.⁷ and Simpson¹⁴ validated the Texas Christian University (TCU) Client Evaluation of Self and Treatment (CEST) a multidimensional instrument covering the main indicators of effectiveness of drug and dependence treatments that can and should be administered in repeated evaluations during the treatment process. The CEST was developed under the DATAR project, founded by the National Institute on Drug Abuse in 1993 (Grant No.DA13093). The CEST has originally 130 items divided into 17 dimensions, taking 30 to 40 minutes to complete, which is its biggest limitation.

So, we aimed to develop a short self-report measure, suitable for both inpatient and outpatient programs, encompassing evaluating of critical indicators of treatment effectiveness, using a problem-solving and a psychosocial models. For that, we selected the previous Portuguese version²⁰ of Texas Christian University – Client Evaluation of Self and Treatment – TCU – CEST. We also aimed to examine the validity of the new instrument comparing it with a psychopathology self-report measure, similarly to previous studies of the TCU-CEST⁷,¹⁴.

Methods

Participants

Six hundred and eight subjects were recruited from Intervention Service on Addictive Behaviours and Substance Dependence (SICAD) – Regional Northern Section, in Portugal. The convenience sample was recruited from 3 main treatment settings: outpatient’s drug addiction treatment (64%; n = 389); outpatient´s alcohol dependence treatment (14%; n = 85), and inpatient’s drug and alcohol dependence treatment (22%; n = 134).

We selected patients who were in treatment for, at least, one month. Furthermore, the researchers took into account changes resulting from psychopathology or recent consumption, which could interfere with the normal completion of the questionnaires

Procedures

After we obtained permission from the authors to use The TCU-CEST, SICAD and DICAD, then the study was submitted to the ethics committee of the Faculty of Psychology and Education Sciences (University of Porto), which approved the study.

Researchers certified on addiction problems administered the instruments at the facilities of DICAD – North Regional Health Administration after the participants signed the free consent inform about the research. The confidentiality was ensured to all participants.

Materials

Socio-Demographic Questionnaire. This questionnaire included information about birth date, gender, age, birthplace, treatment program, treatment time and finally, the identification of primary substance of abuse.

Brief Symptoms Inventory (BSI)²¹,²². This is a reduced version of the Symptom Check List 90 Revised (SCL-90-R), a self-report inventory consisting of 53 items. The participant must specify the degree to which each problem has affected them over the past week, on a Likert-type scale (1 – Never, 2 – Rarely, 3 – Sometimes, 4 – Often, 5 – Very often). This inventory assesses psychopathological symptoms along nine basic dimensions (somatization, obsession-compulsion, interpersonal sensitivity, depression, anxiety, hostility, phobic anxiety, paranoid ideation and psychoticism) and three global indices (positive symptoms distress index – PSDI, global severity index – GSI, and positive symptom total – PST). In the present study, the various dimensions have an internal consistency ranging from moderate to high, with Cronbach’s alpha values between .70 to .85. The internal consistency of the overall BSI in this sample was .96.

Texas Christian University (TCU) – Client Evaluation of Self and Treatment (CEST)⁷,¹⁴. This instrument was developed as part of National Institute on Drug Abuse (NIDA). The original CEST is a 129-item self-rating instrument that includes 17 scales measuring patient functioning and treatment perceptions. Psychometric properties (including reliability and construct validity) of the scales are examined in Joe et al.⁷and acceptable reliabilities (.70 or above) were generally reported, and construct validity was also demonstrated. The response format is a 7-point Likert scale (1 – I strongly disagree to 7 – I strongly agree). We used the Portuguese version²⁰ that included 101 items that match the original 4 dimensions and 13 subscales, with reasonable to good internal consistency values (ranging from .54 to 89), namely, (a) Treatment motivation dimension; (b) Psychological functioning dimension; (c) Social functioning dimension; and (d) Treatment engagement dimension

Statistic analysis

We used the Statistical Package for Social Sciences – SPSS (SPSS Inc., Chicago, IL, version 19.0) to perform the analyses. Concerning the psychometric sensitivity of the items, we found a high kurtosis (|ku| > 7) for item 62 and a high skewness (|sk| > 2) for items 8, 15, 26, 54, 57, 62, 79, 104 and 127²³. Without these items, we sought to establish the factor validity of the questionnaire through the Exploratory Factor Analysis (EFA), using the sample of 296 patients in treatment. In conducting the EFA, we used for factor extraction the principal axis factoring method and oblimin rotation with Kaiser normalization. Following the recommendations of Dziuban and Shirkey²⁴, before the analysis, we explored the psychometric adequacy of the items. Next, we performed a confirmatory factor analysis through AMOS software. We also screened for major violations of normality, taking into account the absolute values of kurtosis (ku < 7) and skewness (sk < 2)²³. The hypothesized model obtained through the EFA was evaluated using indices and respective values for an acceptable fit²⁵: c²/df < 5²⁶Comparative Fit Index – CFI > 0.90; Goodness Fit Index - GFI > 0.90²⁷ Standardized Root Mean Square Residual – SRMR < 0.10²⁸, and Root Mean Square Error of Approximation – RMSEA < 0.08²⁶. A 90% confidence interval (Low90 and Hi90) was used to assess the precision of the RMSEA estimate²⁹. We also considered p Close Fit – prmsea > 0.50²⁹.

Results

Regarding the sample, participants from outpatient´s drug addiction treatment included mainly men (89.4%), with ages ranging from 18 to 61 (M = 39.7; SD = 7.8). The psychoactive substance reported as the primary drug of abuse prior to treatment was heroin for 58% of the sample, followed by poly-consumption (19.8%), cocaine (19.3%), hashish (2.1%) and replacement psychotherapeutic drugs – methadone/Subutex – for 0.3% of this sample. The mean duration of treatment at the time of questionnaire administration was 40.61 months (SD = 40.9). The sample collected in outpatient’s alcohol dependence treatment included mostly men (82.4%), with ages ranging from 28 to 73, (M = 47.3; SD = 9.4). The psychoactive substance reported as the primary drug of abuse prior to treatment was alcohol for 100% of the sample. The mean duration of treatment at the time of questionnaire administration was 32.1 months (SD = 39.9). Finally, the inpatient’s drug and alcohol dependence treatment sample included mainly men (90.1%), with ages ranging from 18 to 60 (M = 39.7; SD = 8.5). The psychoactive substance reported as the primary drug of abuse prior to treatment was alcohol for 56.4% of the sample, followed by heroine (18.8%), cocaine (9.8%), psychotherapeutic drugs – methadone/Subutex (9%), poly-consumption (4.6%), and other unspecified psychotherapeutic drugs (1.5%). The mean duration of treatment at the time of questionnaire administration was 2.6 months (SD = 8.9). Further, the total sample of 608 participants was randomly divided into two subsamples (N = 296 and N = 312) to properly perform the exploratory and confirmatory factor analyses.

Exploratory factor analysis

Bartlett’s Test of Sphericity was significant at p = .000, indicating a good correlation between the variables and the adequacy of the EFA to this scale. The Kaiser-Meyer-Olkin index was higher than the recommended value of .50 (KMO = .822). This last value showed us that the results obtained can be considered good according to Marôco³⁰ and Pestana and Gageiro³¹. Communality coefficients (h² > .40) were also taken into account to assess the psychometric qualities of the instruments³². Nevertheless, we chose to keep the items with communalities above .30 if they are theoretically justified. According to Cattell’s scree plot graphic³¹, there are four to six main factors. We ran EFA, establishing 4 factors because the theoretical model of the original version also showed 4 overall factors⁷. The four-factor solution accounted for 29.9% of the total variance. However, some items had low communality values (h² < .40). In this process, we eliminated the items with poor results (h² < .40 if they are not theoretically justified) and ran the EFA repeatedly until we found a final model in which all items showed adequate results (Table 1). In this four-factor solution all items presented communality values above .35 and factor loadings ranging between .49 and .90 (Table 1). The final EFA revealed a new four-factor solution that accounted for 54.4% of the total variance. We renamed some of the subscales since there is a new combination of items: Therapeutic involvement (26.6%); Negative affects (12.4%); Social support (9.2%); and Peer support (6.2%).

Table 1
Final Exploratory Factor Analysis: communalities and factor loading by item (N = 296)

		Component
	h2	1	2	3	4
84. [Your counselor respects your opinions]	.562	.783	.021	-.036	-.145
63. [Your counselor helps you develop confidence in yourself]	.654	.780	.029	.067	.015
30. [You are satisfied with this program]	.594	.761	-.020	-.019	.057
80. [The staff here is good at doing its job]	.576	.756	-.071	-.113	.102
43. [Your counselor is sensitive to your problems]	.537	.745	-.048	-.142	.086
21. [You are motivated by your counselor]	.547	.739	.050	.046	-.085
2. [You trust your counselor]	.526	.734	.040	.028	-.105
38. [Your counselor recognizes the progress you make in treatment]	.524	.691	-.024	.057	.031
20. [This program is organized]	.480	.644	.039	.032	.124
115. [You have a good personal counseling at this program]	.423	.625	-.090	.018	.017
67. [You have made progress in understanding your behavior]	.371	.493	.059	.219	.015
105. [You feel nervous]	.623	.046	.777	-.010	-.105
36. [You feel a lot of anger inside you]	.554	-.037	.732	-.049	.144
70. [You feel anxious]	.484	-.053	.694	.027	.056
92. [You get mad easily]	.470	-.026	.676	.104	-.092
41. [You have a hot temper]	.503	.046	.657	.270	-.182
90. [You worry or brood a lot]	.425	.052	.644	-.034	.143
74. [You feel sad]	.490	-.072	.589	-.289	.011
76. [You feel extra tired]	.343	-.028	.501	-.244	.001
95. [You have people close to you who help you develop confidence in yourself]	.722	.050	.036	.831	.009
64. [You have people close to you who understand your situation and problems]	.636	-.040	-.074	.794	.031
107. [You have people close to you who respect your efforts in this program]	.725	.120	.013	.786	.059
18. [You have people close to you who help you stay away from drugs/alcohol]	.479	.004	.014	.694	-.008
29. [You have people close to you who can always be trusted]	.470	-.061	-.012	.686	.081
77. [You have improved your personal relationships with other people]	.425	.070	-.043	.604	.053
58. [Other clients at this program care about your situation and problems]	.801	-.015	.032	.022	.895
72. [Other clients at this program are helpful to you]	.777	-.017	-.005	.072	.868
99. [You have developed positive trusting friendships while at this program]	.518	.215	.024	.212	.546

• Note: Boldface indicates the items belonging to the factor.

Confirmatory factor analysis

All items were distributed within the normality range (|ku| < 4.601 and |sk| < 1.899). Sixteen outliers were found using Mahalanobis squared distance (p1 and p2 < 0.00)²⁵. Because the analysis of the model fit without outliers showed similar results, we decided not exclude these cases from the analysis³⁰. The model obtained by the EFA with 4 factors and 28 items was tested (Figure 1). The hypothesized model revealed a poor global fit (Table 2). However, we have analyzed modification indices and we proceeded to the change of paths only when if it made sense according to the theoretical premises: the regression values show us that items 20, 21 29, 43, 77 and 99 are repeatedly associated with items outside the factor they belong to, thus revealing their multidimensionality (see contents of the items in attachment file). Consequently, we decided to remove these items from the model. The modification indices also suggested covariance between the errors of items 74 ↔ 76, 41 ↔ 92 (see contents of the items in attachment file). Following these changes, we obtained a short version with 22 items. For this new version, results allow us to classify the model (Figure 2) as a whole as acceptable/good (c²/df = 1.724, CFI = .941, GFI = .907, SRMR = .155, RMSEA = .048 [.040 - .057]; prmsea = .623)³³.

Figure 1
Model 1 – 4 correlated factors and 28 items.

Table 2
Confirmatory Factor Analysis: global adjustment indices (N = 312)

	χ2	df	χ2/df	CFI	GFI	SRMR	RMSEA		p rmsea
Model 1: 4 correlated factors (with 28 items)	775*	344	2.253	.877	.846	.178	.063	[.058; .069]	.000
Model 2: 4 correlated factors (with 22 items)	346*	201	1.724	.941	.907	.155	.048	[.040; .057]	.623

• * p < .001.

Figure 2
Model 2 – 4 correlated factors and 22 items.

Reliability (Cronbach’s alpha)

According to the recommended default value of > .70³¹, the analyses of the values obtained showed us that the Therapeutic involvement, Negative affects, Social support and Peer support subcales of new version (22 items) have, respectively, Cronbach’s alpha values of .87, .82, .85 e .83. The global instrument has also a high internal consistency (a = .82).

Convergent validity

Correlation coefficients are reported in tables 3 e 4. The results show that there is a positive and significant association between all subscales and the total score of the new instrument (ranging from r = .45 to r = .68, p < .01), except for the subscale Negative affects, whose association is significant, but negative, as expected by theory (r = - .74 , p < .01 ) (Table 3). We also found a significant negative association between the total score of the new instrument and the General Symptom Index of BSI (BSI-GSI) (r = - .52, p < .01), as well as between the total score of the new instrument and all BSI subscales (p < .01) (Table 4). In turn, the subscales Negative affects and Social support are significantly associated not only with the BSI-GSI (r = .59, p < .01 and r = - .26, p < .01), but with all dimensions of the BSI as well (p < .01) (Table 4).

Table 3
Correlation matrix of subscales

	Global score	Therapeutic involvement	Social support	Peer support
Global score
Therapeutic involvement	.569**
Social support	.678**	.494**
Peer support	.454**	.225**	.285**
Negative affects	-.736**	-.034	-.181**	-.065

• **p < .01.

Table 4
Correlation matrix of brief questionnaire and BSI scales

	Global score	Therapeutic involvement	Social Support	Peer support	Negative affects
BSI-GSI	-.515**	-.128**	-.261**	-.011	.592**
Somatization	-.394**	-.132**	-.200**	-.037	.424**
Obsession-compulsion	-.345**	-.061	-.184**	.037	.420**
Interpersonal sensitivity	-.329**	-.047	-.161**	.070	.427**
Depression	-.515**	-.179**	-.313**	-.053	.521**
Anxiety	-.443**	-.079	-.183**	.025	.561**
Hostility	-.493**	-.146**	-.219**	-.035	.562**
Phobic anxiety	-.231**	-.027	-.134**	.023	.284**
Paranoid ideation	-.406**	-.069	-.233**	-.045	.458**
Psychoticism	-.403**	-.101*	-.171**	.017	.489**

• ** p < .01; * p < .05.

Discussion

The main goal of this article was to develop a brief questionnaire of treatment monitoring, suitable for both inpatient and outpatient programs, to assess critical indicators of treatment effectiveness in a short time period. The choice of the TCU-CEST instrument was based upon four basic criteria: (a) adequate psychometric proprieties found in the original instrument, (b) several TCU-CEST subscales have been applied to a variety of dependence treatment programs, (c) wide assessment of several efficacy indicators, and (d) confirmed usefulness in clinical practice and in the present changes in treatment programs⁷,¹⁴. We started from a theoretical model of this original CEST⁷,¹⁴, and the Portuguese version of Moura et al.²⁰, to found a brief questionnaire to evaluate the treatment effectiveness of substance abuse, using both inpatient and outpatient samples, and subsequently EFA and CFA analyses. According to our purposes, we found a different factor structure when compared to the original instrument, with a four-factor model from EFA analyses. These four factors also showed adequate internal reliability. The adequacy of this structure to a different sample was confirmed through the CFA. Nevertheless, a model re-specification was necessary. We believe that the statistical model allowed us to find a different version due to the inclusion of patient characteristics of different treatment settings, compared to the original instrument.

Despite this, our final model covers the main treatment effectiveness areas pointed out in literature. The first factor, identified as Therapeutic involvement dimension, includes items intended to measure treatment engagement, treatment satisfaction and counseling rapport. The second factor, identified as Negative affects dimension, includes items to measure the psychological functioning in terms of depression, hostility and anxiety. Third and fourth factors, originally identified as Social and Peer support scales, respectively, seek to measure social and contextual factors that also affect recovery process dynamics⁷. Former instruments are extensive, which takes a long time to be completed and analyzed, like the software developed by Machado et al.¹, where several instruments were combined resulting in a 126-items instruments. Besides, these instruments don’t include some important constructs stressed by literature such as social and peer support network⁷,¹⁴, the therapeutic relationship, the involvement or active participation in treatment¹³-¹⁵, psychopathological symptoms or negative affects¹⁶-¹⁹.

In this way, when compared with others instruments, this new short-form presented here is a proper response since: (a) it is suitable for both inpatient and outpatient programs; (b) encompasses few, but critical indicators of treatment, using a problem-solving and a psychosocial models; (c) takes short time to be completed; and (d) allows building a database for the monitoring of intervention efficacy available in clinical practice.

Our second aim was to investigate the convergent validity of the brief instrument. There were significant associations of most of these subscales with each other and with the total score, suggesting the concordance of the subscales for the evaluation of different aspects of a common construct. Furthermore, we concluded that this study showed a significant correlation between the brief instrument and the BSI. The majority of the brief instrument subscales (75%) were significantly associated with at least five or more dimensions of the BSI, and half of the brief instrument subscales (50%) were significantly associated with all dimensions of the BSI and total score. Furthermore, the global score of the brief instrument was also significantly associated with all dimensions of the BSI and total score. Exceptionally, the scale Peer support was not significantly associated with any of the BSI’s dimensions. Despite of the low probability that this subscale had any relation with psychopathological symptoms, it was significantly associated with the other subscales of the new instrument, as well as with the total score, being reasonable to keep this scale in the final instrument.

These results are consistent with the literature stating that psychopathology³⁴,³⁵and comorbidity³⁶,³⁷are very commons in this population. Psychopathology was one of the best predictors of the effectiveness of treatment programs³⁵, increasing the likelihood of treatment failure³⁴,³⁵. Therefore, if this brief instrument is strongly and significantly associated with the BSI (a predictor of the effectiveness of treatment programs), these data seem to reinforce the potential of this version in the estimation of treatment efficacy. Considering the practical implications, this new brief questionnaire, with adequate psychometric properties, can be useful to provide a viable and rapid feedback instrument for counselors, treatment programs and policy makers, covering some of the main and most relevant areas related to the monitoring of drugs and alcohol treatment programs³,⁵,⁷-⁹. Furthermore, this practice can also enable the adjustment of the interventions according to the users’ needs and comorbidities⁷,⁹,¹⁴,²⁰. We also suggest that this instrument should be part of the daily work in clinical practice in Portugal. Thus, more studies may be conducted in context: measuring the treatment effectiveness and warning of the need for effective changes in the recovery process.

Limitations

First, these three major treatment settings are not necessarily representative of all treatment programs for drug or alcohol addiction in Portugal. Second, this short version does not measure the same number of dimensions of treatment efficacy as the original version. Thus, we can conceive it only as an important complementary tool for this assessment. Finally, more studies should be conducted in Portugal with this short version to assess temporal validation of this instrument, and to observe correlations with others indicators of treatment progress (such as attendance, time in treatment or completion of treatment) in order to underline its predictive capacity.

Acknowledgements

This research was funded by a doctoral fellowship provided by Foundation for Science and Technology (FCT) Grant nº SFRH/BD/67113/2009. The authors express their appreciation to all participants and to the staff of the Intervention Service on Addictive Behaviors and Substance Dependence (SICAD): Regional Northern Section. The authors also wish to express a special acknowledgement to Integrated Response Centers – West Porto and to the Texas Christian University (TCU) – Institute of Behavioral Research (IBR) for their support. Interpretations and conclusions, however, are entirely those of the authors and do not necessarily represent the position of the FCT, SICAD or TCU-IBR.

References

• ¹
  Machado P, Klein J, Farate C. Monitorização dos resultados terapêuticos no contexto de tratamento das toxicodependências [Monitoring of therapeutic results in the drug treatment context]. Psicologia: Teoria, Investigação e Prática. 2005;1:19-29. Available from: <http://hdl.handle.net/1822/3023>.
  » http://hdl.handle.net/1822/3023>
• ²
  McLellan AT, McKay JR, Forman R, Cacciola J, Kemp J. Reconsidering the evaluation of addiction treatment: from retrospective follow-up to concurrent recovery monitoring. Addiction. 2005;100(4):447-58.
• ³
  White WL. Addiction recovery: its definition and conceptual boundaries. J Subst Abuse Treat. 2007;33(3):229-41.
• ⁴
  Cotter RB, Burke JD, Stouthamer-Loeber M, Loeber R. Contacting participants for follow-up: how much effort is required to retain participants in longitudinal studies? Eval Program Plann. 2005;28(1):15-21.
• ⁵
  Scott CK, White WL. Ethical issues in the conduct of longitudinal studies of addiction treatment. J Subst Abuse Treat. 2005;28 Suppl 1:S91-101.
• ⁶
  Lehman WE, Greener JM, Simpson DD. Assessing organizational readiness for change. J Subst Abuse Treat. 2002;22(4):197-209.
• ⁷
  Joe GW, Broome KM, Rowan-Szal GA, Simpson DD. Measuring patient attributes and engagement in treatment. J Subst Abuse Treat. 2002;22(4):183-96.
• ⁸
  Flynn PM, Brown BS. Implementation research: issues and prospects. Addict Behav. 2011;36(6):566-9.
• ⁹
  Simpson DD, Joe GW. A longitudinal evaluation of treatment engagement and recovery stages. J Subst Abuse Treat. 2004;27(2):89-97.
• ¹⁰
  Tiet QQ, Byrnes HF, Barnett P, Finney JW. A practical system for monitoring the outcomes of substance use disorder patients. J Subst Abuse Treat. 2006;30(4):337-47.
• ¹¹
  Harrison PA, Asche SE. Outcomes monitoring in Minnesota: treatment implications, practical limitations. J Subst Abuse Treat. 2001;21(4):173-83.
• ¹²
  Marsden J, Nizzoli U, Corbelli C, Margaron H, Torres M, Prada de Castro I, et al. New European instruments for treatment outcome research: reliability of the Maudsley Addiction Profile and treatment perceptions questionnaire in Italy, Spain and Portugal. Eur Addict Res. 2000;6(3):115-22.
• ¹³
  Joe GW, Simpson DD, Rowan-Szal GA. Interaction of counseling rapport and topics discussed in sessions with methadone treatment clients. Subst Use Misuse. 2009;44(1):3-17.
• ¹⁴
  Simpson DD. A conceptual framework for drug treatment process and outcomes. Subst Abuse Treat. 2004;27(2):99-121.
• ¹⁵
  Simpson DD, Joe GW. A longitudinal evaluation of treatment engagement and recovery stages. J Subst Abuse Treat. 2004;27(2):89-97.
• ¹⁶
  Baker TB, Piper ME, McCarthy DE, Majeskie MR, Fiore MC. Addiction motivation reformulated: an affective processing model of negative reinforcement. Psychol Rev. 2004;111(1):33-51.
• ¹⁷
  Fox HC, Talih M, Malison R, Anderson GM, Kreek MJ, Sinha R. Frequency of recent cocaine and alcohol use affects drug craving and associated responses to stress and drug-related cues. Psychoneuroendocrinology. 2005;30(9):880-91.
• ¹⁸
  Novak A, Burgess ES, Clark M, Zvolensky MJ, Brown RA. Anxiety sensitivity, self-reported motives for alcohol and nicotine use, and level of consumption. J Anxiety Disord. 2003;17(2):165-80.
• ¹⁹
  Zvolensky MJ, Leen-Feldner EW. Anxiety and stress vulnerability and substance problems: theory, empirical evidence, and directions for future research. Clin Psychol Rev. 2005;25(6):707-12.
• ²⁰
  Moura A, Ferros L, Negreiros J. Evaluation and monitoring instrument: Client Evaluation of Self and Treatment. Arch Clin Psychiatry. 2013;40(4):165-6.
• ²¹
  Derogatis LR, Lipman RS, CoviL. SCL-90: an outpatient psychiatric rating scale – preliminary report. Psychopharmacol Bull. 1973;9(1):13-28.
• ²²
  Canavarro MC. Inventário de sintomas psicopatológicos – BSI [Psychopatological symptoms inventory (BSI)]. In: Simões MR, Gonçalves M, Almeida LS, editors. Testes e provas psicológicas em Portugal [Psychological tests and proofs in Portugal]. Braga: SHO/APPORT; 1999. p. 87-109.
• ²³
  West SG, Finch JF, Curran PJ. Structural equation models with nonnormal variables: problems and remedies. In: Hoyle RH, editor. Structural equation modeling: concepts, issues and applications. Thousand Oaks, CA: Sage; 1995. p. 55-75.
• ²⁴
  Dziuban CD, Shirkey EC. When is a correlation matrix appropriate for factor analysis? Psychol Bull. 1974;81:358-61.
• ²⁵
  Kline RB. Principles and practice of structural equation modeling. 3rd ed. New York: Guilford Press; 2011.
• ²⁶
  Arbuckle JL. Amos 17 users’ guide. Chicago, IL: SPSS; 2008.
• ²⁷
  Bentler PM. Comparative fit indexes in structural models. Psychol Bull. 1990;107:238-46.
• ²⁸
  MacCallum RC, Widamam KF, Zhang S, Hong S. Sample size in factor analysis. Psychol Methods. 1999;4:84-99.
• ²⁹
  Jöreskog KG, Sörbom D. LISREL 8 User’s Reference Guide Ulppsalce. Software Iternational. Seveden: Scientific; 1998.
• ³⁰
  Marôco J. Análise de equações estruturais. Fundamentos teóricos, software & aplicações [Structural equation analysis. Theoretical foundations, software & applications]. Pêro Pinheiro, Portugal: Report Number; 2010.
• ³¹
  Pestana MH, Gageiro JN. Análise de dados para ciências sociais: a complementariedade do SPSS [Data analysis for the social sciences: the complementarity of SPSS]. Lisboa: Edições Sílabo; 2005.
• ³²
  Hair Jr J, Anderson R, Tatham R, Black W. Análise multivariada de dados [Multivariate data analysis]. Porto Alegre: Bookman; 2005.
• ³³
  Schermelleh-Engel K, Moosbrugger H, Müller H. Evaluating the fit of structural equation models: Test of significance and descriptive goodness-of-fit measures. Methods of Psychological Research Online. 2003;8(2):23-74.
• ³⁴
  Dawe S, Loxton NJ, Hides L, Kavanagh DJ, Mattick RP. Review of diagnostic screening instruments for alcohol and other drug use and other psychiatric disorders. 2nd ed. Canberra: Commonwealth Department of Health and Ageing; 2002.
• ³⁵
  Macias JG, Leal FJ, Fernandez-Gil MA, Pacheco DP, Aliño JI. Comorbilidade psiquiátrica en drogodependencias [Psychiatric comorbility on drug addiction]. Psiquiatria.com. 2000;4(4).
• ³⁶
  Marsden V. Comorbilidades entre dependência química, distimia, HIV e HCV: relato de caso [Comorbidity between addiction, dysthymia, HIV and HCV: case report]. Rev Psiquiatr Clin. 2009;36(1):31-3.
• ³⁷
  Fontes A, Figlie NB, Laranjeira R. O comportamento de beber entre dependentes de álcool: estudo de seguimento [Drinking behaviour between alcohol users: a follow-up study]. Rev Psiquiatr Clin. 2006;33(6):304-12.

Publication Dates

• Publication in this collection
  Jul-Aug 2015

History

• Received
  28 Oct 2014
• Accepted
  29 July 2015