A Portuguese version of the Christo Inventory for Substance-Misuse Services: a simple outcome evaluation tool

Braz J Med Biol Res, October 2002, Volume 35(10) 1111-1117 (Review)

A Portuguese version of the Christo Inventory for Substance-Misuse Services: a simple outcome evaluation tool

G. Christo¹ and V.A. Silva²

¹Response Drug Service, Camden and Islington Mental Health and Social Care Trust, London, UK

²Centro Regional Integrado de Atendimento ao Adolescente, Universidade Federal Fluminense, Niterói, RJ, Brasil

^Text

References

^{Correspondence and Footnotes}

Abstract

The Christo Inventory for Substance-Misuse Services (CISS) is a single page outcome evaluation tool completed by drug alcohol service workers either on the basis of direct client interviews or of personal experience of their client supplemented by existing assessment notes. It was developed to assist substance misuse services to empirically demonstrate the effectiveness of their treatments to their respective funding bodies. Its 0 to 20 unidimensional scale consists of 10 items reflecting clients' problems with social functioning, general health, sexual/injecting risk behavior, psychological functioning, occupation, criminal involvement, drug/alcohol use, ongoing support, compliance, and working relationships. Good reliability and validity has already been demonstrated for the CISS [Christo et al., Drug and Alcohol Dependence 2000; 59: 189-197] but the original was written in English and a Portuguese version is presented here. The present review explores its applicability to a Brazilian setting, summarizes its characteristics and uses, and describes the process of translation to Portuguese. A pilot study conducted in a substance misuse service for adolescents indicated it is likely to be suitable for use among a Brazilian population. The simplicity, flexibility and brevity of the CISS make it a useful tool allowing comparison of clients within and between many different service settings.

Key words: Substance misuse, Alcohol, Treatment outcome, Evaluation, Clinical audit

Introduction

Substance misuse services are frequently required to empirically demonstrate the efficacy of their treatments. But evaluation has been a challenge for services that have neither the time nor the expertise to conduct detailed outcome research with multidimensional outcome questionnaires such as the Addiction Severity Index (1,2), the Opiate Treatment Index (3), and the Maudsley Addiction Profile (4). While excellent for individual assessments or academic research, these questionnaires took a while to complete, required the presence of the client in question, and could not combine subsections to produce a single total score for easy analysis by non-researchers. There was still a need for a clinical evaluation instrument that could produce a single score reflective of client problems across all relevant outcome domains. To be acceptable to busy workers, the instrument would have to be short, simple and not dependent on the unreliable attendance of substance misuse service clients. Since workers generally have a wealth of qualitative data from detailed client notes and assessment interviews, it may be assumed that they are able to provide the required information on their clients' behalf. Although the information would largely consist of subjective impressions that may be difficult to validate, competent workers should be familiar with their client's status within the relevant outcome areas.

The areas considered relevant among the above substance misuse outcome questionnaires are social functioning, general health, sexual/injecting risk behavior, psychological functioning, occupation, criminal involvement, and drug/alcohol use. Other factors linked to good outcomes include continued use of ongoing support or aftercare (5,6), and treatment compliance as illustrated by length of stay or treatment level attained (7,8). Thus, the Christo Inventory for Substance-Misuse Services (CISS) was developed from elements within the aforementioned references to elicit workers' assessments of their clients in a quick, standardized and reliable way.

The CISS prototype was used in the evaluation of outcomes of treatment placements purchased by social services (9). This pilot study illustrated the scale's usefulness, sensitivity to change and ease of completion. Feedback from workers was used to refine the nine original items and a tenth item was added reflecting the quality of the working relationship with the client. Therapeutic relationships, pretreatment motivation and program engagement have been shown to be central attributes of effective treatment (10).

A validation study (11) was subsequently conducted to produce comparison scores and reliability data for the refined scale. The simplicity, flexibility and brevity of the CISS made it a useful tool allowing comparison of clients within and between many different service settings. It rapidly became one of the most commonly used substance misuse outcome tools in the UK (12) as commissioners discovered that their service providers were willing to accept the minimal extra workload imposed by its use. The tool is freely distributed via the internet (13), the conditions for use being that it is not sold for profit and the title, instructions and item wording are not altered. There has since been a lot of interest from countries outside the UK and this paper presents a Portuguese translation of the CISS.

The original tool

The CISS was developed to elicit workers' impressions of their clients' dysfunction in a quick, standardized and reliable way. It is a single page outcome evaluation tool completed by drug/alcohol service workers either on the basis of direct client interviews or of personal experience of their client, supplemented by existing assessment notes. Its 0 to 20 unidimensional scale consists of 10 items reflecting clients' problems with social functioning, general health, sexual/injecting risk behavior, psychological functioning, occupation, criminal involvement, drug/alcohol use, ongoing support, compliance, and working relationships. These outcome areas are scored on a three-point scale of problem severity (0 = none, 1 = moderate, 2 = severe) and each point is illustrated with relevant examples for guidance. Thus, a total CISS score of 0 indicates no problems, and a maximum score of 20 indicates severe problems in all areas.

Use of a single outcome measure produces a simple and readable report without the assistance of a professional researcher/statistician. The process is further simplified because the CISS outcome score can be reduced to two or three categories, e.g., 'good/poor' outcome or 'low/average/high' problem severity. For abstinence-oriented treatment, a score of 6 or less is indicative of a good outcome. For drug services based on harm minimization, a score of 0 to 5 = low problem severity, 6 to 12 = average, 13 to 20 = high problem severity. For outpatient alcohol services the score distribution is similar but shifted down one point less than drug users. Alcohol users are less likely to score on problems of social functioning, HIV risk behavior and criminal involvement, but they are more likely to score on psychological problems. Item alpha, test-retest and inter-rater reliability coefficients are 0.74, 0.82 and 0.82, respectively. The inter-rater coefficient increased to 0.91 when retests were conducted on the same day. The CISS produced correlations ranging from 0.43 to 0.99 with the Opiate Treatment Index and measures of trait anxiety, unpleasant life events, poor quality of life and low self-esteem (1).

The translation process

The CISS was translated into Portuguese by a professional whose native language was Portuguese and back into English by a bilingual professional who did not know the original version. This was a scientist working in another area of knowledge having English as his native language. This derived version was then compared to the original by the author of the CISS (Dr. George Christo) and 15 inconsistencies were identified. Another bilingual researcher, who was a psychiatrist familiar with the English version of the CISS and whose native language was Portuguese, found that three of the inconsistencies derived from the translation from English into Portuguese and 12 from the translation from Portuguese into English. The Portuguese version was then corrected in order to solve the problems that could lead to misinterpretation. Portuguese style was also improved. This new version was then submitted to a bilingual specialist in chemical dependency whose native language was English. Five other possible sources of misinterpretation were identified and corrected, thus leading to the final version of the Portuguese CISS (see ^Appendix).

In parallel to this procedure the English version of the CISS was piloted by a bilingual psychiatrist in a Brazilian service (Centro Regional Integrado de Atendimento ao Adolescente, CRIAA, Universidade Federal Fluminense, Niterói, RJ, Brazil) which gives assistance to adolescents engaged in substance misuse. This day service provides counseling, social support and a safe environment for adolescents wishing to stay away from drugs and crime.

An example of use

Preliminary evaluations of the sociodemographic characteristics of the clients assisted at the Brazilian adolescents' clinic indicated that they were mostly cannabis users, males aged on average 15 years and belonging to the working class, born to parents with high rates of alcoholism and of low educational level. Their performance at school was poor.

CISS forms were completed for 35 adolescents at treatment entry and the 10 CISS domains were found to adapt easily to the different cultural settings and ages of the adolescents. Two-tailed nonparametric Mann-Whitney U-tests were used to contrast the Brazilian adolescents with 243 London UK adult clients from the original validation study of Christo et al. (11) (mostly heroin addicts). The authors present this contrast only to illustrate how the CISS can be used to quantify differences between groups; it is not suggested that these populations are in any way comparable. The adolescents were found to score more highly on problems of sexual risk behavior, criminal involvement, drug use, and ongoing support (Figure 1). This reflected the fact that many adolescents were having unprotected sex and were involved with criminal gangs selling drugs. The London UK adults were already enrolled in a drug clinic when they were assessed and that is why they scored lower than the Brazilian adolescents on problems of drug use and ongoing support. The adolescents scored less on problems of social functioning, general health, compliance, and working relationships, thus indicating that they were more likely to have stable accommodation (with their parents), fewer drug using associates, better health, better reliability, and fewer difficulties in relationships with the clinic staff. A two-sample t-test revealed no difference in mean total CISS scores between the Brazilian adolescents (mean ± SD, 8.5 ± 2.5) and the London UK adult addicts (9.1 ± 3.4).

Opportunistic follow-up scores were obtained for 16 of the 35 Brazilian adolescents, and follow-up periods ranged from 1 to 12 months. Figure 2 indicates the changes in item scores between the initial evaluation and the follow-up for this subgroup. There were some reductions in problems of general health, sexual risk behavior, occupation, crime, drug/alcohol use, and support. However, paired-sample nonparametric Wilcoxon tests on all CISS items indicated that only the reductions in criminal behavior were significant. No other significant differences were found but this may have been due to the small number of adolescents in this sample.

Figure 1.
Mean CISS item scores of 243 London UK heroin addicts versus 35 Brazilian adolescents attending Centro Regional Integrado de Atendimento ao Adolescente (CRIAA).

[View larger version of this image (34 K GIF file)]

Figure 2.
Mean CISS item scores of 16 Brazilian adolescents attending Centro Regional Integrado de Atendimento ao Adolescente (CRIAA) at entry and during follow-up.

[View larger version of this image (35 K GIF file)]

Discussion

The CISS demonstrated good general applicability to a Brazilian setting as its items were quite acceptable to workers and clients. Participating workers stated that they preferred it over other instruments that were less simple, took longer to complete and required the presence of the client. Clients generally preferred face-to-face interviews that did not require self-completion questionnaires, and a significant minority of clients also had reading difficulties.

Limitations

The results presented above are preliminary findings from a brief pilot study undertaken to assess the general applicability of the CISS to a Brazilian adolescent setting. They are reported simply as an example to illustrate how the CISS can be used in clinical evaluation work and are not intended to represent formal academic research. Comparison scores enable practitioners at other services to assess their clients' status relative to a 'typical' group of presenting clients. However, the criterion groups used to standardize the CISS were all obtained from London-based services in the UK. The score distributions they have generated are a useful guide, but further research is necessary to ensure that these findings can be generally applied to other services in Brazil. A more thorough study is currently under way using the Brazilian version of the CISS.

The CISS can be completed in the client's absence, but in these cases it is intended for use by individuals who have previously worked with the client in question and who know the client well.

The CISS is principally a clinical evaluation tool and may not be suitable for detailed research where separate measures of functioning in independent domains (e.g., drug use and health) may be required. Breakdowns showing the percentage of clients with severe, moderate or no problems may be produced for separate CISS items. However, the individual item three-point scales were not designed to be used separately in small samples. Sensitivity to change is only gained when comparing items collated from many clients, or when the 10 items are combined to form the 0 to 20 CISS total score. Some services using the CISS have added supplementary scales to produce a more detailed record of an outcome of particular interest (e.g., drug use, self-esteem). However, the CISS has been shown to correlate well with such scales (11) and their use may be unnecessary for evaluation purposes.

Applications

The CISS score is a general index of client problems across domains considered relevant to treatment outcome. Only two of its items are directly related to substance use, so it has sufficient scope to monitor outcomes where abstinence is not the main goal of treatment. However, it also produces markedly reduced scores in cases in which abstinence has been achieved. The CISS is thus being used for ongoing evaluation at abstinence-based and harm minimization treatment services, where it is incorporated as a regular part of the admission, discharge and case review procedures. CISS scores may assist in the prioritization of new clients and the distribution of caseloads of equal difficulty among keyworkers. Self-completion outcome scales are of little use when clients abandon treatment without notice. When using the CISS a keyworker can at least retrospectively report discharge status even after the client is no longer accessible.

The use of a single outcome measure facilitates the production of simple and readable clinical evaluation reports, without the need for a professional researcher/statistician trained in data handling and the analysis of multiple outcomes. Data analysis can be further simplified by reducing the CISS total score to the two or three outcome categories identified earlier (11), e.g., 'good/poor' outcome or 'low/average/high' problem severity.

Although principally designed for outcome evaluation in busy clinics, the CISS has also been found to have a range of clinical purposes such as:

· Balancing staff caseloads, ensuring that the clinical staff do not have an unequal share of difficult clients.

· Quick summaries for other professionals. Staff simply underline some of the words on the CISS form to give a quick but detailed indication of the client's problems.

· Motivational client work using comparisons. If clients do not believe that their drug use has caused them excessive problems, staff may show them their score relative to the distribution of other clients at the clinic.

· Review/reinforce changes made by clients. Staff will complete a CISS with the client during their review, and then compare it to the CISS that was taken at their original admission interview.

· Identifying clients resistant to change. Using the clinic's database it is possible to subtract the follow-up score from the admission score and extract a list of all clients who have increased their scores by more than two points, thus indicating a deterioration in their condition. Also, clients who continue to hold scores above 13 are identified. These clients are then reviewed with their keyworker by a consultant psychiatrist and changes in treatment strategy may be suggested.

· Identifying areas of client need. For example, some clinics have discovered that the highest scoring CISS item was "occupational problems" (it was even greater than "drug use"). This finding has been used to secure funding for a worker who specializes in helping clients engage in work, sport, study, college, or voluntary activities.

· Selecting suitable clients for trainees or for low intervention treatments. CISS scores at admission are printed next to client's names on the waiting list, so the clinic manager can tell immediately if a client might be too difficult to allocate to a trainee or inexperienced worker.

The CISS has been shown to be easy to use and very flexible in adapting to different service requirements. Its single page is easy to photocopy and uses little space in client files. It can be completed face-to-face or from personal experience of a client supplemented by existing assessment notes. Workers familiar with the instrument are able to complete it in three minutes during their administration time. The present review has shown that the CISS has translated well into Portuguese and is likely to be of use in Brazilian substance misuse services.

Acknowledgments

We wish to thank Dr. Foster Brown and Dr. John Dunn for their assistance in the translation process. We are also indebted to all the staff at Centro Regional Integrado de Atendimento ao Adolescente (Universidade Federal Fluminense, Niterói, RJ, Brazil) for their assistance in the pilot study.

Address for correspondence: G. Christo, Response Drug Service, 457 Finchley Road, London NW3 6HN, UK. Fax: [phone]. E-mail: [e-mail]

Presented at the XVI Annual Meeting of the Federação de Sociedades de Biologia Experimental, Caxambu, MG, Brazil, August 29 - September 1, 2001. CISS forms with directions and comparison score tables are available from either author. Received April 3, 2002. Accepted June 6, 2002.

• 1. McLellan AT, Luborsky L, Woody GE & O'Brien CP (1980). An improved diagnostic evaluation instrument for substance abuse patients: the Addiction Severity Index. Journal of Nervous and Mental Disease,168: 26-33.
• 2. McLellan AT, Kushner H, Metzger D, Peters R, Smith I, Grisson G, Pettinati H & Argeriou M (1992). The fifth edition of the Addiction Severity Index. Journal of Substance Abuse Treatment, 9: 199-213.
• 3. Darke S, Hall W, Wodak A, Heather N & Ward J (1992). Development and validation of a multi-dimensional instrument for assessing outcome of treatment among opiate users: the Opiate Treatment Index. British Journal of Addiction, 87: 733-742.
• 4. Marsden J, Gossop M, Stewart D, Best D, Farrell M, Lehmann P, Edwards C & Strang J (1998). The Maudsley Addiction Profile (MAP): a brief instrument for assessing treatment outcome. Addiction, 93: 1857-1868.
• 5. Christo G & Franey C (1995). Drug users' spiritual beliefs, locus of control and the disease concept in relation to Narcotics Anonymous attendance and six-month outcomes. Drug and Alcohol Dependence, 38: 51-56.
• 6. Ouimette PC, Moos RH & Finney JW (1998). Influence of outpatient treatment and 12-step group involvement on one-year substance abuse treatment outcomes. Journal of Studies on Alcohol, 59: 513-522.
• 7. Shwartz M, Mulvey KP, Woods D, Brannigan P & Plough A (1997). Length of stay as an outcome in an era of managed care. Journal of Substance Abuse Treatment, 14: 11-18.
• 8. Toumbourou JW, Hamilton M & Fallon B (1998). Treatment level progress and time spent in treatment in the prediction of outcomes following drug-free therapeutic community treatment. Addiction, 93: 1051-1064.
• 9. Christo G (1998). Outcomes of Residential Care Placements for People with Drug and Alcohol Problems: An Evaluation of Hammersmith and Fulham Social Services The Centre for Research on Drugs and Health Behaviour, London, UK.
• 10. Simpson DD, Joe GW, Rowan-Szal GA & Greener JM (1997). Drug abuse treatment process components that improve retention. Journal of Substance Abuse Treatment, 14: 565-572.
• 11. Christo G, Spurrell S & Alcorn R (2000). Validation of the Christo Inventory for Substance-misuse Services (CISS): a simple outcome evaluation tool. Drug and Alcohol Dependence, 59: 189-197.
• 12. The Audit Commission for Local Authorities and the National Health Service in England and Wales (2002). National Report, Changing Habits: The Commissioning and Management of Community Drug Treatment Services for Adults Audit Commission Publications, Wetherby, UK, 71.
• 13. Christo Research Systems Home Page. [http://users.breathemail.net/drgeorge christo/]. Accessed April 2, 2002.

Appendix

Appendix

Correspondence and Footnotes

Publication Dates

• Publication in this collection
  15 Oct 2002
• Date of issue
  Oct 2002

History

• Received
  03 Apr 2002
• Accepted
  06 June 2002