Performance assessment of CAGE screening test among pschiatric outpatients

Abstracts

Estudo descritivo, transversal com objetivo de avaliar o desempenho do CAGE entre pacientes psiquiátricos ambulatoriais de um hospital universitário terciário. Amostra de conveniência, pacientes do HCFMRP-USP (n=127). Aplicaram-se o CAGE e a entrevista diagnóstica da CID-10 para uso nocivo e dependência de álcool. O desempenho dos escores do CAGE foi avaliado através da análise de curva ROC, tendo os diagnósticos clínicos da CID-10 como padrão-ouro. Sensibilidade e especificidade de acordo com o ponto de corte: >0, S=100%, E=0%; >1, S=100%, E=73,7%; >2, S=53,8%, E=87,7%; >3, S=53,8%, E=94,7%; >4, S=0%, :E=100%. O escore 1 mostrou ser o ponto crítico ideal sensibilidade/especificidade. Entre pacientes psiquiátricos ambulatoriais de um hospital universitário terciário, o ponto de corte mais adequado do CAGE foi >1. A fim de aumentar o poder diagnóstico do teste e a segurança dos seus resultados, recomenda-se a avaliação do desempenho deste na população estudada.

escalas; sensibilidade e especificidade; transtornos relacionados ao uso de substâncias; pacientes ambulatoriais; alcoolismo

ORIGINAL ARTICLE

Performance assessment of cage screening test among pschiatric outpatients

Evaluación del desempeño del cage con pacientes psiquiátricos ambulatorios

Clarissa Mendonça Corradi-Webster^I; Milton Roberto Laprega^II; Erikson Felipe Furtado^II

^IPsychologist; Msc. Community Health, University of São Paulo, e-mail: [e-mail]

^IIMD PhD, School of Medicine of Ribeirão Preto of University of São Paulo, e-mail: [e-mail], e-mail: [e-mail]

ABSTRACT

Descriptive and transversal study to evaluate the performance of CAGE among psychiatric outpatients at a tertiary-level university hospital. Convenience sample composed of patients from HCFMRP-USP (n=127). The instruments used were CAGE and the diagnostic interview based on ICD-10 criteria for harmful use and alcohol dependence. The performance of CAGE scores was evaluated through the analysis of ROC curve, using the ICD-10 clinical diagnostic as a gold standard. The sensitivity and specificity in accordance with the cut-off points are: >0, Sens=100%, Spec=0%; >1, Sens=100%, Spec=73.7%; >2, Sens=53.8%, Spec=87.7%; >3, Sens=53.8%, Spec=94.7%; >4, Sens=0%; Spec=100%. Score 1 showed to be the ideal critical point for sensibility/specificity. >1 was the best cut-off point for CAGE among psychiatric outpatients from a tertiary-level university hospital. To increase the diagnostic power of the test and the reliability of its results, its performance in the study population should be assessed.

Descriptors: scales; sensitivity and specificity; substance-related disorders; outpatients; alcoholism

RESUMEN

Estudio descriptivo, transversal, con objeto de evaluar el desempeño del CAGE entre pacientes psiquiátricos ambulatorios de un hospital universitario terciario. Muestra de conveniencia con pacientes del HCFMRP-USP (n=127). Fue aplicado el CAGE y entrevista diagnóstica del CID-10 para uso nocivo y dependencia de alcohol. El desempeño de la puntuación en el CAGE fue evaluado a través del análisis de la curva ROC, usando los diagnósticos del CID-10 como patrón-oro. La sensibilidad y especificidad de acuerdo con punto de corte: >0, S=100%, E=0%; >1, S=100%, E=73,7%; >2, S=53,8%, E=87,7%; >3, S=53,8%, E=94,7%; >4, S=0%, E=100%. La puntuación 1 mostró ser el punto crítico ideal sensibilidad/especificidad. Entre pacientes psiquiátricos ambulatorios de un hospital universitario terciario el punto de corte más adecuado del CAGE fue >1. A fin de aumentar el poder diagnóstico del test y la seguridad de sus resultados se recomienda que se realice la evaluación del desempeño del test en la población estudiada.

Descriptores: escalas; sensibilidad y especificidad; trastornos relacionados con sustancias; pacientes ambulatorios; alcoholismo

INTRODUCTION

Excessive alcohol consumption by persons with psychiatric problems can be a high risk to health. According to the World Health Organization (WHO), the probability of dependence on alcohol during the lifetime in the general population is 14% and for an individual with a psychiatric disorder 22%. The probability of alcohol dependence is 2.3 times higher among persons with some psychiatric disorder than among persons with no such disorders⁽¹⁾.

It is very important to identify problematic alcohol use at an early stage, as this provides professionals with the opportunity to act preventively and, hence, reduce the problems caused by this consumption. In the case of psychiatric patients, alcohol use can also impair treatment adherence, interact with medication effects and worsen disease prognoses.

In recent years, various instruments have been developed to assist professionals in this identification. These instruments can be used by any health professional with a minimum of training and they help them to do a brief assessment. They are good for day to day use, have easy and rapid applicability and they offer a clear score that indicates the probability of alcohol related problems. Even not being diagnoses instruments, these tests were evaluated based on their ability of differentiated with reliability between individuals that have the clinical condition and those that don't.

CAGE, created in the 70's, is a questionnaire with this purpose. It has only four questions with objectives answers (yes/no) and it is used often because of it's easy application and it's good acceptability among the professionals and patients. It was validated in Brazil in 1983, when the application of its four questions mixed with another ten questions about health habits, to facilitate the interview, was proposed ⁽²⁾. The name CAGE comes from the key-words contained in each one of the questions: 1) Have you ever felt you should Cut down on your drinking? 2) Have people Annoyed you by criticizing your drinking? 3) Have you ever felt bad or Guilt about your drinking? 4) Have you ever had a drink first thing in the morning to stead your nerves or to get rid of a hangover (i.e., as an Eye-opener)? The questions have to be answers with yes or no and answers like "sometimes" are considered as a yes answer.

Another widely used screening test is AUDIT (Alcohol Use Disorders Identification Test). It was developed upon request of WHO to be used as a screening instrument of hazardous use, harmful use and alcohol dependence⁽³⁾. AUDIT consists of 10 questions graduated from zero to four and emphasizes the identification of alcohol use disorders in the last 12 months, assessing alcohol consumption levels, dependence symptoms and problems related to this use. It has been well accepted by professionals and is used by many health services⁽⁴⁾.The screening tests are the objects of many studies, mostly to verify the validity in relation to specific groups, like women, elderly, adolescents and psychiatric patients. Some authors proposed a modification in the CAGE questionnaire to be used with pregnant women⁽⁵⁾. This instrument received the name of T-ACE, and it was validated in Brazil in 2002 ⁽⁶⁾. The literature also proposes that adaptations have to be made changing the cut-off point of the instrument to guarantee satisfactory specificity and sensitivity⁽⁷⁾. When CAGE was validated in Brazil the cut-off point suggested was >2, however, some authors recommend the cut-off point of >1 according to the studied population^(8-9). The AUDIT's manual also recommends the reduction or increase of the cut-off point with some determined groups to improve its power of detection⁽³⁾.

Although the screening instruments indicate the possibility of an existing problem, they can't be used to conclude a psychiatric diagnosis. For this, the instrument that is more commonly used is the structured interview, conforming to the Diagnostic Criteria for Research (DCR-10) of ICD-10. This interview allows the identification of harmful use and alcohol dependence syndrome, using the criteria derivate of Chapter V - Mental and Behavioural Disorders, of ICD-10. It was developed for clinical and general educational use and when it is used by a qualified professional, it can be considered to have high reliability for the diagnose of alcohol use disorder⁽¹⁰⁾.

Following the suggestions of the literature to adapt the screening instruments, this study had as an objective the evaluation of the performance of CAGE in the identification of problematic use of alcohol among patients of psychiatric outpatient service from a university hospital.

METHODS

A quantitative, descriptive and transversal study was carried out, using a convenience sample of an outpatient population, selected according to adherence criteria, without a fixed inclusion order regimen and independent of any interference by the assisting professionals. Data were collected at the Clinical Psychiatric Outpatient Service (SACP) of the University Hospital, Faculty of Medicine of Ribeirão Preto, University of São Paulo.

The participants of the study were patients being assisted in the following Unities: Anxiety Disorder Unity, Schizophrenia and Mood Disorder at SACP. The sample was composed by 127 subjects. The inclusion criteria were as follows: to be older than 18 years, to be attended at SACP after being screened and examined in terms of general psychiatry for a definition of the diagnosis and of the conduct to be followed, to be able to provide reliable information, and to agree to participate in the study. Patients who did not fulfil these criteria or needed urgent intervention were excluded.

The instruments used for the data collection were as follows:

- Structured and standardized interview with the patient in order to obtain sociodemographic information

- CAGE: alcohol screening instrument, used often in research and clinics^(11-12)

- Diagnostic Criteria - ICD-10: structured interview, conforming to the Diagnostic Criteria for Research (DCR-10) of ICD-10, to verify the alcohol use disorders (harmful use and alcohol dependence).

The data were collected by medical students and research collaborators graduated in psychology. All of the interviewers received training based on interview techniques, the use of screening instruments and the assessment to identify alcohol use disorders using ICD-10 diagnosis criteria. They went to the SACP during opening times and at the end of the patient's consultation the resident doctor explained to the patient the objectives of the research and asked them if they wanted to participate. If they agreed, the resident doctor would leave the room and invite the interviewer to go and start the data collection. The interviewers read with the patient the Free and Informed Consent Term, verify if they really wanted to participate and then started the interview for the socio-demographic data collection. After that, they did the application of CAGE and the interview for the diagnose criteria of ICD-10.

The collected information was recorded in Excel - 2002 program. The evaluation of CAGE was done by verifying the sensitivity and specificity of it according to the cut-off point used when compared to the diagnosis of "harmful use" and "alcohol dependence" of ICD-10, used here as golden-standard. The ROC curve was used to analyse the more adequate cut-off point, guaranteeing satisfactory sensitivity and specificity.

This project was approved by the Ethics Committee of the Faculty of Medicine of Ribeirão Preto. Interviews were held after all patients had signed the Free and Informed Consent Term for volunteers patients, respecting the ethics criteria for the resolution n.º 196 from October 10^th 1996. After the interview, all participants received guidelines about low risk drinking and about the situations in which they should avoid the consumption of alcoholic beverages.

RESULTS

Sample characteristics

The sample was composed of a majority of women, which represented 69.3%. Average age was 42.87 years (SD = 13.2; min. = 18 and max. = 77 years). Most participants declared themselves white (74.8%) and 52% lived without a partner (single, divorced or widowed).The majority had received up to 8 years of education (57.5%). Family income for more than half (57.5%) of the interviewees ranged between 1 and 5 minimum wages, and the majority (61.4%) declared that they practiced some religion. Diagnoses included schizophrenia (n=34), bipolar affective disorder (n=30), depressive episode (n=33), phobic anxiety disorders (n=03) and other anxiety disorders (n=27).

Problematic alcohol use

According to the ICD-10, the frequency of harmful use was 6.3% (n=8) and of dependence was 3.9% (n=5), totalling 10.2% (n=13) of alcohol use disorder.

In relation to the score of the sample in CAGE, Figure 01 illustrates the distribution.

Evaluation of CAGE performance among psychiatric patients

The evaluation of CAGE performance was done by using the analyses of the ROC curve, having the clinical diagnose of "harmful use" or "alcohol dependence", from ICD-10, as golden standard. The Figure 02 indicates the values of sensitivity and specificity obtained according to the cut-off point of CAGE.

Using the analysis of the ROC curve, it was verified that the ideal critical point sensitivity/specificity was for the score one at CAGE. Based on these data, it was chosen to use the cut-off point >1. The Figure 03 illustrates the analyses of the ROC curve.

Taking into consideration the cut-off point of >1, the distribution of the positive and negative results in CAGE is shown in Figure 04.

DISCUSSION

The screening instruments for alcohol use are being widely used in research and in the clinic. The use of the CAGE instrument is spreading due to its quick and easy application. However, the literature recommends the adaptation of this instrument for different realities and populations, thus the evaluation of it's performance is becoming important. A study carried out about the performance of five screening instruments for alcohol related problems used with psychiatric patients, including CAGE, suggested that the cut-off points of these instruments were adapted to local standards and to services characteristics⁽¹³⁾.

In the studied sample, it was verified that the best values for sensitivity and specificity for CAGE were found with the cut-off point >1 (Sens = 100%; Spec = 73.7%). When CAGE was translated and validated in Brazil, the cut-off point recommended was >2⁽²⁾. However, in this study it was observed that using this cut-off point the instrument would loose almost 50% of it's sensitivity, not identifying all the cases of patients with harmful use or alcohol dependence. Other authors also found this problem when they compared two groups of patients from a service and they suggested the reduction of the cut-off point for many screening instruments, including CAGE, for women groups⁽⁹⁾. Mostly for white north-american women there is vast literature recommending the use of the cut-off point >1 at CAGE^(14,15). One study that assessed a sample of elderly people also found low sensitivity (48%) with the cut-off point of >2⁽¹⁶⁾. With outpatients from an orthopaedic clinic the cut-off point >1 at CAGE, which showed values of 85% for sensitivity and 89% for specificity, was considered the most adequate⁽¹⁷⁾. A study carried out in Brazil to verify the validity of CAGE among inpatients with alcohol dependence from a clinical nursery also found the values of sensitivity (93.8%) and of specificity (85%) more adequate with the use of the cut-off point >1⁽¹⁸⁾.

It was verified that with the use of CAGE there was a higher inclusion of people as "cases" then with the use of the diagnoses criteria from ICD-10. It is important to highlight that the ICD-10 criteria refers to last year, and CAGE refers to lifetime. This can explain the higher frequency of cases that was found with the use of CAGE. Another possible explanation is that CAGE identified persons with hazardous use of alcohol, but because they did not have any harm, they were not identified by the harmful use criteria from ICD-10. So, CAGE would select persons that are in different points of the continuum of drinking psychopathology, being a reliable clinical mark for any of the following conditions: hazardous use, harmful use and dependence, that had occurred throughout the lifetime.

To help in the identification of problematic use of alcohol among persons with a psychiatric condition, that use outpatient clinics, it is suggested the use of screening instruments. CAGE showed here to be a good instrument for this purpose, being quick, easy to understand and non-intimidating. In this study, it's more adequate cut-off point was >1. To improve the diagnose power of the test and the reliability of the results the evaluation of it's performance in the studied population is recommended.

REFERENCES

Recebido em: 29.8.2005

Aprovado em: 18.11.2005

• 1. World Health Organization. Neuroscience of psychoactive substance use and dependence. Genebra: World Health Organization; 2004. 264p.
• 2. Masur J, Monteiro MG. Validation of the "CAGE" alcoholism screening test in a Brazilian psychiatric inpatient hospital setting. Braz J Med Biol Res 1983; 16(3):215-8.
• 3. Babor TF, Higgins-Biddle JC, Saunders JB, Monteiro MG. AUDIT: teste para identificação de problemas relacionados ao uso de álcool: roteiro para uso em atenção primária. Ribeirão Preto: PAI-PAD; 2003.
• 4. Reinert DF, Allen JP. The Alcohol Use Disorders Identification Test (AUDIT): a review of recent research. Alcohol Clin Exp Res 2002; 26:272-9.
• 5. Sokol RJ, Martier SS, Ager JW. The T-ACE questions: practical prenatal detection of risk-drinking. Am J Obstet Gynecol 1989; 160(4):863-70.
• 6. Fabrri CE. Desenvolvimento e validação de instrumento para rastreamento do uso nocivo de álcool durante a gravidez (T-ACE). [dissertação]. Ribeirão Preto (SP): Faculdade de Medicina de Ribeirão Preto/USP; 2002.
• 7. Aertgeerts B, Buntinx F, Kester A. The value of the CAGE in screening for alcohol abuse and alcohol dependence on general clinical populations: a diagnostic meta-analysis. J Clin Epidemiol 2004; 57:30-9.
• 8. Cherpitel CJ. Gender, injury status and acculturation differences in performance of screening instruments for alcohol problems among US Hispanic emergency department patients. Drug Alcohol Depend 1999; 53:147-57.
• 9. Cherpitel CJ, Borges G. Screening instruments for alcohol problems: a comparison of cut points between Mexican American and Mexican patients in the emergency room. Subst Use Misuse 2000; 35(10):1419-30.
• 10. Organização Mundial da Saúde. Classificação de Transtornos Mentais e de Comportamento da CID-10: critérios diagnósticos para pesquisa. Porto Alegre: Artes Médicas, 1998.
• 11. Paz Filho GJ, Sato LJ, Tuleski MJ, Takata SY, Ranzi CCC, Saruhashi SY, Spadoni B. Emprego do questionário CAGE para detecção de transtornos de uso de álcool em pronto-socorro. Rev Assoc Med Bras 2001; 47(1):65-9.
• 12. Santos MB, Gailit Jr A, Sato MT, Lawryniuk M, Sater JB, Czpak JC. Aplicação do método "CAGE" para estudo da prevalência e detecção precoce do alcoolismo em enfermarias gerais e especializadas de um hospital universitário. Rev Bras Educ Med 1994; 18(1):28-34.
• 13. Watson CG, Detra E, Fox KL, Ewing JW, Gearhart LP, Demotts JR. Comparative concurrent validities of five alcoholism measures in a psychiatric hospital. J Clin Psychol 1995; 51(5):676-84.
• 14. Cherpitel CJ. Analysis of cut points for screening instruments for alcohol problems in the emergency room. J Stud Alcohol 1995; 56(6):695-700.
• 15. Bradley KA, Boyd-Wickiser J, Powell SH, Burman ML. Alcohol screening questionnaires in women: a critical review. JAMA 1998; 280(2):166-71.
• 16. Jones TV, Lindsey BA, Yount P, Soltys R, Farani-Enayat B. Alcoholism screening questionnaires: are they valid in elderly medical outpatients? J Gen Intern Med 1993; 8(12):674-8.
• 17. Bush B, Shaw S, Cleary P, Delbanco TL, Aronson MD. Screening for alcohol abuse using the CAGE questionnaire. Am J Med 1987; 82(2):231-5.
• 18. Castells MA, Furlanetto LM. Validity of the CAGE questionnaire for screening alcohol-dependent inpatients on hospital wards. Rev Bras Psiquiatr 2005; 27(1):54-7.

Publication Dates

• Publication in this collection
  23 Feb 2006
• Date of issue
  Dec 2005

History

• Accepted
  18 Nov 2005
• Received
  29 Aug 2005