Acessibilidade / Reportar erro

Alcohol Use Disorders Identification Test (AUDIT): an updated systematic review of psychometric properties

Abstract

The Alcohol Use Disorders Identification Test (AUDIT) is an instrument used to screen for alcohol-related problems. It has been increasingly used in many different countries in both the original English-language version and its many translated versions. Because of the need for screening instruments of faster administration, shortened versions of the AUDIT have also been developed. This study was aimed at expanding the work by Berner and colleagues (2007) in an attempt to answer some remaining questions as well as to identify and evaluate studies on the validation of modified versions of the AUDIT, which have not been previously analyzed. In order to do so, we identified indexed articles published between 2002 and 2009 related to the psychometric qualities of the AUDIT by matching the keywords: alcohol, Alcohol Use Disorders Identification Test, and AUDIT. We found 47 articles that evaluated the AUDIT in different countries and in diverse health and community contexts, involving adolescent, adult, and elderly samples. The studies confirmed the validity and efficiency of the AUDIT in the identification of harmful use, abuse, and dependence of alcohol, both in the original version and in modified ones. The possibility of using brief and efficient versions is of great value, since certain health contexts demand faster assessment. The results also showed that the reduced versions have satisfactory psychometric qualities, sometimes with sensitivity values higher than those of the AUDIT itself. The studies analyzed confirm the efficiency of the AUDIT both in its original, reduced, and language-adapted versions in different contexts and cultures.

alcohol; Alcohol Use Disorders Identification Test; AUDIT


NEUROPSYCHOPHARMACOLOGY

Alcohol Use Disorders Identification Test (AUDIT): an updated systematic review of psychometric properties

Carolina de Meneses-Gaya; Antonio Waldo Zuardi; Sonia Regina Loureiro; José Alexandre S. Crippa

Department of Neuroscience and Behavior, Faculty of Medicine, University of São Paulo, Ribeirão Preto, São Paulo, Brazil and INCT Translational Medicine, Brazil.

Correspondence regarding this article should be directed to Correspondence regarding this article should be directed to: José Alexandre S. Crippa Departamento de Neurociências e Ciências do Comportamento; Faculdade de Medicina de Ribeirão Preto; Universidade de São Paulo Hospital das Clínicas - Terceiro Andar, Av. Bandeirantes, 3900 - Ribeirão Preto - São Paulo - Brasil - CEP - 14049-900. Phone/fax: (+55)-16-6022201/ (+55)-16-6350713. E-mail: jcrippa@fmrp.usp.br

ABSTRACT

The Alcohol Use Disorders Identification Test (AUDIT) is an instrument used to screen for alcohol-related problems. It has been increasingly used in many different countries in both the original English-language version and its many translated versions. Because of the need for screening instruments of faster administration, shortened versions of the AUDIT have also been developed. This study was aimed at expanding the work by Berner and colleagues (2007) in an attempt to answer some remaining questions as well as to identify and evaluate studies on the validation of modified versions of the AUDIT, which have not been previously analyzed. In order to do so, we identified indexed articles published between 2002 and 2009 related to the psychometric qualities of the AUDIT by matching the keywords: alcohol, Alcohol Use Disorders Identification Test, and AUDIT. We found 47 articles that evaluated the AUDIT in different countries and in diverse health and community contexts, involving adolescent, adult, and elderly samples. The studies confirmed the validity and efficiency of the AUDIT in the identification of harmful use, abuse, and dependence of alcohol, both in the original version and in modified ones. The possibility of using brief and efficient versions is of great value, since certain health contexts demand faster assessment. The results also showed that the reduced versions have satisfactory psychometric qualities, sometimes with sensitivity values higher than those of the AUDIT itself. The studies analyzed confirm the efficiency of the AUDIT both in its original, reduced, and language-adapted versions in different contexts and cultures.

Keywords: alcohol, Alcohol Use Disorders Identification Test, AUDIT.

Introduction

Hazardous alcohol consumption is a major contributor to risky behaviors and adverse health outcomes; hence, the detection of Alcohol Use Disorders (AUD) constitutes a great challenge for public health planners across countries (Gómez, Conde, Santana, & Jorrín, 2005) and screening instruments for alcohol problems have recently become the subject of growing interest.

Screening instruments are highly valuable in that they provide accurate, reproducible, relevant measurements. In addition, their use offers better information records, especially in health care settings with high labor turnover. These instruments also allow for comparisons between similar assessment results and minimize the influence of subjective factors on data collection and recording. Appropriate screening instruments are quick and easy to administer, score, and interpret, in addition to presenting significant reliability and validity coefficients regarding the research objectives.

The Alcohol Use Disorders Identification Test (AUDIT) is a widely used instrument that was developed by the World Health Organization (WHO) for identifying risky or harmful alcohol consumption as well as alcohol dependence and abuse (Babor, La Fuente, Saunders, & Grant, 1992). The 10-item AUDIT includes questions to assess the amount and frequency of alcohol intake (items 1-3), alcohol dependence (questions 4-6) and problems related to alcohol consumption (items 7-10) (Shevlin & Smith, 2007). Scores range from 0 to 40, and the generally accepted cut-off point of the scale to identify potentially hazardous alcohol intake is 8. It is worthwhile highlighting that numerous studies have validated the AUDIT, and reached better sensitivity and specificity values with different cut-off points (Adewuya, 2005; Bradley et al., 2003; Dawson, Grant, & Stinson, 2005a; Dybek et al., 2006; Gache et al., 2005; Knight, Sherritt, Harris, Gates, & Chang, 2003; Pal, Jena, & Yadav, 2004; Pérula et al., 2005). Similarly, it has recently been observed that male and female patients require different cut-off scores (Reinert & Allen, 2002, 2007).

Although the AUDIT was originally designed as an instrument for use in primary care settings, several recent studies have validated it in other health care and community contexts (Lima et al., 2005). The AUDIT has been translated into several languages for use in various countries. Literature contains references to translations in Nigeria (Adewuya, 2005); India (Carey, Carey, & Chandra, 2003; Pal, Jena, & Yadav, 2004); Spain (Gómez et al., 2005; Pérula et al., 2005); Brazil (Lima et al., 2005), Switzerland (Bergman & Källmén 2002; Selin, 2003); China (Chen, Chen, & Cheng, 2004, 2005; Tsai, Tsai, Chen, & Liu, 2005); Germany (Bischof et al., 2005; Dybek et al., 2006; Neumann et al., 2009; Rumpf et al., 2003); Vietnam (Giang, Spak, Dzung, & Allebeck, 2005) and France (Gache et al., 2005). Nonetheless, only a small number of studies have addressed the validity, reliability and factorial structure of these non-English versions. This fact can compromise the comparability across studies.

In view of the need for screening instruments that are faster to apply, abbreviated versions of the AUDIT have been developed. This was possible thanks to the AUDIT's high internal consistency, which allowed shorter versions to be just as efficient as the full-scale version. The reduced versions known to date are the AUDIT-3 – the shortest of all, consisting solely of the third question of the AUDIT (Bradley et al., 2003; Gómez et al., 2005; Wu et al., 2008) -; AUDIT-C, consisting of the first three questions of the full-scale (Aertgeerts, Buntinx, Ansoms, & Fevery, 2002; Bradley et al., 2003; Dawson et al., 2005a; Gómez et al., 2005; Tsai et al., 2005; Bradley et al., 2007; Bischof et al., 2007; Frank et al., 2008); AUDIT QF, comprising the first two questions (Aalto, Tuunanen, Sillanaukee, & Seppa, 2006); AUDIT-PC, with questions one, two, four, five, and 10 (Aertgeerts et al., 2002; Gómez et al., 2005; Aalto et al., 2006; Wu et al., 2008); AUDIT 4, consisting of the first three questions plus question number 10 (Gual, Segura, Contel, Heather, & Colom, 2006; Wu et al., 2008); FAST, which includes questions three, five, eight, and 10 (Hodgson, Alwyn, John, Thom, & Smith, 2002), and the Five-Shot, which integrates three questions of the CAGE with two of the AUDIT (Aertgeerts et al., 2002; Aalto et al., 2006). Most validation studies on these versions focus mainly on the AUDIT-3 and AUDIT-C; with few studies having evaluated the AUDIT-PC, the Five-Shot and the FAST.

The present study aimed to identify, by means of searching in the indexed literature, studies on the psychometric properties of the AUDIT published over the last eight years, in order to analyze validity and reliability characteristics. Another objective was to expand the study by Berner and colleagues (2007) in an attempt to answer some remaining questions and to identify studies on the translation and validation of modified non-English versions of the AUDIT, which have not been previously analyzed.

Method

A systematic search of the literature published up to January 2009 was carried out using the following electronic databases: Medline; LILACS, PsycINFO, Science Citation Index Expanded, BIOSIS Previews, Cumulative Index to Nursing and Allied Health Literature (CINAHL), MEDION, Scopus, and SciELO. The following search terms were used: “alcohol”, “Alcohol Use Disorders Identification Test”, and “AUDIT”.

In the first step, one reviewer assessed the title and abstract of all publications that were retrieved in electronic database searches using the inclusion and exclusion criteria. We included articles published in English, Spanish and Portuguese and in which the psycho¬metric properties of the AUDIT and its abbreviated versions were evaluated. Studies dealing with the clinical and epidemiolo¬gical aspects of alcohol problems were excluded, as well as studies on the therapeutic efficacy to treat dependence and articles published before 2002. The original articles obtained through this search were then reviewed for additional references.

The searches retrieved 807 potentially relevant publications; 760 publications were excluded and 47 studies were analyzed. Table 1 lists the sociodemographic variables of the studies analyzed in this review.

Results

Characteristics of the AUDIT in specific groups

The AUDIT was originally developed to be used with adult populations. However, AUDIT validation studies have also been performed in samples composed of adolescents and university students (Reinert & Allen, 2002).

Over the last years, two studies that validated and compared the AUDIT to other screening instruments, in adolescents and in different contexts, were identified. One of these studies compares the ability of the AUDIT to identify AUD with the Substance Abuse Screening Test among Adolescent Clinic Patients (CRAFFT) and the CAGE, in a sample of adolescent patients from a Sexually Transmitted Diseases Clinic (Cook, Chung, Kelly, & Clark, 2005). The study evaluated these tests' performance in terms of age, gender and ethnicity, which has been little explored. The authors did not observe any significant differences regarding gender, but they did find a significant difference for ethnicity (p < .01), with the prevalence of AUD being higher among white individuals as compared to black populations. No significant differences were found in the other instruments in relation to this variable. At a cut-off point of nine, the AUDIT yielded sensitivity of .76 and specificity of .79. The AUDIT had the best results, with an area under the receiver operating characteristic curve (AUC) of .84, followed by the CRAFFT (.79) and the CAGE (.70).

Different results were found in the study comparing the AUDIT with the Problem Oriented Screening Instrument for Teenagers (POSIT), the CAGE, and the CRAFFT in a sample of adolescents aged 14 to 18 arriving for routine healthcare at a large, hospital-based adolescent clinic. In the study, the AUDIT appeared to be almost insensitive at a cut-off point above eight; authors consider two as the cut-off point for problematic use, and three or more for misuse and dependence. Because these cut-off points are much lower than those usually used, these differences were associated with age differences in the sample (14 to 18), context, or AUD prevalence (8%). Although sensitivity does not typically vary with the prevalence of the disorder, some studies revealed that large differences in frequency may clarify some of the differences found in the screening instruments (Knight et al., 2003).

Both studies recommend the use of the AUDIT, but raise objections in terms of administration time and the fact that it does not screen for other drugs, which are usually associated with alcohol consumption. Taking these factors into consideration, the authors indicated the CRAFT as the best screening instrument, due to its faster administration, high sensitivity and specificity, and to the fact that it screens for other drugs in addition to alcohol. The CAGE was not considered appropriate for this group, since it yielded low sensitivity.

Some studies that investigated the validation of the AUDIT in samples of university students identified that lower sensitivity and specificity values are obtained when the traditional cut-off point is used.

In this review, two studies on the validation of the AUDIT in university students were found. The first study compared the AUDIT with the Alcohol Timeline Followback (TLFB), which measures the amount and frequency of alcohol consumption in the previous 28 days, according to DSM-III-R criteria. The AUDIT showed greater capability to detect hazardous alcohol use in the past 28 days, with an AUC of .87. Internal consistency (Chronbach's alpha) was .81. The study found a cut-off point of six or more to identify AUD (Kokotailo et al., 2004). The second study, performed in Nigeria, confirmed the AUDIT as a valid screening instrument for alcohol-related problems among university students (Adewuya, 2005). The authors recommend a cut-off point of five to screen for hazardous use and a cut-off point above seven for misuse and dependence. The area under the receiver operating characteristic curve was above .93.

The AUDIT revealed high performance in these studies, confirming its validity as an AUD screening instrument among university students. We did not locate any studies that evaluated the psychometric properties of the instrument with a cut-off point of eight. Hence, according to the values found in previous studies, results establish a cut-off point of five or six as the most appropriate.

Several studies prove the significant increase in alcohol-related problems among the elderly, which are frequently neglected in health care units (Philpot et al., 2003). Thus, the need for new AUDIT evaluation research in this group is highlighted, due to the low sensitivity values obtained and the need to verify a more suitable cut-off point, since alcohol tolerance tends to be reduced in older individuals (Reinert & Allen, 2007).

Only one study on the validation of the AUDIT in elderly patients was identified. The study was conducted with a sample of psychiatric patients, and aimed to evaluate and compare the performances of the AUDIT, AUDIT-5, and CAGE. The patients' clinical data were used as diagnostic criteria to evaluate alcohol consumption (Philpot et al., 2003). The area under the curve was .96 for the AUDIT; 0.96 for the AUDIT-5; and .78 for the CAGE. The AUDIT-5 had the best performance, with sensitivity of .75; specificity of .97, and positive predictive value of .83 for a cut-off point of four/five. The AUDIT showed sensitivity of .67; specificity of .96; and positive predictive value of .75 for the cut-off point of seven/eight. The CAGE, however, appeared to be inefficient for the screening of AUD in elderly patients that seek psychiatric health care. It must be highlighted that the tests were more specific and less sensitive, and that the abbreviated version (AUDIT-5) showed better performance than the AUDIT. Nonetheless, these data require further investigation, since there is a need for new research with elderly populations.

It is known that AUD are highly comorbid with other types of psychopathology, thus, practitioners need to be able to identify hazardous drinking as well as alcohol abuse and dependence in psychiatric patients. Cassidy, Schmitz, and Malla (2008) assessed the validity and reliability of the Alcohol Use Disorders Identification Test (AUDIT) and the Drug Abuse Screening Test (DAST) for detecting alcohol and drug use disorders, respectively, in a population with first-episode psychosis (FEP). The DAST and AUDIT can reliably identify FEP patients with substance abuse, the AUDIT with cut-off score of 10 had a sensitivity value of 0.85; specificity of .91 and an AUC of .86.

Health care professionals seem to be less likely to identify women with alcohol problems. One of the explanations found for this fact is that alcohol disorders are less prevalent among women, although this prevalence has increased (Bradley et al., 2003).

Studies point out the need for different cut-off points for men and women, since sensitivity is reduced when the standard AUDIT cut-off point is used for females. Thus, a lower cut-off point (five or six) may be more appropriate for women (Reinert & Allen, 2002, 2007).

In this sense, the validity of the AUDIT and the most adequate cut-off point were assessed in a significant sample of female patients (Pérula et al., 2005). The highest sensitivity and specificity values were found for the cut-off point of six, according to DSM-IV and ICD-10 criteria. Internal consistency was 0.93 (Chronbach's alpha). The psychometric properties of the AUDIT confirm its efficiency in screening for alcohol-related problems in women.

Another study involving exclusively women evaluated and compared the performance of the AUDIT with that of the AUDIT-3 and AUDIT-C. The authors also analyzed modified versions of the tests, in which item three was changed (amount of drinks was changed to four or five on one occasion) (Bradley et al., 2003). The recommended cut-off point for the AUDIT was lower than that found in previous studies. The AUDIT-C proved more efficient than the AUDIT, and the modified versions showed better results than their original counterparts.

A study involving 894 middle-aged women ( > 40 years) investigated the performance of the AUDIT, AUDIT-C, AUDIT-PC, AUDIT-QF, and of the Five-Shot (Aalto et al., 2006). In this report, the reduced versions once again proved to be as efficient as the full AUDIT.

The aforementioned data are evidence that further research is needed to verify differences in terms of age, gender and ethnicity, with the aim of establishing appropriate cut-off points for these groups.

Comparability of the AUDIT with other alcohol screening methods

It is known that patients admitted to general hospitals with acute diseases often report alcohol consumption. For alcohol-dependent individuals, hospitalization means a period of forced abstinence, many times causing clinical withdrawal symptoms (Dolman & Hawkes, 2005). Furthermore, it is worth mentioning that these symptoms can jeopardize treatment and cause additional harm to patients' health. Hence, early detection of alcohol use is essential for appropriate intervention before the onset of further complications.

Three studies have evaluated the ability of the AUDIT and biomarkers to jointly detect problems related to alcohol consumption. The first investigated the capability of the AUDIT and four other traditional biomarkers: Glutamyltransferase (GGT), Alanine Aminotransferase (ALT), Aspartate Aminotransferase (AST), and Mean Corpuscular Volume (MCV) in identifying patients at risk for alcohol abstinence symptoms. In the study, patients who scored eight or more on the AUDIT were monitored and treated. Of the 98 patients screened with the AUDIT, 17 experienced significant alcohol withdrawal symptoms, whereas none of the patients with scores lower than eight presented abstinence symptoms. It must be emphasized that at the cut-off point of 13, sensitivity was not reduced and the positive predictive value increased significantly. When combined, the biomarkers increased the positive predictive values even more, and improved screening for patients at risk for abstinence symptoms. The combination of AST and GGT biomarkers yielded the best results (Dolman & Hawkes, 2005).

These results differ from those by Neumann et al. (2009), who also evaluated whether the accuracy of screening is enhanced by combined use of the AUDIT and biomarkers of alcohol use. In this study, the biomarkers (GGT, MVC, % CDT) showed low sensitivity ( < 0.43), whereas the sensitivity for the AUDIT was .76 for males and .81 for females. The addition of biomarkers added little discriminatory information compared to the use of the AUDIT alone. Aertgeerts et al. (2002), evaluated laboratory tests (GGT, MVC, ASAT, ALAT) along with the CAGE, the AUDIT, and the abbreviated versions of the latter (AUDIT-C, AUDIT-PC, and Five-Shot) in a sample of male patients. The authors considered the laboratory tests insufficient for the screening of AUD, with sensitivity between .1 and .52 In this study, at the cut-off point of eight, the AUDIT showed a low sensitivity (65.5%), despite having been efficient at cut-off points higher than five. Among the abbreviated versions, only the Five-Shot had reasonable diagnostic parameters to be recommended as a screening instrument (Aertgeerts et al., 2002). These differences suggest the importance of performing new studies combining the AUDIT with biomarkers to detect AUD.

It is worth emphasizing that many studies reported that abbreviated versions of the AUDIT yielded results that were similar or even superior to those of the full-scale version and other screening instruments. These results were observed in a study performed in Poland, which analyzed and compared the performance of the Rapid Alcohol Problems Screen (RAPS4), the CAGE, and the AUDIT to identify alcohol dependence, misuse, and harmful use in a sample of emergency room patients (Cherpitel, Ye, Moskalewicz, & Swiatkiewicz, 2005). In comparison with the CAGE, the sensitivity of the RAPS4 and the AUDIT was significantly higher, but specificity was lower for men. A small difference was observed between the sensitivity of the RAPS4 and the AUDIT when applied among men. However, the RAPS4-QF showed significantly higher sensitivity among women. On the other hand, at the cut-off point of three, the performance of the AUDIT among women was similar to that of the RAPS4-QF. Including questions about quantity and frequency in the RAPS4 yielded a significant increase in the area under the curve, although specificity was significantly affected among men. These data also suggest that the instruments perform differently according to gender.

Another validation study performed in India compared the AUDIT with the Short Michigan Alcoholism Screening Test (SMAST). Both tests presented good psychometric properties, and the AUDIT had a very high internal consistency (Chronbach's alpha of 0.92). The AUDIT cut-off points for hazardous use and dependence had higher values than those traditionally established, i.e. 16 and 24, respectively (Pal et al., 2004).

Table 2 lists the main psychometric properties of the AUDIT, its abbreviated versions and other screening instruments in the reviewed studies.

Reliability of the AUDIT

Several studies have evaluated the test-retest reliability of the AUDIT. However, comparability of the results is usually compromised, since the studies use different intervals. Nonetheless, three studies that evaluate test-retest reliability with the same interval (one month) have been identified. In the first study, Selin et al. (2003), verified that the correlation between the responses in the first and second application was somewhere between 0.6 and 0.8; in other words, between good and excellent, except for item nine (“Have you or someone else been injured as a result of your drinking?”), with a correlation of .29. Total score test-retest reliability was .84. Evaluations regarding gender, age, and consumption levels showed a correlation of .80, except among low consumers, who presented a correlation of .5. Kim, Gulick, Nam, and Kim (2008), in their study with Korean Americans, found a similar correlation coefficient for the full AUDIT, of .85.

Another research measured the test-retest reliability of the AUDIT (with a 30-day interval) in a sample of 99 patients. The authors also evaluated the test validity in 10,803 subjects and obtained excellent sensitivity and specificity at the cut-off point of five or higher. Once again, item nine yielded the lowest correlation (.39), and item 10 the highest (.98), with a total score correlation of .95. At the cut-off point of eight or higher, 87.5% of subjects screened in the first test were also classified as positive in the retest, and 98.9% of those who scored below eight in the first test were equally evaluated as negative in the second administration. With a five-point score, 88.9% of the values were correct for positive classifications, and 95.1% for the negative evaluation. These results indicate high reliability at both cut-off points used in this study (Dybek et al., 2006).

Ten studies that evaluated the internal consistency of the AUDIT were identified. In these studies, the mean value of Chronbach's alpha was 80, indicating high internal consistency.

Although the AUDIT was designed to be applied by raters and not as a self-evaluation instrument, no studies on inter-rater reliability were found.

Construct Validity

The factor structure of the AUDIT was examined in several studies. Although some studies regarded the instrument as having a single dimension, there has been greater research support for a two-factor model.

The one-dimensional structure of the questionnaire was supported by Carey et al. (2003) in an investigation of the factor structure of the AUDIT using exploratory and confirmatory factor analysis, in a sample of psychiatric patients in India (n = 671). Out of these patients, 27% were diagnosed with AUD according to ICD-10 criteria. In this study the AUDIT showed a high internal consistency (alpha of 0.94).

Seven studies examined the factor structure of the AUDIT using principal components as well as exploratory and confirmatory factor analysis, and revealed that a two-factor solution is preferable - a consumption factor (items 1–3) and an adverse consequences of drinking factor (items 4–10) (Bergman et al., 2002; Carey et al., 2003; Doyle, Donovan, & Kivlahan, 2007; Lima et al., 2005; Rist, Glockner-Rist, & Demmel, 2009; Shevlin et al., 2007; Von-der-Pahlen et al., 2008).

It is known that the factor analysis of the AUDIT yields different results depending on the sample involved. In populations with a high prevalence of alcohol dependence, the factor analysis usually results in a single factor. On the other hand, in samples with a low prevalence of AUD, two factors are identified: one regarding consumption, and the other concerning problems and consequences of alcohol use (Lima et al., 2005).

Abbreviated versions of the AUDIT

Over the last years, several studies have been published on the validation of abbreviated versions of the AUDIT in different contexts and groups. These versions are of great value, since they allow for faster screening of AUD, particularly in busy medical settings.

The AUDIT-C was evaluated in a significant sample (n = 43,093) in the National Epidemiologic Survey on Alcohol and Related Conditions (NESARC). This abbreviated version yielded better results for screening dependence in women; whereas screening was more efficient for hazardous use in the male sample (table 2). The authors reported that different cut-off points for men (five or higher for hazardous drinking) and women (three or higher) improve the test's efficiency. The AUDIT-C showed high sensitivity and specificity in screening for alcohol-related problems (Dawson et al., 2005b).

Another research was performed in this same group, evaluating the performance of the AUDIT-C in psychiatric patients (n = 10,380) divided into groups with mood, anxiety, and personality disorders (Dawson et al., 2005a). No significant differences in performance were found in relation to the three groups. The AUDIT-C also revealed appropriate psychometric properties. Hence, it was also indicated for use in patients with psychiatric disorders. It must be taken into consideration that the AUDIT-C was integrated into a larger sequence of questions relating to alcohol consumption included in the NESARC. This sequence may affect the performance of the AUDIT-Cas compared to its use as a stand-alone instrument.

The AUDIT-C with a cut-off score of three, showed sensitivity and specificity of .90 and 91.5% of participants were correctly classified using the AUDIT full as a comparative measure in a sample of female detainees (Caviness & Hatgis, 2009). The effectiveness of the AUDIT-C as a screening test for alcohol was also evaluated among primary care patients from the predominant racial/ethnic subgroups (White, African American, and Hispanic). The AUDIT-C was excellent in all three groups as reflected by high AUCs (.85 in all groups). At the recommended cut-off points (three) there were significant differences in the AUDIT-C's sensitivity with values between .67 - .95 (Frank et al., 2008).

The FAST, another abbreviated version, was evaluated using the AUDIT as the gold standard. The objective of this study was two-fold. The first objective was to analyze the possibility of using each question of the AUDIT as a sequential filter. The second was to examine if one question could rapidly screen for a large percentage of the group. The authors observed that item three identified 66% of the patients as having alcohol problems and items five, eight, and 10 completed the screening process. The combination of these questions yielded sensitivity of 91% and specificity of 93% for AUD. These results demonstrated the efficiency of the FAST in screening for subjects with alcohol-related problems in a quick and objective way (Hodgson et al., 2002).

The same authors (Hodgson et al., 2003) performed another study on the FAST, which investigated the sensitivity and specificity of the instrument in terms of age and gender. Moreover, they compared the performance of the FAST with two other abbreviated tests: the Paddington Alcohol Test (PAT) and the CAGE. The FAST showed better results than the other two. Once more, the CAGE proved to have poor sensitivity.

Another study evaluated the psychometric properties of the abbreviated versions AUDIT-3, AUDIT-C, AUDIT-PC, and m-FAST (modified FAST) against the AUDIT. The authors verified that all the abbreviated versions, except for the m-FAST, presented significantly higher sensitivity than that of the AUDIT itself. Nonetheless, the abbreviated versions showed lower specificity compared to the AUDIT, with this difference being non-significant only for the m-FAST. The highest internal consistency was obtained with the AUDIT-C (alpha of .84), followed by the AUDIT (alpha of .81), AUDIT-PC (.72), and m-FAST (0.63). It is worth mentioning that the subjects responded to a single test that grouped all versions, and, therefore, reproducibility in stand-alone administrations of the versions is not possible (Gómez et al., 2005).

Two screening instruments were developed by gathering the two first questions of the AUDIT and some others from different instruments. One of these instruments, the Five-Shot, integrates three items of the CAGE with those two of the AUDIT (Aalto et al., 2006; Aertgeerts et al., 2002). The Fast Alcohol Consumption Evaluation (FACE), validated in France, groups the two questions of the AUDIT, two of the CAGE, and one of the TWEAK (Tolerance, Worry, Eye-opener, Amnesia, Cut-down) (Dewost, Michaud, Arfaoui, Gache, & Lancrenon, 2006). In these studies the instruments proved as efficient as the AUDIT in the detection of AUD.

The results found in these studies show that abbreviated versions have satisfactory psychometric properties. Moreover, they sometimes have higher sensitivity than the AUDIT itself. Table 3 lists all of these brief AUDIT versions.

Psychometric properties of non-English versions

Over the years, the AUDIT has been widely used in various countries, in the original English version as well as in foreign language versions. Yet, some studies have reported that the latter presented unsatisfactory results, which indicated the need for new investigations (Reinert & Allen, 2002, 2007).

In China, a study was performed on the adaptation and validation of the AUDIT and the AUDIT-C. Both tests presented satisfactory psychometric qualities in screening for dependence and harmful alcohol use, with internal consistency of 0.86 for the AUDIT and .91 for the AUDIT-C. Men showed higher scores in both the AUDIT and the AUDIT-C, which gave evidence of a higher probability of diagnosing alcohol problems (Tsai et al., 2005).

The French version of the AUDIT was validated by Gache et al. (2005) with a sample of patients from a primary health care unit. In this study, the AUDIT proved more efficient in detecting alcohol misuse and dependence, compared to the MAST and the CAGE. The AUDIT also showed high internal consistency (Chronbach's alpha of .87) and most questions showed a correlation coefficient of 0.6. In the screening for dependence, with a cut-off point of 13 or higher, it also showed higher sensitivity in the female sample. Thus, by establishing different cut-off points, higher sensitivity was obtained for both male and female samples. Due to its satisfactory results, the French version of the AUDIT was indicated as an appropriate screening instrument for individuals with AUD (Gache et al., 2005).

The AUDIT was also validated in a rural community in North Vietnam, with a sample composed exclusively of male subjects. In the screening process for AUD, following ICD-10 and DSM-IV criteria, and at the cut-off point of seven/eight, the authors identified different sensitivity and specificity values, in that the AUDIT presented better results in the process of screening for dependence than for alcohol misuse (Giang et al., 2005).

Finally, it is worth highlighting that all these studies confirm the efficiency of the AUDIT in screening for alcohol dependence, misuse, and harmful use, both in its original version and in its validated new versions.

Tables 2 and 3 list the psychometric properties of these versions as well as of versions from Nigeria (Adewuya, 2005), Germany (Bischof et al., 2005; Dybek et al., 2006), Belgium (Aertgeerts et al., 2002), India (Carey et al., 2003; Pal et al., 2004), Spain (Gómez et al., 2005; Pérula et al., 2005), Brazil (Lima et al., 2005), and Switzerland (Selin, 2003).

Forms of administration and evaluation of the AUDIT

It is known that the sequence of questions in a questionnaire may affect the results, since the first items could influence the responses to the subsequent questions. Such effects may be classified as assimilation effects or contrast effects. Assimilation effects occur when preceding questions influence the answers to postponed questions in the same direction, while contrast effects occur when postponed questions are influenced in the opposite direction (Bischof et al., 2005). Thus, introductory questions can modify the sensitivity of a screening questionnaire. This is especially true for the AUDIT, since it is an instrument that evaluates alcohol use.

Therefore, this study aimed to verify if the three first questions in the AUDIT inhibited positive responses to posterior questions, thus reducing its sensitivity (Bischof et al., 2005). Therefore, two different versions of the instrument were administered: the original version (AUDIT-1) and a modified version, in which the first three questions - concerning frequency and amount of alcohol use - were placed at the end of the form (AUDIT-2). Bischof et al., (2005) verified that both versions presented satisfactory internal consistency (Chronbach's alpha of 0.81), suggesting that the order of questions in the AUDIT does not influence the results.

However, the results revealed a small but significant effect in relation to context. The AUDIT-1 yielded higher scores in questions concerning consumption, whereas the AUDIT-2 showed higher values in questions focusing on alcohol dependence or misuse. Nonetheless, the total score of the instrument was not affected by the order of the questions (Bischof et al., 2005).

A two-phase study on the validation of the AUDIT used, for the first time, the Stratum-Specific Likelihood Ratios Analysis (SSLR) and the conventional ROC Analysis. The AUDIT presented excellent discriminatory validity with both methods (AUC of .98). For the SSLR analysis, the total score of responses to the AUDIT was divided in three: (i) zero to seven; (ii) eight to 13; (iii) equal to or above 14. Patients who scored zero to seven presented SSLR significantly below 0.1; which indicates very low probability (0.01) for AUD. When the patients' score was 14 or higher, SSLR was significant above 10, pointing to a high probability (.91) of problems related to alcohol use in these subjects. Consequently, participants in both groups were easily evaluated in terms of the presence of alcohol use disorders.

Based on results from this study, the SSLR was recommended in the AUDIT validation, since it stores more information and prevalence rates do not interfere in the results (Chen et al., 2005).

Conclusion

Over the last few years, an increase in alcohol consumption has been observed in many different populations (Giang et al., 2005). Consequently, the use of appropriate screening instruments for each population is essential to prevent, identify, and offer early treatment to alcohol-related problems.

Many studies have evaluated the validity of the AUDIT using samples composed of teenagers, university students, women, and elderly individuals. In these studies, the AUDIT presented satisfactory results, confirming the efficiency of the instrument in screening for AUD. However, it is necessary to evaluate the most appropriate cut-off scores for each population, since different sensitivity and specificity values are observed when the traditional cut-off point is adopted. In addition, it has been suggested that a modified version of the instrument be used for women, since this version presented better results than the original version.

The authors identified many studies that evaluated and compared the psychometric qualities of the AUDIT to those of eight other instruments. The AUDIT presented excellent sensitivity and specificity values, which were often superior to those of the other tests. It must be taken into consideration that the CAGE, among all the analyzed instruments, presented the worst results, although it continues to be widely used over the world due to its simple administration.

Studies that performed the factorial analysis of the AUDIT provide evidence that scores derived from the AUDIT are best explained in terms of two correlated dimensions. It would appear timely to explore the possibilities of scoring and interpreting the scale in a manner that is consistent with such findings.

An appropriate screening instrument should have significant reliability coefficients, since these values indicate whether an instrument provides reproducible measures. The test-retest reliability of the AUDIT has been evaluated in many studies, and results show there are high rates of reproducibility. However, the comparability of the results is compromised since different intervals were adopted between administrations.

No other interrater reliability study was found, despite the AUDIT being extensively used in a hetero-administered way. Hence, further research is needed to evaluate the reliability of the instrument between different raters.

Over the last years, many studies on the validation of abbreviated versions of the AUDIT have been identified. These versions are extremely helpful, since they allow for faster screening for AUD. The results show that these versions have satisfactory psychometric properties, sometimes with sensitivity values above those of the AUDIT. It should be considered that in many studies subjects completed only one test that combined all the versions, which makes its reproducibility impossible in the administration of independent versions.

Finally, it is worth emphasizing that the results presented in this review confirm the efficiency of the AUDIT in screening harmful use, misuse, and addiction to alcohol. Such effectiveness was confirmed for the original version as well as for abbreviate versions and versions adapted to other languages and in different settings and cultures.

Acknowledgements

C.M.G. is recipient of a Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP, Brazil) fellowship. A.W.Z.; S.R.L. and J.A.C. are recipients of Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq, Brazil) Productivity Awards. Research was supported in part by the Fundação de Apoio ao Ensino, Pesquisa e Assistência do Hospital das Clínicas da Faculdade de Medicina de Ribeirão Preto da Universidade de São Paulo (FAEPA, Brazil).

Received 4 November 2008

received in revised form 20 April 2009

accepted 22 June 2009

Available online 29 June 2009

  • Aalto, M., Tuunanen, M., Sillanaukee, P., & Seppa, K. (2006). Effectiveness of structured questionnaires for screening heavy drinking in middle-aged women. Alcoholism: Clinical and Experimental Research, 30, 1884-1888.
  • Adewuya, A.O. (2005). Validation of the alcohol use disorders identification test (audit) as a screening tool for alcohol-related problems among Nigerian university students. Alcohol and Alcoholism, 40, 575-577.
  • Aertgeerts, B., Buntinx, F., Ansoms, S., & Fevery, J. (2002). Questionnaires are better than laboratory tests to screen for current alcohol abuse or dependence in a male inpatient population. Acta Clinica Belgica, 57, 241-249.
  • Babor, T.F., La Fuente, J.R., Saunders, J., & Grant, M. (1992). AUDIT, the Alcohol Use Disorders Identification Test: guidelines for use in primary health care. Geneva: Substance Abuse Department, World Health Organization, WHO/PSA, 4, 1-29.
  • Bergman, H., & Källmén, H. (2002). Alcohol use among Swedes and psychometric evaluation of the Alcohol Use Disorders Identification Test. Alcohol and Alcoholism, 37, 245-251.
  • Berner, M.M., Kriston, L., Bentele, M., & Härter, M. (2007). The alcohol use disorders identification test for detecting at-risk drinking: A systematic review and meta-analysis. Journal of Studies on Alcohol and Drugs, 68, 461-473.
  • Bischof, G., Reinhardt, S., Grothues, J., Dybek, I., Meyer, C., Hapke, U., John, U., & Rumpf, H.J. (2005). Effects of item sequence on the performance of the AUDIT in general practices. Drug and Alcohol Dependence, 79, 373-377.
  • Bischof, G., Grothues, J., Reinhardt, S., John, U., Meyer, C., Ulbricht, S., & Rumpf, H.J. (2007). Alcohol Screening in General Practices Using the AUDIT: How Many Response Categories Are Necessary? European Addiction Research, 13, 25-30.
  • Bradley, K.A., Bush, K.R., Epler, A.J., Dobie, D.J., Davis, T.M.; Sporleder, J.L., Maynard, C., Burman, M.L., & Kivlahan, D.R. (2003). Two brief alcohol-screening tests from the Alcohol Use Disorders Identification Test (AUDIT): Validation in a female Veterans Affairs patient population. Archives of Internal Medicine, 163, 821-829.
  • Bradley, K.A., DeBenedetti, A.F., Volk, R.J., Williams, E.C., Frank, D., & Kivlahan, D.R. (2007). AUDIT-C as a Brief Screen for Alcohol Misuse in Primary Care. Alcoholism: Clinical and Experimental Research, 31, 1208-1217.
  • Carey, K.B., Carey, M.P., & Chandra, P.S. (2003). Psychometric evaluation of the alcohol use disorders identification test and short drug abuse screening test with psychiatric patients in India. Journal of Clinical Psychiatry, 64, 767-774.
  • Cassidy, C.M., Schmitz, N., & Malla, A. (2008). Validation of the alcohol use disorders identification test and the drug abuse screening test in first episode psychosis. Canadian Journal of Psychiatry, 53, 26-33.
  • Caviness, C.M., Hatgis, C., Anderson, B.J., Rosengard, C., Kiene, S.M., Friedmann, P.D., & Stein, M.D. (2009). Three brief alcohol screens for detecting hazardous drinking in incarcerated women. Journal of Studies on Alcohol and Drugs, 70, 50-54.
  • Chen, C.H., Chen ,W.J., & Cheng, A.T. (2004). A Prevalence and identification of alcohol use disorders among non-psychiatric inpatients in one general hospital. General Hospital Psychiatry, 26, 219-225.
  • Chen, C.H., Chen, W.J., & Cheng, A.T. (2005). New approach to the validity of the alcohol use disorders identification test: Stratum-specific likelihood ratios analysis. Alcoholism: Clinical and Experimental Research, 29, 602-608.
  • Cherpitel, C.J., Ye, Y., Moskalewicz, J., & Swiatkiewicz, G. (2005). Screening for alcohol problems in two emergency service samples in Poland: Comparison of the RAPS4, CAGE and AUDIT. Drug and Alcohol Dependence, 80, 201-207.
  • Cook, R.L., Chung, T., Kelly, T.M., & Clark, D.B. (2005). Alcohol screening in young persons attending a sexually transmitted disease clinic. Comparison of AUDIT, CRAFFT, and CAGE instruments. Journal of General Internal Medicine, 20, 96-97.
  • Dawson, D.A., Grant, B.F., & Stinson, F.S. (2005a). The AUDIT-C: screening for alcohol use disorders and risk drinking in the presence of other psychiatric disorders. Comprehensive Psychiatry, 46, 405-416.
  • Dawson, D.A., Grant, B.F., Stinson, F.S., & Zhou, Y. (2005b). Effectiveness of the derived Alcohol Use Disorders Identification Test (AUDIT-C) in screening for alcohol use disorders and risk drinking in the US general population. Alcoholism: Clinical and Experimental Research, 29, 844-854.
  • Dewost, A., Michaud, P., Arfaoui, S., Gache, P., & Lancrenon, S. (2006). Fast alcohol consumption evaluation: A screening instrument adapted for French general practitioners. Alcoholism: Clinical and Experimental Research, 30, 1889-1895.
  • Dolman, J.M., & Hawkes, N.D. (2005). Combining the AUDIT questionnaire and biochemical markers to assess alcohol use and risk of alcohol withdrawal in medical inpatients. Alcohol and Alcoholism, 40, 515-519.
  • Doyle, S.R., Donovan, D.M., & Kivlahan, D.R. (2007). The factor structure of the Alcohol Use Disorders Identification Test (AUDIT). Journal of Studies on Alcohol and Drugs, 68, 474-479.
  • Dybek, I., Bischof, G., Grothues, J., Reinhardt, S., Meyer, C., Hapke, U., John, U., Broocks, A., Hohagen, C., & Rumpf, H.J. (2006). The Reliability and Validity of the Alcohol Use Disorders Identification Test (AUDIT) in a German General Practice Population Sample. Journal of Studies on Alcohol, 67, 473-481.
  • Frank, D., DeBenedetti, A.F., Volk, R.J., Williams, E.C., Kivlahan, D.R., & Bradley, K.A. (2008). Effectiveness of the AUDIT-C as a screening test for alcohol misuse in three race/ethnic groups. Journal of General Internal Medicine, 23, 781-787.
  • Gache, P., Michaud, P., Landry, U., Accietto, C., Arfaoui, S., Wenger, O., & Daeppen, J.B. (2005). The Alcohol Use Disorders Identification Test (AUDIT) as a Screening Tool for Excessive Drinking in Primary Care: Reliability and Validity of a French Version. Alcoholism: Clinical and Experimental Research, 29, 2001-2007.
  • Giang, K.B., Spak, F., Dzung, T.V., & Allebeck, P. (2005). The use of audit to assess level of alcohol problems in rural Vietnam. Alcohol and Alcoholism, 40, 578-583.
  • Gómez, A., Conde, A., Santana, J.M., & Jorrín, A. (2005). Diagnostic usefulness of brief versions of Alcohol Use Disorders Identification Test (AUDIT) for detecting hazardous drinkers in primary care settings. Journal of Studies on Alcohol, 66, 305-308.
  • Gómez, A., Conde, A., Santana, J.M., Jorrín, A., Serrano, I.M., & Medina, R. (2006). The diagnostic usefulness of AUDIT and AUDIT-C for detecting hazardous drinkers in the elderly. Aging Mental Health, 10, 598-561.
  • Gual, A., Segura, L., Contel, M., Heather, N., & Colom, J. (2002). Audit-3 and Audit-4: Effectiveness of two short forms of the alcohol use disorders identification. Alcohol and Alcoholism, 37, 591-596.
  • Hodgson, R., Alwyn, T., John, B., Thom, B., & Smith, A. (2002). The FAST Alcohol Screening Test. Alcohol and Alcoholism, 37, 61-66.
  • Hodgson, R.J., John, B., Abbasi, T., Hodgson, R.C., Waller, S., Thom, B., & Newcombe, R.G. (2003). Fast screening for alcohol misuse. Addictive Behaviors, 28, 1453-1463.
  • Kim, S.S., Gulick, E.E., Nam, K.A., & Kim, S.H. (2008). Psychometric properties of the alcohol use disorders identification test: a Korean version. Archives of Psychiatric Nursing, 22, 190-199.
  • Knight, J.R., Sherritt, L., Harris, S.K., Gates, E.C., & Chang, G. (2003). Validity of brief alcohol screening tests among adolescents: A comparison of the AUDIT, POSIT, CAGE, and CRAFFT. Alcoholism: Clinical and Experimental Research, 27, 67-73.
  • Kokotailo, P.K., Egan, J., Gangnon, R., Brown, D., Mundt, M., & Fleming, M. (2004). Validity of the alcohol use disorders identification test in college students. Alcoholism: Clinical and Experimental Research, 8, 914-920.
  • Lima, C.T., Freire, A.C., Silva, A.P., Teixeira, R.M., Farrel, M., & Farrel, M. (2005). Concurrent and construct validity of the Audit in urban Brazilian sample. Alcohol and Alcoholism, 40, 584-589.
  • Matano, R.A., Koopman, C., Wanat, S.F., Whitsell, S.D., Borggrefe, A., & Westrup, D. (2003). Assessment of binge drinking of alcohol in highly educated employees. Addictive Behaviors, 28, 1299-1310.
  • Neumann, T., Gentilello, L.M., Neuner, B., Weiβ-Gerlach, E., Schürmann, H., Schröder, T., Müller, C., Haas, N.P., & Spies, C.D. (2009). Screening trauma patients with the Alcohol Use Disorders Identification Test and biomarkers of alcohol use. Alcoholism: Clinical and Experimental Research, 33, 970-976.
  • Pal, H.R., Jena, R., & Yadav, D. (2004). Validation of the Alcohol Use Disorders Identification Test (AUDIT) in urban community outreach and de-addiction center samples in north India. Journal of Studies on Alcohol, 65, 794-800.
  • Pérula-de-Torres, L.A., Fernández-García, J.A., Arias-Vega, R., Muriel-Palomino, M., Márquez-Rebollo, E., & Ruiz-Moral, R. (2005). Validity of AUDIT test for detection of disorders related with alcohol consumption in women. Medicina Clínica (Barcelona), 125, 727-730.
  • Philpot, M., Pearson, N., Petratou, V., Dayanandan, R., Silverman, M., & Marshall, J. (2003). Screening for problem drinking in older people referred to a mental health service: A comparison of CAGE and AUDIT. Aging Ment Health, 7, 171-175.
  • Reinert, D.F., & Allen, J.P. (2002). The Alcohol Use Disorders Identification Test (AUDIT): A review of recent research. Alcoholism: Clinical and Experimental Research, 26, 272-279.
  • Reinert, D.F., & Allen, J.P. (2007). The Alcohol Use Disorders Identification Test: An update of research findings. Alcoholism: Clinical and Experimental Research, 31, 185-199.
  • Rist, F., Glockner-Rist, A., & Demmel, R. (2009). The Alcohol Use Disorders Identification Test revisited: establishing its structure using nonlinear factor analysis and identifying subgroups of respondents using latent class factor analysis. Drug and Alcohol Dependence, 100(1-2), 71-82.
  • Rodríguez-Martos, A., & Santamariña, E. (2007). Does the short form of the Alcohol Use Disorders Identification Test (AUDIT-C) work at a trauma emergency department? Substance Use & Misuse, 42, 923-932.
  • Rumpf, H., Hapke, U., Meyer, C., & John, U. (2002). Screening for alcohol use disorders and at-risk drinking in the general population: Psychometric performance of three questionnaires. Alcohol and Alcoholism, 37, 261-268.
  • Rumpf, H.J., et al. (2003). Frühinterventionen bei alkoholbezogenen Störungen in der Allgemeinarztpraxis: Ein Stepped-Care Ansatz (Early intervention for alcohol-related disorders in the general practice: a stepped-care approach). Suchtmedizin, 5, 37-40.
  • Seale, J.P., Boltri, J.M., Shellenberger, S., Velasquez, M.M., Cornelius, M., Guyinn, M., Okosun, I., & Sumner, H. (2006). Primary care validation of a single screening question for drinkers. Journal of Studies on Alcohol, 67, 778-784.
  • Selin, K.H. (2003). Test-retest reliability of the Alcohol Use Disorder Identification Test in a general population sample. Alcoholism: Clinical and Experimental Research, 27, 1428-1435.
  • Shevlin, M, & Smith, G.W. (2007). The factor structure and concurrent validity of the Alcohol Use Disorder Identification Test based on a nationally representative UK sample Alcohol & Alcoholism, 42, 582587.
  • Silva, P., Jayawardana, P., & Pathmeswaran, A. (2008). Concurrent validity of the Alcohol Use Disorders Identification Test (AUDIT). Alcohol & Alcoholism, 43, 4950.
  • Tsai, M.C., Tsai, Y.F., Chen, C.Y., & Liu, C.Y. (2005). Alcohol Use Disorders Identification Test (AUDIT): Establishment of cut-off scores in a hospitalized Chinese population. Alcoholism: Clinical and Experimental Research, 29, 53-57.
  • Tuunanen, M., Aalto, M., & Seppä, K. (2007). Binge drinking and its detection among middle-aged men using AUDIT, AUDIT-C and AUDIT-3. Drug and Alcohol Dependence, 26, 295-299.
  • Von-der-Pahlen, B., Santtila, P., Witting, K., Varjonen, M., Jern, P., Johansson, A., & Sandnabba, N.K. (2008). Factor structure of the Alcohol Use Disorders Identification Test (AUDIT) for men and women in different age groups. Journal of Studies on Alcohol and Drugs, 69, 616-621.
  • Wu, S.I., Huang, H.C., Liu, S.I., Huang, C.R., Sun, F.J., Chang, T.Y., Shih, S.C., & Jeng, K.S. (2008). Validation and comparison of alcohol-screening instruments for identifying hazardous drinking in hospitalized patients in Taiwan. Alcohol & Alcoholism, 43, 577-582.
  • Correspondence regarding this article should be directed to:
    José Alexandre S. Crippa
    Departamento de Neurociências e Ciências do Comportamento;
    Faculdade de Medicina de Ribeirão Preto;
    Universidade de São Paulo Hospital das Clínicas - Terceiro Andar,
    Av. Bandeirantes, 3900 - Ribeirão Preto - São Paulo - Brasil - CEP - 14049-900.
    Phone/fax: (+55)-16-6022201/ (+55)-16-6350713.
    E-mail:
  • Publication Dates

    • Publication in this collection
      17 Jan 2011
    • Date of issue
      June 2009

    History

    • Accepted
      22 June 2009
    • Received
      04 Nov 2008
    • Reviewed
      20 Apr 2009
    Pontificia Universidade Católica do Rio de Janeiro, Universidade de Brasília, Universidade de São Paulo Rua Marques de São Vicente, 225, 22453-900 Rio de Janeiro/RJ Brasil, Tel.: (55 21) 3527-2109, Fax: (55 21) 3527-1187 - Rio de Janeiro - RJ - Brazil
    E-mail: psycneuro@psycneuro.org