Psychometric properties of the Brazilian version of the Personal Experience Screening Questionnaire

Propriedades psicométricas da versão brasileira do Personal Experience Screening Questionnaire

Summary

Introduction:

The Personal Experience Screening Questionnaire (PESQ) is an instrument devised for assessing the magnitude of drug misuse among adolescents. However, its psychometric properties have not been evaluated in adolescent samples outside the United States.

Objective:

To assess the internal reliability and validity of the Brazilian version of the PESQ.

Method:

A cross-sectional study was carried out with 84 adolescents from a clinical sample and a community-based sample. All of them answered the PESQ.

Results:

Among adolescents from the community, the PESQ problem severity index, which can vary from 18 to 72, was 26.48±9.28, whereas the clinical sample scored 42.89±10.02 (p<0.001). Cronbach's alpha was 0.91. Factor analysis resulted in a four-factor solution. Furthermore, both samples also had different mean scores for the other distinct content areas measured by the instrument.

Conclusion:

Evidence to support the reliability and validity of the Brazilian version of the Personal Experience Questionnaire was found.

Keywords:
drugs; screening; scales; validation; reliability

Resumo

Introdução:

o Personal Experience Screening Questionnaire (PESQ) é um instrumento destinado ao rastreamento de problemas relacionados ao uso de drogas entre adolescentes. Entretanto, suas propriedades psicométricas não foram avaliadas em populações de adolescentes de outros países além dos Estados Unidos.

Objetivo:

avaliar a confiabilidade e a validade da versão brasileira do PESQ.

Método:

foi feito um estudo transversal envolvendo 84 adolescentes de uma amostra clínica e de uma amostra não clínica. Todos responderam ao PESQ.

Resultados:

entre os adolescentes da população geral, o índice de severidade de problema da PESQ, que pode variar de 18 a 72, foi de 26,48±9,28. Por outro lado, a população clínica teve pontuação de 42,89±10,02 (p<0,001). O alfa de Cronbach foi de 0,91. A análise fatorial resultou em uma solução de quatro fatores. Além disso, ambas as populações apresentaram pontuações significativamente diferentes em outras subescalas do PESQ.

Conclusão:

as evidências sustentam a validade e a confiabilidade da versão brasileira do PESQ.

Palavras-chave:
drogas; rastreamento; escala; validação; confiabilidade

Introduction

Subjectivism involved in psychiatric diagnosis and the lack of biological markers for mental disorders⁷ can become an important drawback to their identification. This is especially true when it comes to addiction, as the borderlines between use, abuse, and dependence are not clear.¹⁰ In order to minimize diagnostic difficulties, symptom-based screening scales can be particularly useful.¹² A favorable screening instrument should present good cost-benefit relationship, be accurate, and consist of a small number of items, in order to be efficient.¹² Once the at-risk individuals are identified by a screening tool, the subsequent step is a psychiatric interview.^2,9,11 In this second stage, physicians and researchers can obtain a thorough comprehensive diagnosis. Assessment instruments must be developed or adapted to different cultural circumstances. This means that assessment measures developed for specific cultural environments must go under a cross-cultural validation before its use for different populations.¹⁶

In Brazil, national household surveys including those 15 years of age and older reveal a consistent increase of drug consumption.^3,4 In the First Household Survey,⁴ 19.4% of surveyed Brazilians reported they have already used any illicit drugs; this number was of 22.8% in the Second Household Survey.³ Lifetime use of marijuana totaled 6.9% in 2001 and increased to 8.8% in 2005. The same was observed for inhalants (5.8 and 6.1%, respectively) and for cocaine (2.3 and 2.9%). In 2005, the prevalence rates of substance dependence disorder for illicit drugs were as follows: marijuana (1.2%), benzodiazepines without medical prescription (0.5%), inhalants (0.2%), and stimulants (0.2%). Alcohol dependence was present among 11.2% of the population in 2001 and among 12.3% in 2005. The World Drug Reports of 2012 and of 2013^21,22 add some relevant data. In 2012, while South America presented a decline in cocaine consumption, Brazil increased its demand for the drug. Also, the 2013 Report states that Brazil has contributed with some new psychoactive substances, such as krokodil or bath salts, one of the main concerns of United Nations Office on Drugs and Crime (UNODC) in 2013.

Concerning substance abuse or dependence among adolescents, there are a few Brazilian studies. Frequent drinking (at least once a week in the past month) occurs in 9.1% of the Brazilian adolescent population (14 to 17 years old), and binge drinking in 52.9% of the boys and in 37.6% of the girls.¹⁵ A National Survey conducted in Brazil found that 2.8% of adolescents (14 to 19 years old) had past-year use of at least one illicit drug.¹³ According to this study, the most used illicit substances were marijuana (1.6%), solvents (1.1%), and crack/cocaine (0.5%).

The clinical field definitely warrants the need for a screening instrument capable of fast and efficient identification of adolescents likely to have a substance-related problem. A recent systematic review identified only two instruments validated for screening of problems related to substance use among Brazilian adolescents,¹⁷ the Drug Use Screening Inventory (DUSI) and the Teen-Addiction Severity Index (T-ASI). Both instruments are extensive (the DUSI has 149 questions and the T-ASI has 153 questions) and take too much time to be completed. A shorter version of the DUSI (DUSI-R) had its psychometric properties studied among Brazilian adolescents, with good results.⁸ However, the DUSI-R consists of 149-items and, thus, falls short of being an ideal screening tool. It is important that other well-studied instruments are available in the Brazilian cultural context, in order to provide clinicians and researches with a broader range of questionnaires.

The aim of this study was to validate the Brazilian version of a promising screening measure, the Personal Experience Screening Questionnaire (PESQ).²³

Method

Sample

The clinical sample involved all 41 adolescents admitted for treatment at the Addiction Unit (PROAD - Programa de Orientação e Atendimento a Dependentes, in Portuguese) of the Department of Psychiatric at the Federal University of São Paulo (Unifesp), Brazil, during 12 months. All of them were evaluated by a trained psychiatrist and by a psychologist, separately, and both confirmed the diagnosis of drug or alcohol dependence using a semi-structured interview based on DSM-IV-TR dependence criteria.¹

The comparison non-clinical group consisted of youths from a high school in São Paulo, Brazil. Forty-five adolescents were randomly drawn from a 120-adolescent sample. If a selected adolescent presented a lower age than 15 or higher than 18 years old, he/she was not included and another one was selected. The same age criterion was used in the clinical sample. All of them were evaluated by a trained psychiatrist and by a trained psychologist separately in order to assure that they had no substance use-related disorder. Two out of 45 were excluded in this phase, due to a diagnosis of alcohol dependence. Psychiatric comorbidities were not considered as an exclusion criteria. Treatment was offered for both of them and their parents.

The two groups that formed the final sample had the same average age (16.6 years); however, there were more boys in the treatment group (75 vs. 53%) in keeping with findings from many studies documenting a higher prevalence of substance abuse among boys. As expected, a lower proportion of participants in the clinical sample were living with their parents compared to the comparison group (57 vs. 97%). More details are provided in Table 1.

TABLE 1
Socio-demographic information of adolescents under treatment for drug dependence (cases) comparatively to nonusers (controls) (N=84).

	Controls (%) (n=43)	Cases (%) (n=41)	p-value
Gender (%)
Male	53.5	75.6	0.03
Female	46.5	24.4
Age ± SD	16.65±1.00	16.61±1.20	>0.05
Living with parents (%)
Living together – Good relationship	55.8	36.6	>0.05
Living together – Bad relationship	11.6	14.6
Divorced parents	30.2	39.0
Parents deceased	2.3	9.7
School lag (%)	0	42.9	<0.001

Instrument

Both groups were administered the Brazilian version of the PESQ. The PESQ is a self-administered screening questionnaire developed as part of a multi-assessment battery.²³ The instrument is formatted to have two (yes/no), three (never/once or twice/more than once or twice) or four (never/once or twice/sometimes/often) response options. The alcohol or drug use frequency items were formatted using a 7-point scale (1 = never; 7 = 40 times or more).^23,24,26 The 40-item PESQ questionnaire consists of five areas:

1. Drug use problem severity scale (18 items: questions 1 to 8; 10 to 14; and 16 to 20) - aims to provide a global measure of drug or alcohol use severity. This index provides a global measure of problem severity by reflecting the extent to which the individual is psychologically and behaviorally involved with drugs. Scoring of index consists of a simple no-weighted sum of the 18 items, with deviance associated by a high score. Scale scores range from 18 ("never" response to all items) to 72 ("often" response to all items); a nonuser would logically score 18.

2. Efensiveness (five items: questions 22, 24, 26, 29, and 31) - based on the Marlowe-Crowne Social Desirability Scale,⁵ this set of items measures faking-good response tendencies. It identifies teenagers with a tendency to minimize or deny problems. The responses given usually reflect "social desirability." On questions 24, 26, and 31, a positive answer adds one point and a negative one adds two points to the score, whereas on questions 22 and 29 scoring is similar but reversed.

3. Infrequency (three items: questions 9, 15, and 21) - a high score on this subscale is associated with faking bad, as it inquires about unlikely drug use behavior.

4. Psychological indicators (eight items: questions 23, 25, 27, 28, 30, 32, 33, and 34) - covers personal or environmental problems associated with adolescent drug or alcohol use. The presence of at least one positive answer for the questions above indicates the existence of a significant psychological stressor.

5. Drug use history (six items: questions 35 to 40) - summarizes frequency of alcohol and illicit drug use in the past year.

It is not a public domain instrument and it requires a fourth grade reading level by the respondent. Ten minutes are necessary to complete all questions. The characteristics of the scale result in a fast and efficient instrument for drug use screening and for referral for a diagnostic evaluation. Even though its use on epidemiological researches is still limited, it is recommended as a standard part of clinical evaluation of adolescents.¹⁴ Norms for non-clinical, juvenile offender and drug-abusing populations are available.²⁶

The paper and pencil format of the PESQ was translated into Brazilian Portuguese. The translation was conducted using a two-step procedure. A bilingual psychiatrist first translated the items from English to Brazilian Portuguese followed by back translation into English conducted by a linguist. Discrepancies between the two versions were resolved during discussion by modifying word selection.

PESQ was administered to the clinical sample as part of the intake assessment upon treatment admission. The non-clinical sample data were gathered by a trained team of field researchers. Anonymous standardized paper-and-pencil questionnaire was used. Interviews were conducted in the school, without a teacher or school member present in the classroom. Research objectives, anonymity, confidentiality, and volunteering were explained to all adolescents by the research team. On average, students took 10 minutes to complete the questionnaire.

The DUSI was used as an instrument of comparison. It is a validated instrument for screening of drug use among adolescents. The characteristics of the scale results in a fast and efficient self-administered questionnaire for drug use screening, also useful for planning treatment and monitoring outcomes.

Statistical analysis

Collected data were analyzed through the software package SPSS 20.0. Comparisons were held through chi-square and t-test. The variables were tested in order to verify its normal distribution (skewness and kurtosis) and Pearson's coefficient was used in order to establish correlations. Internal reliability consistency was evaluated using Cronbach's alpha. A level of significance of 5% was considered.

Factor analysis was conducted with 18-item PESQ Scale (Problem Severity Index). This was followed by principal components extraction with Varimax and Oblimin rotation. We used a) a minimum eingenvalue of 1; b) a minimum factor loading of 0.45 and above; c) a minimum factorial complexity (multiple loading); d) a meaningful interpretation of the factor, and e) a percentage of extracted variance (5%) to determine the number of factors. A screen test between factors was plotted to consolidate the total number of factors to be extracted in factor analysis. The screen test based on the cut-off eigenvalues further guided toward the number of factors to be retained for further analysis.

Ethical approval

The research project was carried out in accordance with the latest version of the Declaration of Helsinki and approved by Unifesp's ethics committee (Ref.: CEP n. 464/99). All participants and their guardians gave informed written consent before participating.

Results

Problem severity index

The mean score of PESQ problem severity index was 26.48 (SD 9.3) among the non-clinical population and 42.89 (SD 10.0) among the clinical population (p<0.001). This index was highly correlated with scores obtained by these populations on the DUSI-R (1.9±2.5 for the non-clinical sample and 6.7±2.8 for the clinical sample, p<.001) (Spearman coefficient: 0.83, p<0.001).

Cronbach's alpha of the problem severity index was of 0.91, comparable to scores obtained in other PESQ studies.^23,25 An evaluation of the load of each item on the alpha result was performed and all items contributed in an important way to this value.

After factor analysis of this 18-item index, a four-factor solution was reached. These four factors captured 68.70% of the unrotated and rotated variances. The four factors extracted were as follows: i) factor I - items 1 to 6 (eingenvalue: 7.68); ii) factor II - items 8 and 10 to 14 (eingenvalue: 1.91); iii) factor III - items 7 and 17 to 20 (eingenvalue: 1.52); and iv) factor IV - item 16 (eingenvalue: 1.24). Data on factor loading and on variance are on Tables 2 and 3. Factors I, II and III present good internal consistency (Cronbach's alpha of 0.870, 0.798 and 0.852, respectively). Internal consistency of factor IV could not be calculated, as it is comprised of just one question.

TABLE 2
Factor loading from the Varimax rotated factor structure matrix for the PESQ: Principal component extraction with Varimax rotation and Kaiser normalization.

	Factor 1	Factor 2	Factor 3	Factor 4
Item 1	0.581
Item 2	0.625
Item 3	0.684
Item 4	0.808
Item 5	0.818
Item 6	0.773
Item 7			0.514
Item 8		0.479
Item 10		0.485
Item 11		0.578
Item 12		0.702
Item 13		0.775
Item 14		0.763
Item 16				0.874
Item 17			0.635
Item 18			0.785
Item 19			0.635
Item 20			0.648

TABLE 3
Total variance explained (extraction method): Principal component analysis.

Component	Initial eigenvalues			Extraction sums of squared loadings			Rotation sums of squared loadings
	Total	% of variance	Cumulative %	Total	% of variance	Cumulative %	Total	% of variance	Cumulative %
1	7.688	42.710	42.710	7.688	42.710	42.710	4.178	23.209	23.209
2	1.916	10.642	53.352	1.916	10.642	53.352	3.258	18.099	41.308
3	1.518	8.435	61.787	1.518	8.435	61.787	2.872	15.958	57.266
4	1.246	6.921	68.707	1.246	6.921	68.707	2.059	11.441	68.707
5	0.852	4.736	73.443
6	0.741	4.116	77.559
7	0.645	3.586	81.145
8	0.624	3.464	84.609
9	0.475	2.638	87.247
10	0.463	2.571	89.818
11	0.355	1.972	91.790
12	0.332	1.845	93.635
13	0.297	1.648	95.283
14	0.249	1.385	96.668
15	0.195	1.081	97.749
16	0.177	0.983	98.732
17	0.142	0.787	99.519
18	0.087	0.481	100.000

• Extraction method: Principal component analysis.

Other indexes

The mean score on the defensiveness index was 6.6±0.9 among the non-clinical sample and 7.27±1.26 among the clinical sample (p=0.007). This suggests that both study groups did not provide compromised self-report data. The mean scores on the infrequency index were 0.1±0.4 and 1.6±1.9, respectively (p<.001). Although there was a significant difference between groups, both had very low mean scores on this scale and thus we can assume that, in general, adolescents did not show a tendency to fake bad behavior. The mean scores on the psychological indicators index were 2.0±1.6 and 2.7±2.2, respectively (p>.05).

Discussion

The data analysis provides support for the reliability and validity of the PESQ in a Brazilian youth population (15 to 18 years old). It appears to be an efficient instrument for screening substance-related disorders among adolescents. Statistically significant differences between the clinical and non-clinical samples were found on the Problem Severity Index. No cut-off was studied, but the higher the score, the higher the need of a detailed clinical evaluation. It is important to emphasize that PESQ evaluates not only parameters related to substance use but also considers individual and environmental characteristics. This enriches the comprehension of the dependence phenomenon and contributes to a more detailed psychiatric diagnosis.

Principal component factor analysis of the Brazilian version of PESQ resulted in a four-factor solution. In the validation of the original version of the instrument, only one factor was identified (eigenvalue of 6.8; percent of variance 38.1). Item factor loadings ranged from .50 to .74 (median .62). Separate principal component factor analyses were also computed by gender, and similar results were found. It is a concern that only one item loads on the fourth factor, as there was no covariance to consider except the item's own variance. Different rotation methods were used in order to confirm this finding, always yielding the same results. The item (question 16) could not be excluded, as it was important for the whole scale internal consistency.

This is of great interest, since the PESQ could be used as a large-scale screening instrument. Recently, the World Health Organization (WHO) has focused great attention on integrating mental disorders into primary care.²⁷ This is especially important among low and middle-income countries (LAMIC), like Brazil. It is well-known that millions of people suffer from mental disorders, including substance abuse and dependence, and this leads to a great social burden.^20,27 Training primary care professionals for early identification of individuals at-risk is one of the most cost-effective interventions available.²⁸ This is also the best way to reduce the gap on mental health facilities,^20,28 present in many LAMIC, including Brazil. Having a short, cheap and effective instrument such as PESQ available for the Brazilian context is very useful. It could be widely used in schools or in primary care facilities by any health professional.

Some limitations of this study must also be considered. First, a high alpha value does not necessarily establish good internal consistency. As the alpha is affected by the length of the test, adding related items testing the same concept increases its value.¹⁹ This probably is not the case here, because in the problem severity index we are dealing with an 18-item scale, not a long one. Moreover, sample size and gender difference between both samples may have biased our findings. Factor analysis is very sensitive to the size of correlation, and correlation coefficients tend to be less reliable when estimated from small sample sizes.¹⁸ However, the high load of each factor obtained and the small number of factors found probably assure that this was not a problem, either.⁶ Finally, additional validity data are needed regarding how well the Brazilian PESQ predicts a criterion variable (e.g., substance use disorder diagnosis).

• Study conducted at the Department of Psychiatry, Universidade Federal de São Paulo (Unifesp), São Paulo, SP, Brazil

References

• ¹
  American Psychiatric Association (APA). Diagnostic and statistical manual of mental disorders. 4. ed., text rev. APA; 2000.
• ²
  Canino G, Lewis-Fernandez R, Bravo M. Methodological challenges in cross-cultural mental health research. Transcult Psychiatry. 1997; 34(2):163-84.
• ³
  Carlini EA, Galduróz JCF, Noto AR, Fonseca AM, Carlini CM, Oliveira LG, et al. [Second household survey on psychotropic substances use in Brazil: study involving 108 large cities in the country]. São Paulo: CEBRID; 2005.
• ⁴
  Carlini EA, Galduróz JCF, Noto AR, Nappo SA. [First household survey on psychotropic substances use in Brazil]. São Paulo: CEBRID; 2001.
• ⁵
  Crowne DP, Marlowe D. A new scale of social desirability independent of psychopathology. J Consult Psychol. 1960; 24(4):349-54.
• ⁶
  De Geus F, Denys D. Pitfalls in factor analytic techniques. Am J Psychiatry. 2004; 161(3):579-80.
• ⁷
  Fidalgo TM, Morales-Quezada JL, Muzy GS, Chiavetta NM, Mendonca ME, Santana MV, et al. Biological markers in noninvasive brain stimulation trials in major depressive disorder: a systematic review. J ECT. 2014; 30(1):47-61.
• ⁸
  Fidalgo TM, Tarter R, da Silveira ED, Kirisci L, da Silveira DX. Validation of a short version of the revised drug use screening inventory in a Brazilian sample of adolescents. Am J Addict. 2010; 19(4):364-7.
• ⁹
  Flaherty JA, Gaviria FM, Pathak D, Mitchell T, Wintrob R, Richman JA, et al. Developing instruments for cross-cultural psychiatric research. J Nerv Ment Dis. 1988; 176(5):257-63.
• ¹⁰
  Hasin DS, O'Brien CP, Auriacombe M, Borges G, Bucholz K, Budney A, et al. DSM-5 criteria for substance use disorders: recommendations and rationale. Am J Psychiatry. 2013; 170(8):834-51.
• ¹¹
  Kaiser BN, Kohrt BA, Keys HM, Khoury NM, Brewster AR. Strategies for assessing mental health in Haiti: local instrument development and transcultural translation. Transcult Psychiatry. 2013; 50(4):532-58.
• ¹²
  Lu F, Petkova E. A comparative study of variable selection methods in the context of developing psychiatric screening instruments. Stat Med. 2014; 33(3):401-21.
• ¹³
  Madruga CS, Laranjeira R, Caetano R, Pinsky I, Zaleski M, Ferri CP. Use of licit and illicit substances among adolescents in Brazil - a national survey. Addict Behav. 2012; 37(10):1171-5.
• ¹⁴
  Milin R, Walker S. Adolescent substance abuse. In: Ruiz P, Strain E, editors. Lowinson and Ruiz's substance abuse - A comprehensive textbook. Philadelphia: Lippincott Williams & Wilkins; 2011.
• ¹⁵
  Pinsky I, Sanches M, Zaleski M, Laranjeira R, Caetano R. Patterns of alcohol use among Brazilian adolescents. Rev Bras Psiquiatr. 2010; 32(3):242-9.
• ¹⁶
  Prince M, Acosta D, Chiu H, Scazufca M, Varghese M; 10/66 Dementia Research Group . Dementia diagnosis in developing countries: a cross-cultural validation study. Lancet. 2003; 361(9361): 909-17.
• ¹⁷
  Rosário AMM. [Evaluation of drug and alcohol abuse screening instruments among adolescents: literature review] [dissertation]. São Paulo: Faculdade de Medicina, Universidade de São Paulo; 2011.
• ¹⁸
  Stevens J. Applied multivariate statistics for the social sciences. 3. ed. Mahwah: Laurence Erlbaum Associates; 1996.
• ¹⁹
  Tavakol M, Dennick R. Making sense of Cronbach's alpha. Int J Med Educ. 2011; 2:53-5.
• ²⁰
  Thornicroft G, Alem A, Antunes dos Santos R, Barley E, Drake RE, Gregorio G, et al. WPA guidance on steps, obstacles and mistakes to avoid in the implementation of community mental health care. World Psychiatry. 2010; 9(2):67-77.
• ²¹
  United Nations Office on Drugs and Crime. World Drug Report 2012. United Nations Publication, Sales No. E.12.XI.1. Vienna: United Nations Office on Drugs and Crime; 2012.
• ²²
  United Nations Office on Drugs and Crime. World Drug Report 2013. United Nations Publication, Sales No. E.13.XI.6. Vienna: United Nations Office on Drugs and Crime; 2013.
• ²³
  Winters KC. Development of an adolescent alcohol and other drug abuse screening scale: Personal Experience Screening Questionnaire. Addict Behav. 1992a; 17(5):479-90.
• ²⁴
  Winters KC. Personal experience self questionnaire manual. Los Angeles: Western Psychological Services; 1992b.
• ²⁵
  Winters KC, DeWolfe J, Graham D, St. Cyr W. Screening American Indian youth for referral to drug abuse prevention and intervention services. J Child Adolesc Subst Abuse. 2006; 16(1):39-52.
• ²⁶
  Winters KC, Kaminer Y. Screening and assessing adolescent substance use disorders in clinical populations. J Am Acad Child Adolesc Psychiatry. 2008; 47(7):740-4.
• ²⁷
  World Health Organization. Integrating mental health into primary health care: a global perspective. Geneva: World Health Organization; 2008a.
• ²⁸
  World Health Organization. Mental Health Gap Action Programme: Scaling up care for mental, neurological, and substance use disorders. Geneva: World Health Organization; 2008b.

Publication Dates

• Publication in this collection
  Nov 2016

History

• Received
  27 Jan 2016
• Accepted
  15 Feb 2016