Structural validity and reliability of the Positive and Negative Affect Schedule (PANAS): Evidence from a large Brazilian community sample

Abstract

Objective:

Positive and negative affect are the two psychobiological-dispositional dimensions reflecting proneness to positive and negative activation that influence the extent to which individuals experience life events as joyful or as distressful. The Positive and Negative Affect Schedule (PANAS) is a structured questionnaire that provides independent indexes of positive and negative affect. This study aimed to validate a Brazilian interview-version of the PANAS by means of factor and internal consistency analysis.

Methods:

A representative community sample of 3,728 individuals residing in the cities of São Paulo and Rio de Janeiro, Brazil, voluntarily completed the PANAS. Exploratory structural equation model analysis was based on maximum likelihood estimation and reliability was calculated via Cronbach's alpha coefficient.

Results:

Our results provide support for the hypothesis that the PANAS reliably measures two distinct dimensions of positive and negative affect.

Conclusion:

The structure and reliability of the Brazilian version of the PANAS are consistent with those of its original version. Taken together, these results attest the validity of the Brazilian adaptation of the instrument.

Emotion; epidemiology; structural equation modeling; psychometrics; Positive and Negative Affect Schedule

Introduction

Positive affect (PA) and negative affect (NA) are the two dominant mood factors that emerge from self-report analysis of semantic affect terms.¹-⁴ They are conceived as psychobiological-dispositional dimensions reflecting proneness to positive and negative activation that influence the extent to which individuals experience life events as joyful or as distressful, respectively.⁵

The Positive and Negative Affect Schedule⁶ (PANAS) is a brief self-report measure designed to provide independent PA and NA indexes. Originally, the PANAS was designed by empirically selecting items from a larger pool of mood descriptors that functioned as relatively ‘pure’ indicators of PA and NA for adults in the American English natural language. Since the development of the PANAS, different versions have been successfully elaborated in different languages,⁷-¹⁰ for children¹¹-¹³ and for adolescents.¹⁴ These studies consistently have validated the orthogonal bi-factor latent structure believed to underlie the PANAS item set.

The international popularity of the PANAS may be attributed to its brevity and, most importantly, to its association with an influential conceptualization of anxiety and depression: the tripartite model.¹⁵-¹⁷ This model aimed to differentiate depression from anxiety by designing a hierarchical structure that allowed the identification of both specific and common factors. It posits that, in addition to a factor of NA common to all unipolar mood and anxiety disorders, there are specific domains to anxiety and depression that allow them to be differentiated: anxiety is further characterized by physiological hyperarousal, and depression, by anhedonia or low PA. This model has been directly tested and the PANAS has shown to be effective at differentiating between depression and anxiety in clinical samples.¹⁸

Furthermore, the PANAS has also been largely employed in research that aims to understand subjective well-being.¹⁹,²⁰ Indeed, PA is considered to be a key element of well-being and general adaptability.¹⁰,²⁰

The aim of the current study is to validate a Brazilian Portuguese interview-version of the PANAS by means of factor and internal consistency analysis in a large and representative community sample from the cities of Rio de Janeiro and São Paulo, Brazil.

Method

Ethical issues

The Ethics Committee of Universidade Federal de São Paulo approved the study. Informed consent was obtained from all respondents, and participants who met diagnostic criteria have been offered a referral to outpatient clinics at Universidade Federal de São Paulo and Universidade Federal do Rio de Janeiro.

Study design

The dataset of the current study was derived from a single-session, population-based cross-sectional survey carried out in the Brazilian cities of São Paulo and Rio de Janeiro conducted to assess the impact of urban violence on the prevalence of mental health problems. A detailed description of the protocol of this study - including sampling and data collection procedure - is provided by Andreoli et al.²¹

Sample

A representative community sample of 3,728 individuals, 1,614 male and 2,114 female, mean age 39.38±15.52 years (range = 15 to 75 years), residing in the cities of São Paulo and Rio de Janeiro, participated voluntarily in this study, which included the administration of the PANAS along with other instruments.

Instrument

The PANAS consists of two 10-item mood scales designed to provide independent measures of PA and NA. The PANAS was originally designed as a self-report questionnaire; however, the administration of the PANAS via face-to-face interviews was preferred in this study to allow standardization of data gathering procedures regardless of the literacy level of the participant. Respondents were asked to rate, on a 5-point Likert scale (very slightly or not at all to very much), the extent to which they experienced each particular emotion within a general time frame (i.e., in general, in your life as a whole), yielding trait-oriented scores. The items that composed this interview version of the PANAS were based on a translation/back-translation procedure performed on the original American English version.⁶

Statistical analysis

Exploratory factor analysis (EFA) via structural equation modeling (SEM) and internal consistency reliability were used to verify the structural coherence and unidimensionality of the PANAS item set, respectively. All structural analyses were undertaken in the Mplus version 5 computer package and Cronbach's alpha coefficients were computed using the Statistical Package for the Social Sciences (SPSS) version 20 for Windows.

Factor analysis seeks to identify continuous latent factors that hypothetically account for the pattern of correlations among the observed variables.²² If conducted in the framework of SEM, EFA produces parameter estimates such as standard errors, model-data misfit, goodness of fit, and indicators of specific areas of misfit.²³ The goodness of fit for a candidate model reflects the degree to which the model is able to account for the structure of the observed data and thus serves as an elegant indicator of which factor solution (number of factors) should be retained.

PANAS dimensionality was assessed via maximum likelihood EFA with varimax rotations. Factor solutions ranging from one to four factors were evaluated and the number of retained factors was based on cutoff values for selected absolute and parsimonious goodness-of-model-fit indexes and standard errors. Root mean square residual (RMR) was used as an indicator of absolute fit (values below 0.05 indicate good absolute fit), root mean square error of approximation (RMSEA) was used as an indicator of parsimonious fit (values lower than 0.05 indicate close and those up to 0.08 reasonable fit), and standard errors values below 0.8 are regarded as indicative of and an acceptable level of misfit.²⁴ We considered factor loadings interpretable only if they exceeded a value of 0.32 (10% of common variance accounted for).

Cronbach's alpha coefficients were computed to estimate internal consistency reliability for the extracted factors based on the sum of item scores. Values above 0.7 are acceptable and above 0.8 are desirable.

Results

Sample adequacy indicators attested to the suitability of the dataset for implementation of factor analysis: a KMO value of 0.888 was observed and Bartlett's test of sphericity was significant (p < 0.001).

An orthogonal two-factor solution was retained (RMR = 0.04, RMSEA = 0.08, standard errors ≥ 0.08). By contrast, the one-factor solution did not show acceptable goodness of fit (RMR = 0.153; RMSEA = 0.146) and the extraction of models with three (RMR = 0.0276; RMSEA = 0.062) or four factors (RMR = 0.018; RMSEA = 0.043), while improving goodness of model-fit, produced factor solutions with cross-loadings for several items and of difficult theoretical interpretation.

The first factor accounted for nine of the original 10 PA items and the second factor accounted for the all NA items. The item proud did not load significantly in any factor (factor loading < 0.3) and it was consequently excluded from further analysis. Results were replicated in the female (RMR = 0.042, RMSEA = 0.08) and male (RMR = 0.039, RMSEA = 0.06) subsets of the sample. Reliability analysis revealed high internal consistencies for both the NA and PA scales (Cronbach's alpha coefficients = 0.87 and 0.88, respectively). Table 1 portrays the retained factor structure and parameter estimates for the total sample after the exclusion of the item proud.

Table 1
Maximum likelihood exploratory factor solution with promax rotation for the PANAS item set

Original items	Brazilian Portuguese items	Positive affect	Negative affect	Residual variance
Active	Ativo	0.69	-0.06	0.54
Alert	Alerta	0.57	0.05	0.65
Attentive	Atento	0.57	-0.02	0.69
Determined	Determinado	0.76	-0.07	0.45
Enthusiastic	Entusiasmado	0.64	0.07	0.54
Excited	Empolgado	0.63	0.08	0.56
Inspired	Inspirado	0.68	0.00	0.54
Interested	Interessado	0.75	-0.08	0.47
Strong	Forte	0.60	-0.08	0.66
Afraid	Com medo	0.01	0.61	0.54
Ashamed	Envergonhado	0.05	0.47	0.68
Distressed	Aflito	0.05	0.73	0.40
Guilty	Culpado	0.01	0.51	0.60
Hostile	Hostil	0.14	0.42	0.65
Irritable	Irritável	0.08	0.61	0.58
Jittery	Inquieto	0.21	0.53	0.61
Nervous	Nervoso	0.04	0.72	0.45
Scared	Apavorado	-0.02	0.69	0.44
Upset	Chateado	-0.02	0.66	0.50
RMR	0.04
RMSEA	0.07
% Variance		32.99	17.06
Alpha		0.88	0.87

• Factor loadings with value above 0.32 are in bold. RMR = root mean square residual; RMSEA = root mean square error of approximation.

Discussion

This study sought to verify the structural coherence and internal consistency reliability of a Brazilian interview-version of the PANAS in a large community-based sample of participants residing in the cities of São Paulo and Rio de Janeiro.

Our findings support the hypothesis that an orthogonal two-factor structure adequately represents the dimensionality of the PANAS item set. In turn, this structure appears consistent with neuropsychological evidence that approach and defensive motivational systems, presumed to underlie positive and negative mood, can be viewed as alternative activation (arousal) systems distinguished by the valence (pleasantness/distress) of the activation states with which they are associated.²⁵,²⁶ The orthogonality of these constructs may account for both pure positive or negative mood states and mixed/conflictuous mood states, where both positive and negative arousals are simultaneously active.

The current study has strengths and limitations. First, this study utilized the largest participant sample ever reported in a PANAS structural validation study. The size and representativeness of the current community sample and its structural replication for the male and female halves of the sample strongly supports generalization of prior findings for American participants to the Brazilian population as a whole. Furthermore, the replication of the purported orthogonal bi-factor structure, designed for self-report versions of the PANAS, in our interview version confers robustness to the PA/NA model, as it indicates its structural invariance regardless of data collection procedure. Also, the use of SEM to select the optimal solution in an exploratory factor procedure allowed the direct estimation of goodness-of-fit indexes and residual variance, which represents a statistical advancement in relation to classical methods of fit estimation such as scree plots, eigenvalues, and parallel analysis.

One limitation in particular warrants mention. As previously stated, the original version of the PANAS⁶ and some subsequent versions¹⁰-¹³ were designed by empirically choosing items from a larger pool of mood descriptors that worked as fairly ‘pure’ indexers of PA and NA in colloquial language. The version used in the current study was created by translating and back-translating the original version of the PANAS into Brazilian Portuguese without the realization of a previous psycholexical study. This could account for the fact that the item proud did not load in any factor, being excluded from the analysis. We recommend that a psycholexical study be conducted in future applications of this version of the PANAS, specifically as to the identification of an affect term culturally comparable to proud.

Finally, the data presented in this manuscript is part of larger epidemiological survey²¹ that aimed to understand the effect of violence in mental health. This survey gathered a wide range of social, biological, psychological and psychiatric variables hypothetically associated with violence including measures of PA and NA. Therefore, the validation of our interview version of the PANAS is a basic requirement to support the use of PANAS scores in subsequent studies using this data set.

In sum, the results of the current investigation, when taken together, provide psychometric evidence of structural validity and internal consistency for the PANAS version used in this study. Additionally, our findings support the hypothesis that an orthogonal two-factor structure adequately represents the dimensionality of the PANAS item set in the Brazilian population, and suggest that lexical adaptation in one PA item - proud - is required.

This study was supported by Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) and Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES).

References

• ¹
  Russell JA. A circumplex model of affect. J Pers Soc Psychol. 1980; 39:1161-78.
• ²
  Russell JA. Core affect and the psychological construction of emotion. Psychol Rev. 2003;110:145-72.
• ³
  Tellegen A, Watson D, Clark LA. On the dimensional and hierarchical structure of affect. Psychol Sci. 1999;10:297-303.
• ⁴
  Remington NA, Fabrigar LR, Visser PS. Reexamining the circumplex model of affect. J Pers Soc Psychol. 2000;79:286-300.
• ⁵
  Watson D, Wiese D, Vaidya J, Tellegen A. The two general activation systems of affect: structural findings, evolutionary considerations, and psychobiological evidence. J Pers Soc Psychol. 1999;76:820-38.
• ⁶
  Watson D, Clark LA, Tellegen A. Development and validation of brief measures of positive and negative affect: the PANAS scales. J Pers Soc Psychol. 1988;54:1063-70.
• ⁷
  Galinha IC, Pais-Ribeiro JL. Contribuição para o estudo da versão portuguesa da positive and negative affect schedule (PANAS): II - estudo psicométrico. Anal Psicol. 2005;23(2):219-227.
• ⁸
  Sandín B, Chorot P, Lostao L, Joiner TE, Santed MA, Valiente RM. [The PANAS scales of positive and negative affect: factor analytic validation and cross-cultural convergence]. Psicothema. 1999;11:37-51.
• ⁹
  Giacomoni CH, Hutz CS. A mensuração do bem-estar subjetivo: escala de afeto positivo e negativo e escala de satisfação de vida [abstract]. In: Anais do XXVI Congresso Interamericano de Psicologia; 1997. p. 313.
• ¹⁰
  Krohne HW, Egloff B, Kohlmann CW, Tausch A. [Investigations with a German version of the positive and negative affect Schedule (PANAS)]. Diagnostica. 1996;42:139-56.
• ¹¹
  Laurent J, Catanzaro SJ, Joiner TE, Rudolph K D, Potter KI, Lambert S, et al. A measure of positive and negative affect for children: scale development and preliminary validation. Psychol Assess. 1999;11:326-38.
• ¹²
  Giacomoni CH, Hutz CS. [Positive and negative affect schedule for children: development and validation studies]. Psicol Esc Educ. 2006;10:235-45.
• ¹³
  Damásio BF, Pacífico JC, Poletto M, Koller SH. Refinement and psychometric properties of the eight-item Brazilian positive and negative affect scale for children (PANAS-C8). J Happiness Stud. 2012;1-16
• ¹⁴
  Segabinazi JD, Zortea M, Zanon C, Bandeira DR, Giacomoni CH, Hutz CS. [Positive and negative affect scale for adolescents: adaptation, standardization and validity evidence]. Aval Psicol. 2012;11:1-12.
• ¹⁵
  Watson D, Clark L, Carey G. Positive and negative affectivity and their relation to anxiety and depressive disorders. J Abnorm Psychol. 1988;97:346-53.
• ¹⁶
  Clark LA, Watson D. Tripartite model of anxiety and depression: psychometric evidence and taxonomic implications. J Abnorm Psychol. 1991;100:316-36.
• ¹⁷
  Watson D. Differentiating the mood and anxiety disorders: a quadripartite model. Annu Rev Clin Psychol. 2009;5:221-47.
• ¹⁸
  Dyck MJ, Jolly JB, Kramer T. An evaluation of positive affectivity, negative affectivity, and hyperarousal as markers for assessing between syndrome relationships. Pers Individ Dif. 1994;17:637-46.
• ¹⁹
  Tennant R, Hiller L, Fishwick R, Platt S, Joseph S, Weich S, et al. The Warwick-Edinburgh mental well-being scale (WEMWBS): development and UK validation. Health Qual Life Outcomes. 2007;5:63
• ²⁰
  Albuquerque AS, Tróccoli BT. [Development of a subjective well-being scale]. Psic: Teor e Pesq. 2004;20:153-64.
• ²¹
  Andreoli SB, Ribeiro WS, Quintana MI, Guindalini C, Breen G, Blay SL, et al. Violence and post-traumatic stress disorder in Sao Paulo and Rio de Janeiro, Brazil: the protocol for an epidemiological and genetic survey. BMC Psychiatry. 2009;9:34.
• ²²
  McDonald RP. Factor analysis and related methods. Hillsdale (NJ): Lawrence Erlbaum Associates, Inc.; 1985.
• ²³
  Asparouhov T, Muthén B. Exploratory structural equation modeling. Struct Equ Modeling. 2009;16:397-438.
• ²⁴
  Marôco J. Análise de equações estruturais: fundamentos teóricos, software ? aplicações. Lisbon: Report Number; 2010.
• ²⁵
  Patrick CJ, Bernat EM. The construct of emotion as a bridge to between personality and psychopathology. In: Krueger RF, Tackett JL, editors. Personality and psychopathology. New York: The Guilford Press; 2006. p. 174-209.
• ²⁶
  Clark LA. Temperament as a unifying basis for personality and psychopathology. J Abnorm Psychol. 2005;114:505-21.

Publication Dates

• Publication in this collection
  April-June 2013

History

• Received
  27 Aug 2012
• Accepted
  22 Nov 2012