Abstract
Objective:
To examine the psychometric properties of the Liebowitz Social Anxiety Scale-Self Report (LSAS-SR) based on a large sample recruited from 16 Latin American countries, Spain, and Portugal.
Methods:
Two groups of participants were included: a non-clinical sample involving 31,243 community subjects and a clinical sample comprising 529 patients with a diagnosis of social anxiety disorder (SAD). Exploratory factor analysis (EFA), confirmatory factor analysis (CFA) and exploratory structural equation modeling (ESEM) were used in order to determine the psychometric properties of the LSAS-SR.
Results:
EFA identified five factors with eigenvalues greater than 1.00 explaining 50.78% of the cumulative variance. CFA and ESEM supported this 5-factor structure of the LSAS-SR. The factors included: 1) speaking in public; 2) eating/drinking in front of other people; 3) assertive behaviors; 4) working/writing while being observed; and 5) interactions with strangers. Other psychometric properties such as inter-factor correlations, invariance, reliability, and validity of the scale were also found.
Conclusion:
Psychometric data support the internal consistency and convergent validity of the LSAS-SR. It seems to be a valid and reliable measure of global social anxiety for Spanish and Portuguese-speaking countries, although when considering a multidimensional approach (factor-based assessment) it seems to be lacking some relevant social situations that are feared in those countries.
Social anxiety disorder; questionnaires; psychometrics; cross-cultural comparison
Introduction
Social anxiety disorder (SAD) or social phobia refers to a “marked fear or anxiety about one or more social situations in which the individual is exposed to possible scrutiny by others.”11. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5). Arlington: American Psychiatric Publishing; 2013. SAD is among the most prevalent anxiety disorders, with 12-month prevalence rates of 6.8% in the United States of America.22. Kessler RC, Üstün TB. The WHO Mental Health Surveys: global perspectives on the epidemiology of mental disorders. New York: Cambridge University; 2008. Nevertheless, little is known about the prevalence of SAD in Spanish and Portuguese-speaking countries. Extant epidemiological surveys suggest a 12-month SAD prevalence rate of 0.60% in Spain, 2.8% in Colombia, 2.60% among Mexican women, and 1.40% among Mexican men.22. Kessler RC, Üstün TB. The WHO Mental Health Surveys: global perspectives on the epidemiology of mental disorders. New York: Cambridge University; 2008. The prevalence of this condition in Portugal and other Latin American countries remains unknown.
SAD has been associated with substantial functional and occupational impairment, a chronic, unremitting course, an elevated risk of comorbid depression,33. Grant BF, Hasin DS, Blanco C, Stinson FS, Chou SP, Goldstein RB, et al. (2005). The epidemiology of social anxiety disorder in the United States: results from the National Epidemiologic Survey on Alcohol and Related Conditions. J Clin Psychiatry. 2005;66:1351-61. and lower social skills.44. Caballo VE, Salazar IC, Equipo de Investigación CISO-A España. Desarrollo y validación de un nuevo instrumento para la evaluación de las habilidades sociales: El “Cuestionario de habilidades sociales” (CHASO). Psicol Conductual. 2017;25:5-24. Specifically, individuals with SAD are more likely to be single, of a lower socioeconomic status, and undereducated.33. Grant BF, Hasin DS, Blanco C, Stinson FS, Chou SP, Goldstein RB, et al. (2005). The epidemiology of social anxiety disorder in the United States: results from the National Epidemiologic Survey on Alcohol and Related Conditions. J Clin Psychiatry. 2005;66:1351-61. The economic burden of this condition to both society and the individual is also significant.55. Gómez O, Casas C, Ortega-Ruiz R. Ansiedad social en la adolescencia: factores psicoevolutivos y de contexto familiar. Psicol Conductual. 2016;24:29-49. Because of its high prevalence and societal burden, SAD has attracted increasing attention from researchers in recent decades, leading to innovations in both the treatment and assessment of this condition. Numerous measures have been developed to assess this condition, ranging from semi-structured interviews to self-report inventories.
One of the most commonly used self-report SAD measures internationally is the Liebowitz Social Anxiety Scale (LSAS).66. Liebowitz MR. Social phobia. Mod Probl Pharmacopsychiatry. 1987;22:141-73. The original 24-item semi-structured interview involves a two-factor model with separate subscales to assess fear and avoidance of situations involving social interaction and performance/observation by others. However, the two-factor model has been shown to provide an inadequate fit of the data, and a self-report version (LSAS-SR) divided into four subscales was proposed instead.77. Beard C, Rodriguez BF, Moitra E, Sibrava NJ, Bjornsson A, Weisberg RB, et al. Psychometric properties of the Liebowitz Social Anxiety Scale (LSAS) in a longitudinal study of African Americans with anxiety disorders. J Anxiety Disord. 2011;25:722-6. 8. Oakman J, Van Ameringen M, Mancini C, Farvolden P. A confirmatory factor analysis of a self-report version of the Liebowitz Social Anxiety Scale. J Clin Psychol. 2003;59:149-61.-99. Safren SA, Heimberg RG, Horner KJ, Juster HR, Schneier FR, Liebowitz MR. Factor structure of social fears: the Liebowitz Social Anxiety Scale. J Anxiety Disord. 1999;13:253-70. This four-dimensional factor structure has been used in many studies.1010. Fresco DM, Coles ME, Heimberg RG, Liebowitz MR, Hami S, Stein MB, et al. The Liebowitz Social Anxiety Scale: a comparison of the psychometric properties of self-report and clinician-administered formats. Psychol Med. 2001;31:1025-35.,1111. Ha J, Lim SW, Shin YC, Oh KS. Comparison of anxiety-related traits between generalized and nongeneralized subtypes of social anxiety disorder. J Nerv Ment Dis. 2011;199:390-3. Nevertheless, different studies with various populations have detected a different number of factors – three,1212. Romm KL, Rossberg JI, Berg AO, Hansen CF, Andreassen OA, Melle I. Assessment of social anxiety in first episode psychosis using the Liebowitz Social Anxiety scale as a self-report measure. Eur Psychiatry. 2011;26:115-21. four,88. Oakman J, Van Ameringen M, Mancini C, Farvolden P. A confirmatory factor analysis of a self-report version of the Liebowitz Social Anxiety Scale. J Clin Psychol. 2003;59:149-61. five,1313. Baker SL, Heinrichs N, Kim H, Hofmann SG. The Liebowitz social anxiety scale as a self-report instrument: A preliminary psychometric analysis. Behav Res Ther. 2002;40:701-15. six,1414. Stein DJ, Kasper S, Andersen EW, Nil R, Lader M. Escitalopram in the treatment of social anxiety disorder: analysis of efficacy for different clinical subgroups and symptom dimensions. Depress Anxiety. 2004;20:175-81. or even eight.1515. Heeren A, Maurage P, Rossignol M, Vanhaelen M, Peschard V, Eeckhout C, et al. Self-report version of the Liebowitz Social Anxiety Scale: psychometric properties of the French version. Can J Behav Sci. 2012;44:99-107. These studies usually only factor-analyze the fear or anxiety subscale,88. Oakman J, Van Ameringen M, Mancini C, Farvolden P. A confirmatory factor analysis of a self-report version of the Liebowitz Social Anxiety Scale. J Clin Psychol. 2003;59:149-61.,1212. Romm KL, Rossberg JI, Berg AO, Hansen CF, Andreassen OA, Melle I. Assessment of social anxiety in first episode psychosis using the Liebowitz Social Anxiety scale as a self-report measure. Eur Psychiatry. 2011;26:115-21.,1313. Baker SL, Heinrichs N, Kim H, Hofmann SG. The Liebowitz social anxiety scale as a self-report instrument: A preliminary psychometric analysis. Behav Res Ther. 2002;40:701-15. because factor analysis of the fear and avoidance subscales produces similar results.1212. Romm KL, Rossberg JI, Berg AO, Hansen CF, Andreassen OA, Melle I. Assessment of social anxiety in first episode psychosis using the Liebowitz Social Anxiety scale as a self-report measure. Eur Psychiatry. 2011;26:115-21.,1414. Stein DJ, Kasper S, Andersen EW, Nil R, Lader M. Escitalopram in the treatment of social anxiety disorder: analysis of efficacy for different clinical subgroups and symptom dimensions. Depress Anxiety. 2004;20:175-81. Thus the fear or anxiety subscale is usually the only one analyzed.
Regarding the psychometric properties of the LSAS-SR, the literature has reported good test-retest reliability, adequate internal consistency, and adequate convergent and discriminant validity,1010. Fresco DM, Coles ME, Heimberg RG, Liebowitz MR, Hami S, Stein MB, et al. The Liebowitz Social Anxiety Scale: a comparison of the psychometric properties of self-report and clinician-administered formats. Psychol Med. 2001;31:1025-35.,1313. Baker SL, Heinrichs N, Kim H, Hofmann SG. The Liebowitz social anxiety scale as a self-report instrument: A preliminary psychometric analysis. Behav Res Ther. 2002;40:701-15. even in versions translated into other languages.1515. Heeren A, Maurage P, Rossignol M, Vanhaelen M, Peschard V, Eeckhout C, et al. Self-report version of the Liebowitz Social Anxiety Scale: psychometric properties of the French version. Can J Behav Sci. 2012;44:99-107. 16. Sugawara N, Yasui-Furukori N, Kaneda A, Sato Y, Tsuchimine S, Fujii A, et al. Factor structure of the Liebowitz Social Anxiety Scale in community-dwelling subjects in Japan. Psychiatry Clin Neurosci. 2012;66:525-8.-1717. Terra MB, Barros HM, Stein AT, Figueira I, Athayde LC, Gonçalves Mde S, et al. Internal consistency and factor structure of the Portuguese version of the Liebowitz Social Anxiety Scale among alcoholic patients. Rev Bras Psiquiatr. 2006;28:265-9. The LSAS-SR has also been used to establish the convergent validity of other self-report measures of social anxiety, such as the Social Phobia Inventory (SPIN),1818. Connor KM, Davidson JR, Churchill LE, Sherwood A, Foa E, Weisler RH. Psychometric properties of the Social Phobia Inventory (SPIN). New self-rating scale. Br J Psychiatry. 2000;176:379-86. the Social Phobia and Anxiety Inventory (SPAI),1919. Caballo VE, Salazar I, Irurtia MJ, Arias B, Nobre-Sandoval L. The assessment of social anxiety through five self-report measures, LSAS-SR, SPAI, SPIN, SPS, and SIAS: a critical analysis of their factor structure. Behav Psychol. 2013;21:423-48. the Social Interaction Anxiety Scale (SIAS) and the Social Phobia Scale (SPS),2020. de Beurs E, Tielen D, Wollmann L. The Dutch social interaction anxiety scale and the social phobia scale: reliability, validity, and clinical utility. Psychiatry J. 2014;2014:360193. and the short forms of the SIAS and the SPS.2121. Le Blanc AL, Bruce LC, Heimberg RG, Hope DA, Blanco C, Schneier FR, et al. Evaluation of the psychometric properties of two short forms of the social interaction anxiety scale and the social phobia scale. Assessment. 2014;21:312-23. Furthermore, it is commonly used to assess treatment outcomes in SAD patients receiving pharmacological treatment66. Liebowitz MR. Social phobia. Mod Probl Pharmacopsychiatry. 1987;22:141-73.,2222. Bhogal KS, Baldwin DS. Pharmacological treatment of social phobia. Psychiatry. 2007;6:217-23. or cognitive-behavioral therapy.2323. Smits JA, Rosenfield D, McDonald R, Telch MJ. Cognitive mechanisms of social anxiety reduction: an examination of specificity and temporality. J Consult Clin Psychol. 2006;74:1203-12.
Although the psychometric properties of the LSAS-SR have been examined in different countries, few studies have been conducted in Spanish or Portuguese-speaking countries.1717. Terra MB, Barros HM, Stein AT, Figueira I, Athayde LC, Gonçalves Mde S, et al. Internal consistency and factor structure of the Portuguese version of the Liebowitz Social Anxiety Scale among alcoholic patients. Rev Bras Psiquiatr. 2006;28:265-9.,1919. Caballo VE, Salazar I, Irurtia MJ, Arias B, Nobre-Sandoval L. The assessment of social anxiety through five self-report measures, LSAS-SR, SPAI, SPIN, SPS, and SIAS: a critical analysis of their factor structure. Behav Psychol. 2013;21:423-48.,2424. Bobes J, Badía X, Luque A, García M, González MP, Dal-Ré R. Validación de las versiones en español de los cuestionarios Liebowitz Social Anxiety Scale, Social Anxiety and Distress Scale y Sheehan Disability Inventory para la evaluación de la fobia social. Med. Clin. 1999;112:530-8.,2525. González MP, Bobes J, García M, Badía X, Luque A, Dal-Ré R. Assessing social phobia. The Spanish validation of the “gold standard” clinical scales: the LSAS and the SADS. Eur. Neuropsychopharmacol. 1998;8:S259-S60. The aim of this study was to examine the factor structure, invariance, internal consistency, and convergent validity of the LSAS-SR with clinical and non-clinical samples from Spain, Portugal, and 16 Latin American countries.
Methods
Participants
The first group of participants consisted of 31,243 non-clinical individuals (mean age [M] = 25.50 years, standard deviation [SD]= 10.13, range: 16-87 years) from 18 countries (22.68% Mexico, 18.19% Colombia, 12.30% Spain, 9.82% Peru, 7.75% Brazil, 3.84% Argentina, 3.35% Uruguay, 3.06% Venezuela, 1.75% Puerto Rico, 3.30% Portugal, 3.37% Chile, 1.91% Paraguay, 0.82% Costa Rica, 1.44% Honduras, 2.63% Bolivia, 1.92% El Salvador, 0.63% Dominican Republic, and 1.22% Guatemala). The sample included 56.71% women (M = 25.10 years, SD = 9.87) and 43.14% men (M = 26.00 years, SD = 10.43), with 0.15% of participants not reporting their gender. Regarding education and type of occupation, 58.89% were higher education students, 14.99% were workers with a college diploma, 9.58% were secondary education students, 8.69% were workers with no higher education, and 7.22% did not match any of the former categories (e.g., housewife, retired, or unemployed). No data on occupation were available for 0.63% of the participants.
The second group of participants consisted of 529 patients (M age = 31.73 years, SD = 11.96, range = 16-72) from 13 countries (164 Spain, 112 Mexico, 64 Brazil, 59 Argentina, 44 Colombia, 31 Peru, 27 Chile, 11 Portugal, 10 Uruguay, three Venezuela, two Bolivia, one Panama, and one Puerto Rico); there were 337 women (M = 32.61 years, SD = 12.14) and 192 men (M = 30.18 years, SD = 11.49). For inclusion in this group, patients had to meet a primary diagnosis of SAD according to the criteria of the DSM-IV-TR2626. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision (DSM-IV-TR). Arlington: American Psychiatric Publishing; 2000. or ICD-10.2727. World Health Organization (WHO). CIE-10 trastornos mentales y del comportamiento: descripciones clínicas y pautas para el diagnóstico. Madrid: Meditor; 1992. Each center conducted its own diagnostic assessment of individual patients. Patients with a DSM-IV-TR or ICD-10 diagnosis of SAD were included in the study even if they had other disorders in addition to SAD (Table 1).
A second inclusion criterion was a score ≥ 60 on the LSAS-SR.2828. Mennin DS, Fresco DM, Heimberg RG, Schneier FR, Davies SO, Liebowitz MR. Screening for social anxiety disorder in the clinical setting: using the Liebowitz Social Anxiety Scale. J Anxiety Disord. 2002;16:661-73. It should be noted that although Mennin et al. used a score of 60 as indicative of generalized social anxiety disorder (GSAD) and of 30 as non-GSAD (NSAD), a previous Brazilian study found scores between 52 and 81 for moderate phobia.1717. Terra MB, Barros HM, Stein AT, Figueira I, Athayde LC, Gonçalves Mde S, et al. Internal consistency and factor structure of the Portuguese version of the Liebowitz Social Anxiety Scale among alcoholic patients. Rev Bras Psiquiatr. 2006;28:265-9. Furthermore, it has been found that while 21% of a Spanish non-clinical sample scored higher than 60 on the LSAS-SR, this percentage rose to 68% with a cutoff score of 30.2929. Caballo VE, Arias B, Salazar IC, Irurtia MJ, Hofmann SG; CISO-A Research Team. Psychometric properties of an innovative self-report measure: the Social Anxiety Questionnaire for adults. Psychol Assess. 2015;27:997-1012. Given these results, a cutoff score of 60 was considered more appropriate than a score of 30 for the present sample.
Patients were excluded for several reasons (e.g., five or more unanswered items, presence of psychotic disorders, SAD not the primary diagnosis – which in fact was the main reason for exclusion). From a pool of 907 patients diagnosed with SAD, 529 satisfied all the inclusion criteria. Regarding occupation, 25.14% were workers with a college diploma, 20.60% were workers with no college diploma, 17.58% were higher education students from different majors, 7.75% were secondary education students, 1.89% were higher education psychology students, 0.76% were psychologists, and 23.63% did not match any of the former categories (e.g., housewife, retired or unemployed). No data on occupational status were obtained for the remaining 14 patients.
Measures
Liebowitz Social Anxiety Scale (LSAS)66. Liebowitz MR. Social phobia. Mod Probl Pharmacopsychiatry. 1987;22:141-73.
The LSAS is a 24-item interviewer-rated instrument that assesses fear/anxiety and avoidance of specific social situations. Each of the 24 items serves to assess both variables. Respondents are asked to rate their fear/anxiety (LSAS-anxiety subscale) on a four-point scale ranging from 0 (none) to 3 (severe), and avoidance (LSAS-avoidance subscale) on a four-point scale ranging from 0 (never) to 3 (usually). The total score is obtained by adding the scores obtained on both subscales. The LSAS has also been used as a self-report instrument (LSAS-SR) in the literature with these same characteristics.1010. Fresco DM, Coles ME, Heimberg RG, Liebowitz MR, Hami S, Stein MB, et al. The Liebowitz Social Anxiety Scale: a comparison of the psychometric properties of self-report and clinician-administered formats. Psychol Med. 2001;31:1025-35.,1313. Baker SL, Heinrichs N, Kim H, Hofmann SG. The Liebowitz social anxiety scale as a self-report instrument: A preliminary psychometric analysis. Behav Res Ther. 2002;40:701-15. With regard to the psychometric characteristics of the Spanish version, one study reported a four-factor structure2525. González MP, Bobes J, García M, Badía X, Luque A, Dal-Ré R. Assessing social phobia. The Spanish validation of the “gold standard” clinical scales: the LSAS and the SADS. Eur. Neuropsychopharmacol. 1998;8:S259-S60. and another study reported five-factors.1919. Caballo VE, Salazar I, Irurtia MJ, Arias B, Nobre-Sandoval L. The assessment of social anxiety through five self-report measures, LSAS-SR, SPAI, SPIN, SPS, and SIAS: a critical analysis of their factor structure. Behav Psychol. 2013;21:423-48. A Portuguese version of the scale showed a five-factor structure.1717. Terra MB, Barros HM, Stein AT, Figueira I, Athayde LC, Gonçalves Mde S, et al. Internal consistency and factor structure of the Portuguese version of the Liebowitz Social Anxiety Scale among alcoholic patients. Rev Bras Psiquiatr. 2006;28:265-9. Reported reliability indices are adequate.1919. Caballo VE, Salazar I, Irurtia MJ, Arias B, Nobre-Sandoval L. The assessment of social anxiety through five self-report measures, LSAS-SR, SPAI, SPIN, SPS, and SIAS: a critical analysis of their factor structure. Behav Psychol. 2013;21:423-48.,2424. Bobes J, Badía X, Luque A, García M, González MP, Dal-Ré R. Validación de las versiones en español de los cuestionarios Liebowitz Social Anxiety Scale, Social Anxiety and Distress Scale y Sheehan Disability Inventory para la evaluación de la fobia social. Med. Clin. 1999;112:530-8.,2525. González MP, Bobes J, García M, Badía X, Luque A, Dal-Ré R. Assessing social phobia. The Spanish validation of the “gold standard” clinical scales: the LSAS and the SADS. Eur. Neuropsychopharmacol. 1998;8:S259-S60. In this way, these last three studies have found scores for internal consistency (Cronbach’s alpha) of 0.93, 0,83, and 0,87, respectively, for the LSAS-anxiety subscale. Split-half reliability coefficient (Guttman) found has been 0.90,1919. Caballo VE, Salazar I, Irurtia MJ, Arias B, Nobre-Sandoval L. The assessment of social anxiety through five self-report measures, LSAS-SR, SPAI, SPIN, SPS, and SIAS: a critical analysis of their factor structure. Behav Psychol. 2013;21:423-48. and test-retest reliability 0.89.2525. González MP, Bobes J, García M, Badía X, Luque A, Dal-Ré R. Assessing social phobia. The Spanish validation of the “gold standard” clinical scales: the LSAS and the SADS. Eur. Neuropsychopharmacol. 1998;8:S259-S60.
Social Anxiety Questionnaire for Adults (SAQ)2929. Caballo VE, Arias B, Salazar IC, Irurtia MJ, Hofmann SG; CISO-A Research Team. Psychometric properties of an innovative self-report measure: the Social Anxiety Questionnaire for adults. Psychol Assess. 2015;27:997-1012.
30. Caballo VE, Salazar I, Arias B, Irurtia MJ, Calderero M, Graña JL, et al. Validation of the Social Anxiety Questionnaire for Adults (SAQ-A30) with Spanish university students: similarities and differences among degree subjects and regions. Behav Psychol. 2010;18:5-34.-3131. Caballo VE, Salazar IC, Irurtia MJ, Arias B, Hofmann SG; CISO-A Research Team. The multidimensional nature and multicultural validity of a new measure of social anxiety: the Social Anxiety Questionnaire for adults. Behav Ther. 2012;43:313-28.
The SAQ is a 30-item self-reported questionnaire which was empirically developed in Spanish and Portuguese speaking countries to assesses five dimensions of social anxiety: 1) speaking in public/talking with people in authority; 2) interactions with the opposite sex; 3) assertive expression of annoyance, disgust, or displeasure; 4) criticism and embarrassment; and 5) interactions with strangers. Each item is answered on a five-point Likert scale to indicate the level of unease, stress, or nervousness in response to each social situation: 1 = not at all or very slight; 2 = slight; 3 = moderate; 4 = high; and 5 = very high or extremely high. Cronbach’s alpha for the total scale has been shown to range from 0.88 to 0.93,2929. Caballo VE, Arias B, Salazar IC, Irurtia MJ, Hofmann SG; CISO-A Research Team. Psychometric properties of an innovative self-report measure: the Social Anxiety Questionnaire for adults. Psychol Assess. 2015;27:997-1012. 30. Caballo VE, Salazar I, Arias B, Irurtia MJ, Calderero M, Graña JL, et al. Validation of the Social Anxiety Questionnaire for Adults (SAQ-A30) with Spanish university students: similarities and differences among degree subjects and regions. Behav Psychol. 2010;18:5-34.-3131. Caballo VE, Salazar IC, Irurtia MJ, Arias B, Hofmann SG; CISO-A Research Team. The multidimensional nature and multicultural validity of a new measure of social anxiety: the Social Anxiety Questionnaire for adults. Behav Ther. 2012;43:313-28. with split-half reliability coefficients (Guttman) ranging from 0.90 to 0.93.3232. Salazar IC. Fiabilidad y validez de una nueva medida de autoinforme para la evaluación de la ansiedad/fobia social en adultos [dissertation]. Granada: Universidad de Granada; 2013. Regarding the five dimensions, Cronbach’s alpha ranged from 0.74 to 0.90,2929. Caballo VE, Arias B, Salazar IC, Irurtia MJ, Hofmann SG; CISO-A Research Team. Psychometric properties of an innovative self-report measure: the Social Anxiety Questionnaire for adults. Psychol Assess. 2015;27:997-1012. 30. Caballo VE, Salazar I, Arias B, Irurtia MJ, Calderero M, Graña JL, et al. Validation of the Social Anxiety Questionnaire for Adults (SAQ-A30) with Spanish university students: similarities and differences among degree subjects and regions. Behav Psychol. 2010;18:5-34.-3131. Caballo VE, Salazar IC, Irurtia MJ, Arias B, Hofmann SG; CISO-A Research Team. The multidimensional nature and multicultural validity of a new measure of social anxiety: the Social Anxiety Questionnaire for adults. Behav Ther. 2012;43:313-28. with split-half reliability coefficients (Guttman) ranging from 0.57 to 0.95.3232. Salazar IC. Fiabilidad y validez de una nueva medida de autoinforme para la evaluación de la ansiedad/fobia social en adultos [dissertation]. Granada: Universidad de Granada; 2013.
Procedure
Our CISO-A Research Team consists of researchers and psychologists from Spain, Portugal, and most Latin American countries. The LSAS-SR and the SAQ were administered to 529 patients with SAD and to 31,243 non-clinical participants from the community. Given that we did not find significant differences among Spain, Portugal, and most Latin American countries regarding assessment of social anxiety,2929. Caballo VE, Arias B, Salazar IC, Irurtia MJ, Hofmann SG; CISO-A Research Team. Psychometric properties of an innovative self-report measure: the Social Anxiety Questionnaire for adults. Psychol Assess. 2015;27:997-1012.,3131. Caballo VE, Salazar IC, Irurtia MJ, Arias B, Hofmann SG; CISO-A Research Team. The multidimensional nature and multicultural validity of a new measure of social anxiety: the Social Anxiety Questionnaire for adults. Behav Ther. 2012;43:313-28. we grouped all these countries together for analysis. For the assessment of the clinical group, our collaborators administered the two questionnaires (LSAS-SR and SAQ) individually to patients. For the non-clinical sample, the questionnaires were administered to groups of subjects. Collaborators working in high schools, colleges, or universities administered the questionnaires to people in classes and meetings of teachers or professors. Those working in companies convened voluntary meetings for workers. No compensation was provided to participants.
The two questionnaires were sent to each collaborator and in order to derive the Portuguese version, both questionnaires were translated and back-translated from Portuguese to English (LSAS-SR) or Spanish (SAQ) until agreement was reached between translators. Both questionnaires were administered together, but the order of administration was random.
Ethical considerations
Participation in the study was voluntary and the questionnaires were filled out anonymously. Informed verbal consent was obtained from all respondents, who were free to withdraw at any time or to refuse to answer the questionnaires. The study and all its procedures were approved by Spain’s Ministry of Science and Technology. This study does not break the agreements of the Helsinki Declaration.
Data analysis
To cross-validate the factor structure of the LSAS-SR, the total sample of 31,243 individuals was randomly split in two halves (n1 =15,566; n2 =15,677). There were missing data in 2.19% of the responses to the LSAS-SR. Given the low rate of missing data, and also the absence of evidence incompatible with a missing completely at random structure, pairwise deletion was used to handle the missing data.3333. Brown TA. Confirmatory factor analysis for applied research. New York: Guilford; 2006.,3434. Schafer JL, Graham JW. Missing data: our view of the state of the art. Psychol Methods. 2002;7:147-77.
First, we conducted an exploratory analysis with parallel analysis and exploratory factor analysis (EFA) to examine the internal structure of the LSAS-SR. Parallel analysis was implemented with LSAS-SR anxiety subscale data from subsample 1 using the Monte Carlo procedure with 1,000 replications. Parallel analyses compared the eigenvalues extracted from the observed correlation matrix to be analyzed with the eigenvalues obtained from uncorrelated normal variables (parallel components derived from random data). EFA (unweighted least squares with direct oblimin oblique rotation)3535. Osborne JW. What is rotating in exploratory factor analysis? [Internet]. 2015 [cited 2018 01 30]. pareonline.net/getvn.asp?v=20&n=2
pareonline.net/getvn.asp?v=20&n=2...
was computed on the first subsample of non-clinical participants. We conducted separate EFAs on the fear/anxiety and the avoidance items. Because we obtained similar results with both subscales and because their distinctiveness has been questioned,88. Oakman J, Van Ameringen M, Mancini C, Farvolden P. A confirmatory factor analysis of a self-report version of the Liebowitz Social Anxiety Scale. J Clin Psychol. 2003;59:149-61.,1919. Caballo VE, Salazar I, Irurtia MJ, Arias B, Nobre-Sandoval L. The assessment of social anxiety through five self-report measures, LSAS-SR, SPAI, SPIN, SPS, and SIAS: a critical analysis of their factor structure. Behav Psychol. 2013;21:423-48. we decided to continue the analysis with only the fear/anxiety subscale. We also conducted an EFA with the clinical sample.
Then, we tested the factor structure of the LSAS-SR anxiety subscale unveiled by EFA results using confirmatory factor analysis (CFA)3636. Brown TA. Confirmatory factor analysis for applied research. New York: Guilford; 2014. and exploratory structural equation modeling (ESEM).3737. Asparouhov T, Muthén B. Exploratory structural equation modeling (ESEM). Struct Equ Modeling. 2009;16:397-438. ESEM models have been recently developed to address a common limitation of CFA models, which often produce overly restrictive measurement models that do not provide acceptable goodness of fit for most psychological instruments.3838. Marsh HW, Morin AJ, Parker PD, Kaur G. Exploratory structural equation modeling: an integration of the best features of exploratory and confirmatory factor analysis. Annu Rev Clin Psychol. 2014;10:85-110. The ESEM model is a special case of CFA in which the assumption that the cross-loadings are 0 is relaxed, so that both models can be considered nested and their fit comparable.3737. Asparouhov T, Muthén B. Exploratory structural equation modeling (ESEM). Struct Equ Modeling. 2009;16:397-438. The CFA and ESEM were completed with the second non-clinical subsample of participants (n2 =15,677) using weighted least squares with adjusted means and variances (WLSMV) estimation. Four models were tested: M1 = unifactorial model; M2 = two correlated factors model (social interaction and performance situations of the original model); M3 = four-correlated factors model (social interaction, public speaking, observation by others, and eating and drinking in public of the Safren’s model); M4 = five-correlated factors model; and M5 = ESEM. These same five models were also examined in the clinical sample.
To appraise overall model fit, a number of fit indices were examined, including the root mean square error of approximation (RMSEA), comparative fit index (CFI), and Tucker-Lewis index (TLI). Values of the RMSEA exceeding 0.07 indicate poor fit, whereas values ≥ 0.90 would indicate acceptable fit for the CFI and TLI.
Subsequently, we estimated the correlations among the factors of the LSAS-SR, internal consistency (Cronbach’s alpha), and the convergent validity of the instrument with another self-report measure of social anxiety (i.e., the SAQ) for the non-clinical and clinical samples separately.
In the case of invariance by country, and given the large number of groups involved, Multiple Indicators Multiple Causes models (MIMIC)3939. Jöreskog KG, Goldberger AS. Estimation of a model with multiple indicators and multiple causes of a single latent variable. J Am Stat Assoc. 1975;70:631-9. were used, according to the procedure described in a former study.4040. Morin AJ, Arens AK, Marsh HW. A bifactor exploratory structural equation modeling framework for the identification of distinct sources of construct-relevant psychometric multidimensionality. Struct Equ Modeling. 2016;23:116-39. In a MIMIC model, the grouping variable (previously dummy-coded) acts as a predictor of either the latent or the observed variables. Three nested MIMIC models were estimated:
-
A null model (M10) in which the paths between the grouping variable and the other terms of the model were set to 0 (zero); M10 reflects the null hypothesis that the country of origin has no effect either on thresholds (i.e., scalar invariance holds) or on latent variables (i.e., there is no country-dependent differences in LSAS-SR scores).
-
An invariant model (M11) in which the paths leading from the grouping variable to the factors were freely estimated, setting the paths to the observable variables at 0 (zero).
-
A saturated model (M12) in which the paths leading to factors were set to 0 (zero), but the paths from the grouping variable to all observable indicators were freely estimated (i.e., hypothesizing the non-invariance of measurement relative to the country of origin).
Once the three models were estimated, M10 was compared with M11 and M12. Since M11 and M12 are more parameterized than M10, they should tend to a better fit. However, if the M10 fit is not substantially worse than that of the other models, the hypothesis that the country has negligible influence on the measurement model would not be rejected. Conversely, if the M12 fit is substantially better than that of M10 and M11, possibly certain items would be at risk of differential functioning.
Finally, we analyzed the mean differences between clinical and non-clinical samples in the factors and subscales of the LSAS-SR, reporting their effect sizes (Cohen's d), as well as the differences in latent means expressed in SDs of the clinical group from the general group, as estimated from the scalar invariance model. These differences are directly interpretable as a Cohen's d.3636. Brown TA. Confirmatory factor analysis for applied research. New York: Guilford; 2014.
All statistical analyses were performed using Statistica version 12,4141. Statistica [computer program]. Version 12. Tulsa: StatSoft, Inc.; 2013. SPSS version 22,4242. IBM SPSS Statistics [computer program]. Version 22.0. Chicago: IBM; 2013. and MPlus version 7.4.4343. MPlus [computer software]. Version 7.4. Los Angeles: Muthén & Muthén; 2015.
Results
Exploratory and CFA of the LSAS-SR
The results of parallel analyses of subsample 1 with the LSAS-SR anxiety subscale showed that the five-factor solution was the best fit for the data, given that only the eigenvalues of these five factors were greater than the randomly generated eigenvalues.
Furthermore, the EFA with the LSAS-SR anxiety subscale identified five factors with eigenvalues greater than 1.00, explaining 50.78% of the cumulative variance. The five factors were the following: 1) speaking in public (six items); 2) eating/drinking in front of other people (four items); 3) assertive behaviors (four items); 4) working/writing while being observed (two items); and 5) interactions with strangers (five items) (Table 2). All items loaded above 0.40 on only one factor except for items 7 and 13, which loaded above 0.40 on two factors. Thus, items 7 and 13 were assigned to factors 2 and 3, respectively, because those are the individual factors with which these items were most strongly associated. Items 1, 14, and 18 did not load above 0.40 on any factor. In order to empirically test the redundant contribution of the avoidance subscale, an EFA was also performed. We identified the same five factors explaining 48.58% of the cumulative variance. Eighteen items from the EFA of this subscale loaded above 0.40 on the same factors as the anxiety subscale. Item 7 loaded strongly on two factors and item 1 did not load above 0.40 on any factor.
The best factor solution for the clinical sample based on the scree test was again a five-factor structure with eigenvalues greater than 1.00, explaining 47.23% of the cumulative variance. The first factor, interactions with strangers (eigenvalue: 5.91, items: 1, 10, 11, 12), explained 24.61% of the variance. The second factor, speaking in public (eigenvalue: 1.84, items: 5, 6, 15, 16, 20), explained 7.68% of the total variance. Factor 3, working/writing while being observed (eigenvalue: 1.52, items: 8, 9), explained 6.33% of the variance. Factor 4, assertive behaviors (eigenvalue: 1.41, items: 13, 17, 18, 22, 24), explained 5.89% of the variance. Finally, factor 5, eating/drinking in front of other people (eigenvalue: 1.32, items: 2, 3, 4, 7), explained 5.51% of the variance. All items loaded above 0.40 on only one factor. Items 14, 19, and 21 did not load above 0.40 on any factor. Seventeen of the 24 items loaded on the same factors as observed for non-clinical subsample 1.
Table 3 shows the results obtained with the CFA of non-clinical subsample 2 and ESEM of subsample 1. The fit indices corroborated the five correlated factor model (M4) as the best fitting model. The fit of the ESEM model was better than that of the CFA, although this result can be expected given that the ESEM is a more parameterized model (with 64 degrees of freedom less than the equivalent CFA model). Table 3 also displays the results of these analyses in the clinical sample. M4 also received the most support as the best fitting model in the clinical sample.
Correlations among LSAS-SR subscales and factors
The LSAS-SR total score was highly correlated with the anxiety and avoidance subscales (non-clinical sample, 0.93 and 0.93; clinical sample, 0.91 and 0.93 respectively). Anxiety and avoidance subscales were also highly correlated (r non-clinical = 0.73: r clinical = 0.70).
Correlations among the five factors obtained with the LSAS-SR in the non-clinical sample were moderate (from 0.37 to 0.58) and higher than those in the clinical sample, which also ranged from moderate to low (from 0.27 to 0.52).
Internal consistency of the LSAS-SR
Alpha coefficients for the LSAS-SR were high, ranging from 0.93 to 0.88 for the two subscales and the total score in the non-clinical sample and from 0.89 to 0.83 in the clinical sample. Alphas for the five factors were lower, ranging from 0.80 to 0.58 in the non-clinical sample and from 0.63 to 0.43 in the clinical sample. These results do not seem to support the consistency of some of the factors of the LSAS-SR in our sample. Only two factors had an alpha higher than 0.60 in clinical and non-clinical samples, interacting with strangers and speaking in public, the two factors with the highest correlations with similar dimensions of the SAQ (Table 4).
Invariance by country of origin
Regarding invariance by country of origin (Table 5), the null model (M10) presented better fit than both the invariant (ΔRMSEA = 0.008, ΔCFI = -0.014, ΔTLI = -0.018) and the saturated models (ΔRMSEA = 0.01, ΔCFI = -0.013, ΔTLI = -0.023). This suggests that: 1) the measurement model was invariant with respect to the country of origin; and 2) the latent mean differences between countries were not substantial. This was verified by the inspection of the parameters of M11 and M12. The standardized paths from the cluster variable to both the factors and items were close to zero (range 0.04-0.07 in M11, and range 0.00-0.011 in M12).
Differences between clinical and non-clinical samples
Mean differences and their effect size were calculated between clinical (n=529) and non-clinical (n=31,243) samples in all the possible subscales of the LSAS-SR (Table 6). As can be seen, the effect sizes of the differences between these two samples are consistently large (Cohen’s d > 0.80).
Convergent validity
Correlations between the LSAS-SR factors and the SAQ dimensions assessing similar constructs exhibited the highest associations (i.e., speaking in public, interacting with strangers) whereas the factor assertive behaviors had only a modest relationship with dimensions of the SAQ. The other two factors (i.e., eating/drinking in front of other people and working/writing while being observed) were not strongly associated with dimensions of the SAQ (Table 4).
Discussion
The aim of this study was to examine the psychometric characteristics of the LSAS-SR in a large cross-cultural sample, comprising clinical and non-clinical subjects from Spain, Portugal, and most Latin American countries.
Specifically, we investigated the factor structure of the LSAS-SR using exploratory analysis and CFA. The results indicate a five-factor structure as the best model in both clinical and non-clinical samples. Safren’s four-factor model1313. Baker SL, Heinrichs N, Kim H, Hofmann SG. The Liebowitz social anxiety scale as a self-report instrument: A preliminary psychometric analysis. Behav Res Ther. 2002;40:701-15. exhibited good fit in the non-clinical sample, but not in the clinical sample. Although the four-factor model has been frequently used,77. Beard C, Rodriguez BF, Moitra E, Sibrava NJ, Bjornsson A, Weisberg RB, et al. Psychometric properties of the Liebowitz Social Anxiety Scale (LSAS) in a longitudinal study of African Americans with anxiety disorders. J Anxiety Disord. 2011;25:722-6.,88. Oakman J, Van Ameringen M, Mancini C, Farvolden P. A confirmatory factor analysis of a self-report version of the Liebowitz Social Anxiety Scale. J Clin Psychol. 2003;59:149-61.,1515. Heeren A, Maurage P, Rossignol M, Vanhaelen M, Peschard V, Eeckhout C, et al. Self-report version of the Liebowitz Social Anxiety Scale: psychometric properties of the French version. Can J Behav Sci. 2012;44:99-107. it has been disputed.1313. Baker SL, Heinrichs N, Kim H, Hofmann SG. The Liebowitz social anxiety scale as a self-report instrument: A preliminary psychometric analysis. Behav Res Ther. 2002;40:701-15.,1616. Sugawara N, Yasui-Furukori N, Kaneda A, Sato Y, Tsuchimine S, Fujii A, et al. Factor structure of the Liebowitz Social Anxiety Scale in community-dwelling subjects in Japan. Psychiatry Clin Neurosci. 2012;66:525-8. The single-factor or two-factor (performance and social interaction) models had poor fit, which is consistent with the results of prior research.77. Beard C, Rodriguez BF, Moitra E, Sibrava NJ, Bjornsson A, Weisberg RB, et al. Psychometric properties of the Liebowitz Social Anxiety Scale (LSAS) in a longitudinal study of African Americans with anxiety disorders. J Anxiety Disord. 2011;25:722-6.,99. Safren SA, Heimberg RG, Horner KJ, Juster HR, Schneier FR, Liebowitz MR. Factor structure of social fears: the Liebowitz Social Anxiety Scale. J Anxiety Disord. 1999;13:253-70.,1515. Heeren A, Maurage P, Rossignol M, Vanhaelen M, Peschard V, Eeckhout C, et al. Self-report version of the Liebowitz Social Anxiety Scale: psychometric properties of the French version. Can J Behav Sci. 2012;44:99-107. It appears that for this specific group of Spanish and Portuguese-speaking countries, the five-factor model is the best solution for the LSAS-SR, a finding that is consistent with other studies with Spanish-speaking university students.1919. Caballo VE, Salazar I, Irurtia MJ, Arias B, Nobre-Sandoval L. The assessment of social anxiety through five self-report measures, LSAS-SR, SPAI, SPIN, SPS, and SIAS: a critical analysis of their factor structure. Behav Psychol. 2013;21:423-48. Although some studies1010. Fresco DM, Coles ME, Heimberg RG, Liebowitz MR, Hami S, Stein MB, et al. The Liebowitz Social Anxiety Scale: a comparison of the psychometric properties of self-report and clinician-administered formats. Psychol Med. 2001;31:1025-35.,1111. Ha J, Lim SW, Shin YC, Oh KS. Comparison of anxiety-related traits between generalized and nongeneralized subtypes of social anxiety disorder. J Nerv Ment Dis. 2011;199:390-3. have used the original theoretical structure of the LSAS-SR (anxiety and avoidance subscales, including performance and social interaction sections in each of them), the empirically derived factor structure obtained for this scale appears to be very different.88. Oakman J, Van Ameringen M, Mancini C, Farvolden P. A confirmatory factor analysis of a self-report version of the Liebowitz Social Anxiety Scale. J Clin Psychol. 2003;59:149-61.,99. Safren SA, Heimberg RG, Horner KJ, Juster HR, Schneier FR, Liebowitz MR. Factor structure of social fears: the Liebowitz Social Anxiety Scale. J Anxiety Disord. 1999;13:253-70.,1212. Romm KL, Rossberg JI, Berg AO, Hansen CF, Andreassen OA, Melle I. Assessment of social anxiety in first episode psychosis using the Liebowitz Social Anxiety scale as a self-report measure. Eur Psychiatry. 2011;26:115-21. 13. Baker SL, Heinrichs N, Kim H, Hofmann SG. The Liebowitz social anxiety scale as a self-report instrument: A preliminary psychometric analysis. Behav Res Ther. 2002;40:701-15.-1414. Stein DJ, Kasper S, Andersen EW, Nil R, Lader M. Escitalopram in the treatment of social anxiety disorder: analysis of efficacy for different clinical subgroups and symptom dimensions. Depress Anxiety. 2004;20:175-81.,1717. Terra MB, Barros HM, Stein AT, Figueira I, Athayde LC, Gonçalves Mde S, et al. Internal consistency and factor structure of the Portuguese version of the Liebowitz Social Anxiety Scale among alcoholic patients. Rev Bras Psiquiatr. 2006;28:265-9.,1919. Caballo VE, Salazar I, Irurtia MJ, Arias B, Nobre-Sandoval L. The assessment of social anxiety through five self-report measures, LSAS-SR, SPAI, SPIN, SPS, and SIAS: a critical analysis of their factor structure. Behav Psychol. 2013;21:423-48.,4444. Perugi G, Nassini S, Maremmani I, Madaro D, Toni C, Simonini E, et al. Putative clinical subtypes of social phobia: a factor-analytical study. Acta Psychiatr Scand. 2001;104:280-8. The findings of previous studies, of three to six factor structures, attest to considerable individual variability in the possible factor structure of the LSAS-SR.
The most commonly identified factors of the LSAS-SR have been speaking in public, eating/drinking in front of other people (seven studies),88. Oakman J, Van Ameringen M, Mancini C, Farvolden P. A confirmatory factor analysis of a self-report version of the Liebowitz Social Anxiety Scale. J Clin Psychol. 2003;59:149-61.,99. Safren SA, Heimberg RG, Horner KJ, Juster HR, Schneier FR, Liebowitz MR. Factor structure of social fears: the Liebowitz Social Anxiety Scale. J Anxiety Disord. 1999;13:253-70.,1313. Baker SL, Heinrichs N, Kim H, Hofmann SG. The Liebowitz social anxiety scale as a self-report instrument: A preliminary psychometric analysis. Behav Res Ther. 2002;40:701-15. 14. Stein DJ, Kasper S, Andersen EW, Nil R, Lader M. Escitalopram in the treatment of social anxiety disorder: analysis of efficacy for different clinical subgroups and symptom dimensions. Depress Anxiety. 2004;20:175-81.-1515. Heeren A, Maurage P, Rossignol M, Vanhaelen M, Peschard V, Eeckhout C, et al. Self-report version of the Liebowitz Social Anxiety Scale: psychometric properties of the French version. Can J Behav Sci. 2012;44:99-107.,1717. Terra MB, Barros HM, Stein AT, Figueira I, Athayde LC, Gonçalves Mde S, et al. Internal consistency and factor structure of the Portuguese version of the Liebowitz Social Anxiety Scale among alcoholic patients. Rev Bras Psiquiatr. 2006;28:265-9.,4444. Perugi G, Nassini S, Maremmani I, Madaro D, Toni C, Simonini E, et al. Putative clinical subtypes of social phobia: a factor-analytical study. Acta Psychiatr Scand. 2001;104:280-8. interactions with strangers (usually included as a minor part of the social interaction factor) (seven studies),88. Oakman J, Van Ameringen M, Mancini C, Farvolden P. A confirmatory factor analysis of a self-report version of the Liebowitz Social Anxiety Scale. J Clin Psychol. 2003;59:149-61.,1212. Romm KL, Rossberg JI, Berg AO, Hansen CF, Andreassen OA, Melle I. Assessment of social anxiety in first episode psychosis using the Liebowitz Social Anxiety scale as a self-report measure. Eur Psychiatry. 2011;26:115-21. 13. Baker SL, Heinrichs N, Kim H, Hofmann SG. The Liebowitz social anxiety scale as a self-report instrument: A preliminary psychometric analysis. Behav Res Ther. 2002;40:701-15. 14. Stein DJ, Kasper S, Andersen EW, Nil R, Lader M. Escitalopram in the treatment of social anxiety disorder: analysis of efficacy for different clinical subgroups and symptom dimensions. Depress Anxiety. 2004;20:175-81.-1515. Heeren A, Maurage P, Rossignol M, Vanhaelen M, Peschard V, Eeckhout C, et al. Self-report version of the Liebowitz Social Anxiety Scale: psychometric properties of the French version. Can J Behav Sci. 2012;44:99-107.,1717. Terra MB, Barros HM, Stein AT, Figueira I, Athayde LC, Gonçalves Mde S, et al. Internal consistency and factor structure of the Portuguese version of the Liebowitz Social Anxiety Scale among alcoholic patients. Rev Bras Psiquiatr. 2006;28:265-9.,4444. Perugi G, Nassini S, Maremmani I, Madaro D, Toni C, Simonini E, et al. Putative clinical subtypes of social phobia: a factor-analytical study. Acta Psychiatr Scand. 2001;104:280-8. working/writing while being observed (usually included as a minor part of the observation or non-verbal performance factors) (seven studies)88. Oakman J, Van Ameringen M, Mancini C, Farvolden P. A confirmatory factor analysis of a self-report version of the Liebowitz Social Anxiety Scale. J Clin Psychol. 2003;59:149-61.,99. Safren SA, Heimberg RG, Horner KJ, Juster HR, Schneier FR, Liebowitz MR. Factor structure of social fears: the Liebowitz Social Anxiety Scale. J Anxiety Disord. 1999;13:253-70.,1212. Romm KL, Rossberg JI, Berg AO, Hansen CF, Andreassen OA, Melle I. Assessment of social anxiety in first episode psychosis using the Liebowitz Social Anxiety scale as a self-report measure. Eur Psychiatry. 2011;26:115-21. 13. Baker SL, Heinrichs N, Kim H, Hofmann SG. The Liebowitz social anxiety scale as a self-report instrument: A preliminary psychometric analysis. Behav Res Ther. 2002;40:701-15. 14. Stein DJ, Kasper S, Andersen EW, Nil R, Lader M. Escitalopram in the treatment of social anxiety disorder: analysis of efficacy for different clinical subgroups and symptom dimensions. Depress Anxiety. 2004;20:175-81.-1515. Heeren A, Maurage P, Rossignol M, Vanhaelen M, Peschard V, Eeckhout C, et al. Self-report version of the Liebowitz Social Anxiety Scale: psychometric properties of the French version. Can J Behav Sci. 2012;44:99-107.,4444. Perugi G, Nassini S, Maremmani I, Madaro D, Toni C, Simonini E, et al. Putative clinical subtypes of social phobia: a factor-analytical study. Acta Psychiatr Scand. 2001;104:280-8. and assertive behaviors (two studies).1313. Baker SL, Heinrichs N, Kim H, Hofmann SG. The Liebowitz social anxiety scale as a self-report instrument: A preliminary psychometric analysis. Behav Res Ther. 2002;40:701-15.,1414. Stein DJ, Kasper S, Andersen EW, Nil R, Lader M. Escitalopram in the treatment of social anxiety disorder: analysis of efficacy for different clinical subgroups and symptom dimensions. Depress Anxiety. 2004;20:175-81. However, some studies include a major social interaction factor that is a mixture of heterogeneous, and sometimes unrelated, items.88. Oakman J, Van Ameringen M, Mancini C, Farvolden P. A confirmatory factor analysis of a self-report version of the Liebowitz Social Anxiety Scale. J Clin Psychol. 2003;59:149-61.,99. Safren SA, Heimberg RG, Horner KJ, Juster HR, Schneier FR, Liebowitz MR. Factor structure of social fears: the Liebowitz Social Anxiety Scale. J Anxiety Disord. 1999;13:253-70.,1212. Romm KL, Rossberg JI, Berg AO, Hansen CF, Andreassen OA, Melle I. Assessment of social anxiety in first episode psychosis using the Liebowitz Social Anxiety scale as a self-report measure. Eur Psychiatry. 2011;26:115-21.,1313. Baker SL, Heinrichs N, Kim H, Hofmann SG. The Liebowitz social anxiety scale as a self-report instrument: A preliminary psychometric analysis. Behav Res Ther. 2002;40:701-15.,1717. Terra MB, Barros HM, Stein AT, Figueira I, Athayde LC, Gonçalves Mde S, et al. Internal consistency and factor structure of the Portuguese version of the Liebowitz Social Anxiety Scale among alcoholic patients. Rev Bras Psiquiatr. 2006;28:265-9.,4444. Perugi G, Nassini S, Maremmani I, Madaro D, Toni C, Simonini E, et al. Putative clinical subtypes of social phobia: a factor-analytical study. Acta Psychiatr Scand. 2001;104:280-8. Our interactions with strangers factor includes a smaller, but much more homogeneous, number of items, usually found under the social interaction factor in these former studies. Generally speaking, we can say that four of the five factors found in our study are also frequently found in many studies regarding the factor structure of the LSAS-SR, but the items composing the four factors of our study do not match the items composing the same four factors found in any of the studies reviewed for the present work. It seems that the selected Spanish and Portuguese-speaking countries have roughly the same dimensions of social anxiety as English speaking countries, but the behaviors through which these dimensions are expressed are partially different (only 15 out of 24 items loaded in the same factors found in the literature).
When comparing the factors of the LSAS-SR with the SAQ, a relatively new measure of social anxiety, empirically developed from 16 Latin American countries, Spain, and Portugal,2929. Caballo VE, Arias B, Salazar IC, Irurtia MJ, Hofmann SG; CISO-A Research Team. Psychometric properties of an innovative self-report measure: the Social Anxiety Questionnaire for adults. Psychol Assess. 2015;27:997-1012. 30. Caballo VE, Salazar I, Arias B, Irurtia MJ, Calderero M, Graña JL, et al. Validation of the Social Anxiety Questionnaire for Adults (SAQ-A30) with Spanish university students: similarities and differences among degree subjects and regions. Behav Psychol. 2010;18:5-34.-3131. Caballo VE, Salazar IC, Irurtia MJ, Arias B, Hofmann SG; CISO-A Research Team. The multidimensional nature and multicultural validity of a new measure of social anxiety: the Social Anxiety Questionnaire for adults. Behav Ther. 2012;43:313-28. some questions emerge regarding the validity of the LSAS-SR. Speaking in public and interactions with strangers seem to be dimensions of social anxiety that are well represented in the LSAS-SR (i.e., have good correlations with similar dimensions of the SAQ). The low correlation between the LSAS-SR and SAQ factors that are supposed to measure assertive behaviors could indicate that these factors reflect different constructs in each instrument. Finally, two LSAS-SR factors are not associated with any of the SAQ factors (i.e., eating/drinking in front of other people and working/writing while being observed). It might be that situations such as drinking in public places, writing while being observed or urinating in a public bathroom may be specific to US samples and less relevant for SAD patients from Latin American countries.2929. Caballo VE, Arias B, Salazar IC, Irurtia MJ, Hofmann SG; CISO-A Research Team. Psychometric properties of an innovative self-report measure: the Social Anxiety Questionnaire for adults. Psychol Assess. 2015;27:997-1012. 30. Caballo VE, Salazar I, Arias B, Irurtia MJ, Calderero M, Graña JL, et al. Validation of the Social Anxiety Questionnaire for Adults (SAQ-A30) with Spanish university students: similarities and differences among degree subjects and regions. Behav Psychol. 2010;18:5-34.-3131. Caballo VE, Salazar IC, Irurtia MJ, Arias B, Hofmann SG; CISO-A Research Team. The multidimensional nature and multicultural validity of a new measure of social anxiety: the Social Anxiety Questionnaire for adults. Behav Ther. 2012;43:313-28. Furthermore, items such as telephoning in public (1), entering a room when others are already seated (14), and expressing disagreement or disapproval to people you don't know very well (18) were not included in any factor of the LSAS-SR for the present Spanish and Portuguese-speaking samples. Item 14, the most problematic, has also been omitted from the factor solution in other studies.99. Safren SA, Heimberg RG, Horner KJ, Juster HR, Schneier FR, Liebowitz MR. Factor structure of social fears: the Liebowitz Social Anxiety Scale. J Anxiety Disord. 1999;13:253-70.,1414. Stein DJ, Kasper S, Andersen EW, Nil R, Lader M. Escitalopram in the treatment of social anxiety disorder: analysis of efficacy for different clinical subgroups and symptom dimensions. Depress Anxiety. 2004;20:175-81.,1919. Caballo VE, Salazar I, Irurtia MJ, Arias B, Nobre-Sandoval L. The assessment of social anxiety through five self-report measures, LSAS-SR, SPAI, SPIN, SPS, and SIAS: a critical analysis of their factor structure. Behav Psychol. 2013;21:423-48. This is also the case for item 181414. Stein DJ, Kasper S, Andersen EW, Nil R, Lader M. Escitalopram in the treatment of social anxiety disorder: analysis of efficacy for different clinical subgroups and symptom dimensions. Depress Anxiety. 2004;20:175-81.,1919. Caballo VE, Salazar I, Irurtia MJ, Arias B, Nobre-Sandoval L. The assessment of social anxiety through five self-report measures, LSAS-SR, SPAI, SPIN, SPS, and SIAS: a critical analysis of their factor structure. Behav Psychol. 2013;21:423-48. and item 1.99. Safren SA, Heimberg RG, Horner KJ, Juster HR, Schneier FR, Liebowitz MR. Factor structure of social fears: the Liebowitz Social Anxiety Scale. J Anxiety Disord. 1999;13:253-70.,1414. Stein DJ, Kasper S, Andersen EW, Nil R, Lader M. Escitalopram in the treatment of social anxiety disorder: analysis of efficacy for different clinical subgroups and symptom dimensions. Depress Anxiety. 2004;20:175-81. Finally, items such as urinating in a public bathroom (13), taking a test (17), trying to pick up someone (21) or giving a party (23) appear to load onto multiple factors99. Safren SA, Heimberg RG, Horner KJ, Juster HR, Schneier FR, Liebowitz MR. Factor structure of social fears: the Liebowitz Social Anxiety Scale. J Anxiety Disord. 1999;13:253-70.,1212. Romm KL, Rossberg JI, Berg AO, Hansen CF, Andreassen OA, Melle I. Assessment of social anxiety in first episode psychosis using the Liebowitz Social Anxiety scale as a self-report measure. Eur Psychiatry. 2011;26:115-21. 13. Baker SL, Heinrichs N, Kim H, Hofmann SG. The Liebowitz social anxiety scale as a self-report instrument: A preliminary psychometric analysis. Behav Res Ther. 2002;40:701-15.-1414. Stein DJ, Kasper S, Andersen EW, Nil R, Lader M. Escitalopram in the treatment of social anxiety disorder: analysis of efficacy for different clinical subgroups and symptom dimensions. Depress Anxiety. 2004;20:175-81.,1919. Caballo VE, Salazar I, Irurtia MJ, Arias B, Nobre-Sandoval L. The assessment of social anxiety through five self-report measures, LSAS-SR, SPAI, SPIN, SPS, and SIAS: a critical analysis of their factor structure. Behav Psychol. 2013;21:423-48. and do not contribute to a clear factor structure of the LSAS-SR.
The present results suggest that the LSAS-SR includes items that may not be relevant to Spanish and Portuguese-speaking countries, at least the ones included in the present study. Although the LSAS-SR captures some features of the SAQ (e.g., interactions with the opposite sex and criticism and embarrassment), it does not do so comprehensively. For instance, the LSAS-SR only lists one single item measuring what appears to be an attempt at a sexual/romantic interaction (i.e., trying to pick up someone). One might argue that some LSAS-SR items touch upon fear of criticism or embarrassment (e.g., talking to people in authority, going to a party, speaking up at a meeting, or being the center of attention). However, these items do not form a single factor. Given that people with SAD typically report an extreme fear of criticism and embarrassment, this omission is surprising.
Our research did not find significant differences among the countries included in the study (invariance by country of origin). In this way, given the size of the samples of the study, the psychometric information included in Table 6 could be very informative for people using the LSAS-SR in the studied Spanish and Portuguese-speaking countries at clinical and non-clinical settings. However, the available literature suggests that the factor structure of the LSAS-SR has varied across individual studies, even when they are conducted within the same culture. It is thus not surprising that the current study supports a different factor structure than the original theoretical framework of the LSAS-SR for at least some Spanish and Portuguese-speaking cultures. Additionally, simple translation is insufficient to determine the construct validity of measures across cultures. Simple translation has been a common strategy among most questionnaires assessing social anxiety (including the LSAS-SR). However, new questionnaires have been developed to overcome this limitation (e.g., the SAQ).2929. Caballo VE, Arias B, Salazar IC, Irurtia MJ, Hofmann SG; CISO-A Research Team. Psychometric properties of an innovative self-report measure: the Social Anxiety Questionnaire for adults. Psychol Assess. 2015;27:997-1012.
30. Caballo VE, Salazar I, Arias B, Irurtia MJ, Calderero M, Graña JL, et al. Validation of the Social Anxiety Questionnaire for Adults (SAQ-A30) with Spanish university students: similarities and differences among degree subjects and regions. Behav Psychol. 2010;18:5-34.-3131. Caballo VE, Salazar IC, Irurtia MJ, Arias B, Hofmann SG; CISO-A Research Team. The multidimensional nature and multicultural validity of a new measure of social anxiety: the Social Anxiety Questionnaire for adults. Behav Ther. 2012;43:313-28. Furthermore, psychometric properties can vary across dimensions of an assessment, and this is particularly true with the factors obtained in this study for the LSAS-SR. The internal consistency ranged from 0.80 to 0.43, the convergent validity ranged from 0.28 to 0.68, and the five-factor structure was different from what other studies have found.88. Oakman J, Van Ameringen M, Mancini C, Farvolden P. A confirmatory factor analysis of a self-report version of the Liebowitz Social Anxiety Scale. J Clin Psychol. 2003;59:149-61.,1212. Romm KL, Rossberg JI, Berg AO, Hansen CF, Andreassen OA, Melle I. Assessment of social anxiety in first episode psychosis using the Liebowitz Social Anxiety scale as a self-report measure. Eur Psychiatry. 2011;26:115-21.
13. Baker SL, Heinrichs N, Kim H, Hofmann SG. The Liebowitz social anxiety scale as a self-report instrument: A preliminary psychometric analysis. Behav Res Ther. 2002;40:701-15.
14. Stein DJ, Kasper S, Andersen EW, Nil R, Lader M. Escitalopram in the treatment of social anxiety disorder: analysis of efficacy for different clinical subgroups and symptom dimensions. Depress Anxiety. 2004;20:175-81.-1515. Heeren A, Maurage P, Rossignol M, Vanhaelen M, Peschard V, Eeckhout C, et al. Self-report version of the Liebowitz Social Anxiety Scale: psychometric properties of the French version. Can J Behav Sci. 2012;44:99-107.
Despite these limitations, the LSAS-SR is a self-report measure of social anxiety associated with good psychometric characteristics and a five-factor solution. Although there has been some support for the convergent validity of this instrument, there may be a number of cultural differences that undermine the utility of this scale in Spanish and Portuguese speaking countries. Thus, the assessment of social anxiety may require judicious deliberation about the role of cultural factors, which can influence the expression of SAD. Although the results of the present study suggest a multifactorial solution of the LSAS-SR, it may not be an ideal instrument (including for poor internal consistency) to assess the various dimensions of social anxiety that are relevant to individuals from Spanish and Portuguese speaking countries. However, it could be a useful self-report instrument to measure general social anxiety in these countries, particularly when using the anxiety/fear subscale.
Acknowledgements
This study was partially supported by a grant from Spain’s Ministry of Science and Technology (reference BSO2003-07029/PSCE) and co-financed by the European Regional Development Fund (ERDF) as well as by a grant awarded to the first author by the Spain’s Ministry of Education, Culture and Sport (reference PRX16/00223). The manuscript’s translation was supported financially by the European Regional Development Fund (ERDF) and by the regional government of Andalusia as part of Universidad de Granada’s program Fortalecimiento de la I+D+i. Financial assistance from the Foundation for the Advancement of Clinical Behavioral Psychology (FUNVECA) is also acknowledged. SGH receives grant support from the National Institutes of Health/National Center for Complementary and Integrative Health (NIH/NCCHI; R01AT007257, R01MH099021, R34MH099311, R34MH086668, R21MH102646, R21MH101567, K23MH100259), and the Department of the Army.
CISO-A Research Team: Argentina: E. Barinaga, G. Bustamante, M. Correche, G. del Valle, L. Gómez, A. Grichener, M. Lipovetzky, L. Mariñelarena, M. Rivarola, P. Robles, S. Savoini, M. Tapia; Bolivia: D. Pinelo; Brazil: B. Donato, R. Lopes, L. Lourenço, C. Neufeld, M. Oliveira, M. Wagner; Chile: P. Álvarez, C. Guerra, S. Villanueva, C. Soto, M. Viveros; Colombia: M. Arias, L. Ariza, D. Barreto, C. Benavides, G. Cajiao, C. Carmona, F. Coconubo, Ó. David, P. Díaz, N. Estupiñán, V. Gómez, M. Lemos, R. Mazo, A. Meneses, M. Ortega, S. Ramírez, L. Rocha, M. Varela, D. Villa-Roel; Costa Rica: H. Cabezas; Dominican Republic: P. Beato, L. Blanco, Y. Ramírez; El Salvador: Ó. Olmedo, O. Olmedo M., A. Zúñiga; Guatemala: G. Aguilar, A. Musso; Honduras: R. Ardón; Mexico: S. Anguiano, M. Ascencio, P. Balcázar, M. Bonilla, Á. Camarena, I. Carrillo, R. del Pino, E. Erosa, G. García, M. García, C. González, M. González, G. Gurrola, D. Hernández, S. Hernández, M. Karam, R. Landero, M. Lomelí, J. Olvera, F. Páez, C. Reyes, M. Ríos, R. Robles, A. Tapia, J. Tejero, P. Vázquez, M. Villaseñor; Paraguay: C. Bettina, A. Caballero, R. Estigarribia, M. Silva; Peru: V. Barreda, L. Benites, L. Dueñas, A. Galli, M. Grijalva, R. Guzmán, J. Marchena, C. Merino, M. Salazar, C. Segura, G. Verme; Portugal: R. Barroso, F. Cardoso, P. Carvalho, M. Loureiro, E. Ribeiro; Puerto Rico: S. Fernández, N. Ortiz, J. Pons, E. Rosario, A. Santos, K. Suárez; Spain: C. Antona, B. Arias, P. Barbado, P. Bas, J. Delgado, M. Fernández, A. Goñi, V. Guillén, M. J. Irurtia, S. Lago, C. Loureiro, M. Muñoz, C. Rausell, S. Torrecillas; Uruguay: A. Conde, R. García, M. Golberg, M. Lagos, L. Moreira, A. Nassar, M. Reich, L. Scafarelli; Venezuela: L. Feldman, Z. Lugli, J. Pellicer, E. Vivas.
The first author is the director and the person responsible for the CISO-A Research Team. He also holds all rights for this team.
References
-
1American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5). Arlington: American Psychiatric Publishing; 2013.
-
2Kessler RC, Üstün TB. The WHO Mental Health Surveys: global perspectives on the epidemiology of mental disorders. New York: Cambridge University; 2008.
-
3Grant BF, Hasin DS, Blanco C, Stinson FS, Chou SP, Goldstein RB, et al. (2005). The epidemiology of social anxiety disorder in the United States: results from the National Epidemiologic Survey on Alcohol and Related Conditions. J Clin Psychiatry. 2005;66:1351-61.
-
4Caballo VE, Salazar IC, Equipo de Investigación CISO-A España. Desarrollo y validación de un nuevo instrumento para la evaluación de las habilidades sociales: El “Cuestionario de habilidades sociales” (CHASO). Psicol Conductual. 2017;25:5-24.
-
5Gómez O, Casas C, Ortega-Ruiz R. Ansiedad social en la adolescencia: factores psicoevolutivos y de contexto familiar. Psicol Conductual. 2016;24:29-49.
-
6Liebowitz MR. Social phobia. Mod Probl Pharmacopsychiatry. 1987;22:141-73.
-
7Beard C, Rodriguez BF, Moitra E, Sibrava NJ, Bjornsson A, Weisberg RB, et al. Psychometric properties of the Liebowitz Social Anxiety Scale (LSAS) in a longitudinal study of African Americans with anxiety disorders. J Anxiety Disord. 2011;25:722-6.
-
8Oakman J, Van Ameringen M, Mancini C, Farvolden P. A confirmatory factor analysis of a self-report version of the Liebowitz Social Anxiety Scale. J Clin Psychol. 2003;59:149-61.
-
9Safren SA, Heimberg RG, Horner KJ, Juster HR, Schneier FR, Liebowitz MR. Factor structure of social fears: the Liebowitz Social Anxiety Scale. J Anxiety Disord. 1999;13:253-70.
-
10Fresco DM, Coles ME, Heimberg RG, Liebowitz MR, Hami S, Stein MB, et al. The Liebowitz Social Anxiety Scale: a comparison of the psychometric properties of self-report and clinician-administered formats. Psychol Med. 2001;31:1025-35.
-
11Ha J, Lim SW, Shin YC, Oh KS. Comparison of anxiety-related traits between generalized and nongeneralized subtypes of social anxiety disorder. J Nerv Ment Dis. 2011;199:390-3.
-
12Romm KL, Rossberg JI, Berg AO, Hansen CF, Andreassen OA, Melle I. Assessment of social anxiety in first episode psychosis using the Liebowitz Social Anxiety scale as a self-report measure. Eur Psychiatry. 2011;26:115-21.
-
13Baker SL, Heinrichs N, Kim H, Hofmann SG. The Liebowitz social anxiety scale as a self-report instrument: A preliminary psychometric analysis. Behav Res Ther. 2002;40:701-15.
-
14Stein DJ, Kasper S, Andersen EW, Nil R, Lader M. Escitalopram in the treatment of social anxiety disorder: analysis of efficacy for different clinical subgroups and symptom dimensions. Depress Anxiety. 2004;20:175-81.
-
15Heeren A, Maurage P, Rossignol M, Vanhaelen M, Peschard V, Eeckhout C, et al. Self-report version of the Liebowitz Social Anxiety Scale: psychometric properties of the French version. Can J Behav Sci. 2012;44:99-107.
-
16Sugawara N, Yasui-Furukori N, Kaneda A, Sato Y, Tsuchimine S, Fujii A, et al. Factor structure of the Liebowitz Social Anxiety Scale in community-dwelling subjects in Japan. Psychiatry Clin Neurosci. 2012;66:525-8.
-
17Terra MB, Barros HM, Stein AT, Figueira I, Athayde LC, Gonçalves Mde S, et al. Internal consistency and factor structure of the Portuguese version of the Liebowitz Social Anxiety Scale among alcoholic patients. Rev Bras Psiquiatr. 2006;28:265-9.
-
18Connor KM, Davidson JR, Churchill LE, Sherwood A, Foa E, Weisler RH. Psychometric properties of the Social Phobia Inventory (SPIN). New self-rating scale. Br J Psychiatry. 2000;176:379-86.
-
19Caballo VE, Salazar I, Irurtia MJ, Arias B, Nobre-Sandoval L. The assessment of social anxiety through five self-report measures, LSAS-SR, SPAI, SPIN, SPS, and SIAS: a critical analysis of their factor structure. Behav Psychol. 2013;21:423-48.
-
20de Beurs E, Tielen D, Wollmann L. The Dutch social interaction anxiety scale and the social phobia scale: reliability, validity, and clinical utility. Psychiatry J. 2014;2014:360193.
-
21Le Blanc AL, Bruce LC, Heimberg RG, Hope DA, Blanco C, Schneier FR, et al. Evaluation of the psychometric properties of two short forms of the social interaction anxiety scale and the social phobia scale. Assessment. 2014;21:312-23.
-
22Bhogal KS, Baldwin DS. Pharmacological treatment of social phobia. Psychiatry. 2007;6:217-23.
-
23Smits JA, Rosenfield D, McDonald R, Telch MJ. Cognitive mechanisms of social anxiety reduction: an examination of specificity and temporality. J Consult Clin Psychol. 2006;74:1203-12.
-
24Bobes J, Badía X, Luque A, García M, González MP, Dal-Ré R. Validación de las versiones en español de los cuestionarios Liebowitz Social Anxiety Scale, Social Anxiety and Distress Scale y Sheehan Disability Inventory para la evaluación de la fobia social. Med. Clin. 1999;112:530-8.
-
25González MP, Bobes J, García M, Badía X, Luque A, Dal-Ré R. Assessing social phobia. The Spanish validation of the “gold standard” clinical scales: the LSAS and the SADS. Eur. Neuropsychopharmacol. 1998;8:S259-S60.
-
26American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision (DSM-IV-TR). Arlington: American Psychiatric Publishing; 2000.
-
27World Health Organization (WHO). CIE-10 trastornos mentales y del comportamiento: descripciones clínicas y pautas para el diagnóstico. Madrid: Meditor; 1992.
-
28Mennin DS, Fresco DM, Heimberg RG, Schneier FR, Davies SO, Liebowitz MR. Screening for social anxiety disorder in the clinical setting: using the Liebowitz Social Anxiety Scale. J Anxiety Disord. 2002;16:661-73.
-
29Caballo VE, Arias B, Salazar IC, Irurtia MJ, Hofmann SG; CISO-A Research Team. Psychometric properties of an innovative self-report measure: the Social Anxiety Questionnaire for adults. Psychol Assess. 2015;27:997-1012.
-
30Caballo VE, Salazar I, Arias B, Irurtia MJ, Calderero M, Graña JL, et al. Validation of the Social Anxiety Questionnaire for Adults (SAQ-A30) with Spanish university students: similarities and differences among degree subjects and regions. Behav Psychol. 2010;18:5-34.
-
31Caballo VE, Salazar IC, Irurtia MJ, Arias B, Hofmann SG; CISO-A Research Team. The multidimensional nature and multicultural validity of a new measure of social anxiety: the Social Anxiety Questionnaire for adults. Behav Ther. 2012;43:313-28.
-
32Salazar IC. Fiabilidad y validez de una nueva medida de autoinforme para la evaluación de la ansiedad/fobia social en adultos [dissertation]. Granada: Universidad de Granada; 2013.
-
33Brown TA. Confirmatory factor analysis for applied research. New York: Guilford; 2006.
-
34Schafer JL, Graham JW. Missing data: our view of the state of the art. Psychol Methods. 2002;7:147-77.
-
35Osborne JW. What is rotating in exploratory factor analysis? [Internet]. 2015 [cited 2018 01 30]. pareonline.net/getvn.asp?v=20&n=2
» pareonline.net/getvn.asp?v=20&n=2 -
36Brown TA. Confirmatory factor analysis for applied research. New York: Guilford; 2014.
-
37Asparouhov T, Muthén B. Exploratory structural equation modeling (ESEM). Struct Equ Modeling. 2009;16:397-438.
-
38Marsh HW, Morin AJ, Parker PD, Kaur G. Exploratory structural equation modeling: an integration of the best features of exploratory and confirmatory factor analysis. Annu Rev Clin Psychol. 2014;10:85-110.
-
39Jöreskog KG, Goldberger AS. Estimation of a model with multiple indicators and multiple causes of a single latent variable. J Am Stat Assoc. 1975;70:631-9.
-
40Morin AJ, Arens AK, Marsh HW. A bifactor exploratory structural equation modeling framework for the identification of distinct sources of construct-relevant psychometric multidimensionality. Struct Equ Modeling. 2016;23:116-39.
-
41Statistica [computer program]. Version 12. Tulsa: StatSoft, Inc.; 2013.
-
42IBM SPSS Statistics [computer program]. Version 22.0. Chicago: IBM; 2013.
-
43MPlus [computer software]. Version 7.4. Los Angeles: Muthén & Muthén; 2015.
-
44Perugi G, Nassini S, Maremmani I, Madaro D, Toni C, Simonini E, et al. Putative clinical subtypes of social phobia: a factor-analytical study. Acta Psychiatr Scand. 2001;104:280-8.
Publication Dates
-
Publication in this collection
11 Oct 2018 -
Date of issue
Mar-Apr 2019
History
-
Received
16 Aug 2017 -
Accepted
11 Jan 2018