Cross-cultural validation of the Brief Social Phobia Scale for use in Portuguese and the development of a structured interview guide

Validação transcultural para o português da Brief Social Phobia Scale e desenvolvimento de um roteiro de perguntas-guia

BRIEF REPORT

Cross-cultural validation of the Brief Social Phobia Scale for use in Portuguese and the development of a structured interview guide

Validação transcultural para o português da Brief Social Phobia Scale e desenvolvimento de um roteiro de perguntas-guia

Flávia de Lima Osório; José Alexandre de Souza Crippa; Sonia Regina Loureiro

Department of Neurology, Psychiatry and Medical Psycology, Faculdade de Medicina de Ribeirão Preto, Universidade de São Paulo (USP), Ribeirão Preto (SP), Brazil

^{Correspondence}

ABSTRACT

OBJECTIVE: To present the translation and validation of the Brief Social Phobia Scale for use in Brazilian Portuguese, to develop a structured interview guide in order to systemize its use and to perform a preliminary study of inter-rater reliability.

METHOD: The instrument was translated and adapted to Portuguese by specialists in anxiety disorders and rating scales. A structured interview guide was created with the aim of covering all of the items of the instrument and grouping them into six categories. Specialists in mental health evaluated the guide. These professionals also watched the videotaped interviews of patients with and without social anxiety disorders, and, based on the interview guide, they rated the scale to evaluate its reliability.

RESULTS: No semantic or linguistic adjustments were needed. For the complete scale, the general evaluation showed a percentage of agreement of 0.84 and intraclass coefficient of 0.91. The mean inter-rater correlation was 0.84.

CONCLUSIONS: The Portuguese-language version of the Brief Social Phobia Scale is available for use in the Brazilian population, with rather acceptable indicators of inter-rater reliability. The interview guide was useful in providing these values. Further studies are needed in order to improve the reliability and to study other psychometric properties of the instrument.

Keywords: Anxiety disorders; Phobic disorders; Validation studies [Publication type]; Evaluation studies; Reproducibility of results

RESUMO

OBJETIVO: Apresentar a tradução e validação da Brief Social Phobia Scale para o português, um roteiro de perguntas-guia para sistematizar sua aplicação e o estudo preliminar da confiabilidade inter-avaliadores.

MÉTODO: O instrumento foi traduzido e adaptado para o português por especialistas em transtornos de ansiedade e escalas de avaliação. Um roteiro de perguntas-guia foi elaborado de modo a abranger todos os itens do instrumento, agrupando-os em seis categorias. O roteiro foi submetido à avaliação de profissionais de saúde mental. Tais profissionais também assistiram a entrevistas gravadas em videotape de pacientes com e sem transtorno de ansiedade social, as quais tiveram como referência o roteiro para pontuar a escala para o cálculo da confiabilidade.

RESULTADOS: Não foram necessárias reformulações e adaptações semânticas e lingüísticas. Para a escala total, a concordância geral foi 0.84 e o coeficiente de correlação intraclasse 0.91. O nível médio de concordância dos avaliadores foi 0.84.

CONCLUSÕES: a versão para o português está disponível para uso na população brasileira, apresentando indicadores preliminares de confiabilidade inter-avaliadores bastante aceitáveis, sendo que o uso do roteiro de perguntas-guia foi bastante útil para favorecer tais valores. Futuros estudos são necessários para aprimorar o valor desta e das demais propriedades psicométricas do instrumento.

Descritores: Transtorno da ansiedade; Fobia social; Estudos de validação, escalas; Estudos de avaliação; Confiabilidade da tecnologia

Introduction

Despite its high prevalence and adverse impact, social anxiety disorder (SAD) remains an under-recognized and underdiagnosed condition.^1-2 However, SAD responds well to pharmacological treatment and cognitive therapy.³ Therefore, early detection is essential to preventing the development of comorbid conditions and improving the prognosis.

Rating scales are important instruments for characterizing clinical profiles, as well as for determining the severity of psychiatric disorders. Previous review studies^4-5 had identified a variety of available instruments, both self- and observer-rated, for SAD assessment.

Some scales are available for use in the Brazilian population.^6-10 However, to our knowledge, no observer-rated SAD assessment instrument is available for such use. The importance of such an instrument, specifically for SAD assessment, lies in the fact that individuals with SAD tend to underestimate their SAD symptoms, associating them and the consequent harms to comorbid situations such as depression.

Among the observer-rated instruments, the Brief Social Phobia Scale (BSPS)¹¹ is of great importance. It consists of 18 items, grouped in three subscales (fear, avoidance and physiological arousal), a score corresponding to one of five levels of severity (0 to 4) being assigned for each item. The BSPS has shown good psychometric qualities: test-retest reliability (0.91); internal consistency (0.81); and convergent validity (0.70) with the Liebowitz Social Anxiety Scale – observer-rated version. The factor analysis identifies six factors: generalized symptoms of fear and avoidance; physiological arousal; doing something while being watched; speaking in public; speaking at social gatherings; and talking to strangers.¹² This scale has been used to assess the severity of SAD and the response to treatment.

In the original proposition,¹¹ the BSPS is applied after an unstructured interview. This procedure presumes that raters are experienced and have received appropriate training for using the instrument, as well as for recognizing and classifying the psychopathological symptoms. The absence of such requisites may lead to reduced reliability.¹³ Therefore, semistructured interviews and interview guides might increase standardization, thereby reducing variability in the interpretation of the information by different raters and minimizing the effect of rater experience with the instrument and with the disorder assessed.^14-15

Objective

The study had multiple objectives: to present the translation and cross-cultural validation of the BSPS for use in Portuguese; to evaluate a structured interview guide, created to facilitate and systemize the use of the scale; and to conduct a preliminary study of inter-rater reliability when the scale is used in conjunction with the interview guide.

Method

This study was approved by the local research ethics committee (Nº HCRP 11570/2003), and all subjects (patients and controls) gave written informed consent after being fully informed of the research procedure.

1. Translation and validation of the BSPS for use in Portuguese

The BSPS was translated from the original English into Portuguese by two psychiatrists and one psychopharmacologist specializing in anxiety disorders with experience in rating scales. All were Brazilian by birth and proficient in the English language. The three versions were compared and a single final version was obtained after reaching a consensus. The final Portuguese-language version was back translated by a bilingual psychiatrist who did not have access to the original English version, and the authors of the original version then compared the back-translation to the original.

2. Interview guide

In order to facilitate and systematize the use of the scale, we proposed a semistructured interview, which was based on a structured interview guide and addressed all items in the scale. We grouped the assessed items into six categories to favor a logical sequence of questions.

Eight researchers from the mental health field (four psychologists, two occupational therapists and two psychiatrists) individually reviewed the interview guide. This group of researchers consisted of one undergraduate in psychology and seven graduate professionals with one to twelve years of postgraduate experience and little (n = 4) or sufficient (n = 4) experience in using rating scales in a clinical and research contexts. The reviewers evaluated the guide in terms of clarity of the instructions, as well as in terms of pertinence/understandability of the content and the guide questions for the assessment of each item in the scale.

The members of the research group then watched a videotaped interview of an individual with SAD. The interview was conducted according to the previously created interview guide and took an average of 40 minutes. We then asked the researchers to complete the scale independently. The answers were compared, and the points of agreement/disagreement were discussed until a consensus was obtained.

After rating the instrument, the reviewers pointed out the difficulties they experienced in using the guide and suggested some changes. All suggestions were discussed and were included in the final version of the guide.

3. Preliminary study of BSPS inter-rater reliability

The aforementioned group of researchers participated as reviewers for the preliminary test of the reliability of the scale. At this stage, the group watched, at two different times, four videotaped interviews with four individuals: three presenting various degrees of SAD severity and one without SAD. After each video, the researchers completed the scale independently. The interviews were conducted using the final version of the guide as a reference. After the scale had been completed, each answer was evaluated by the group, obtaining a consensus answer for each patient, which was used as a standard for establishing the inter-rater agreement level. The evaluations were compared for each item and each rater was compared individually to the group.

Results

1. Translation and validation of the BSPS for use Portuguese

The back-translation of the BSPS was evaluated by the authors of the original version, who stated that no reformulations or semantic and linguistic adjustments were necessary. The Portuguese-language version was dubbed the Escala Breve de Fobia Social (^{Annex 1}).

2. Development of a structured interview guide

In evaluating the guide, the raters found that, in general, the questions were appropriate, clear and favored the systematization of usage. However, they suggested that some questions be reformulated and others added. The problems identified were related to the criteria for the severity of fear subscale, which the raters felt needed more clarity and objectivity. Therefore, it was proposed that each of the four severity criteria be further detailed to make them more objective. After this reformulation, the researchers reviewed the material and concluded that the difficulties had been solved and that the instrument could now be submitted to the next stage of the test.

The final version of the guide (^{Annex 2}) consists of questions grouped into six categories of investigation:

-Category 1: Social Gatherings: part 1– item 6 (fear and avoidance)

-Category 2: Physiological Arousal: part 2– items 1, 2, 3 and 4.

-Category 3: Being Observed: part 1– item 7 (fear and avoidance)

-Category 4: Speaking/Talking: part 1– items 1, 2 and 3 (fear and avoidance)

-Category 5: Being Embarrassed: part 1– item 4 (fear and avoidance)

-Category 6: Feeling Criticized: part 1– item 5 (fear and avoidance)

3. Preliminary study of BSPS inter-rater reliability

In the first stage of scoring, the percentage of agreement of the items ranged from 0.14 to 1.00 for the scale as a whole, from 0.14 to 0.86 for fear, from 0.57 to 1.00 for avoidance and from 0.57 to 0.86 for physiological arousal. Inter-rater agreement ranged from 0.55 to 0.88. The general mean was 0.68.

In the second stage, the percentage of agreement was 0.84 and the intraclass coefficient (ICC) was 0.91. Table 1 shows the information regarding the subscales.

Individual rater agreement with the group ranged from 0.79 to 0.94, with a mean of 0.84.

Discussion

The Portuguese-language version of the BSPS is now available and meets an important need, since, to date, no observer-rated instruments were available for assessing SAD. As in previous studies of observer-rated instruments,^13-15 using a structured interview guide proved useful for the systematic use of the instrument and increased its reliability, independently of the level of experience of the raters. In the preliminary study of the psychometric quality of the scale, we observed that all items presented significance, with an overall percentage of agreement of 0.84 and an ICC of 0.91, the latter being a benchmark of excellent agreement.

The additional detail and specification given to the severity of fear criteria provided a further increase in the levels of inter-rater agreement on each item, strongly contributing to the acquisition of better reliability values. Although this psychometric quality has not been studied in the original version, the observed preliminary value is consistent with the psychometric values presented in the original study,¹² showing that the scale is appropriate for our context.

We believe that the present version is equivalent to the original in terms of language, concepts and items included. The back-translation was accepted by the authors of the original instrument, with no need for reformulations, and was thus designated the official version for use in the Portuguese language. The original layout of the instrument was maintained.

Conclusion

The Portuguese-language version of the BSPS was created through a careful process of translation and adjustment. In this preliminary study, inter-rater reliability was found to be quite acceptable.

Further studies, involving larger samples, control groups and double-blind assessments, are needed in order to increase the value of this and other psychometric properties of the scale. Such studies might further the evaluation of the instrument in clinical and research contexts.

Acknowledgments

The authors thank Graeff FG, Zuardi AW, Hetem LA, Busatto GF, Davidson J and Connor K for their collaboration in the process of translating the instrument.

Submitted: 20 March 2006

Accepted: 17 May 2006

Financing: Fundação de Apoio ao Ensino, Pesquisa e Assistência do Hospital das Clínicas da Faculdade de Medicina de Ribeirão Preto FAEPA-USP and Fundação de Amparo à Pesquisa do Estado de São Paulo (Fapesp) - 02113197-2

Conflict of interest: None

Annex 1 - click here to enlarge

Annex 2 - Click here to enlarge

• 1. Davidson JR, Hughes DL, George LK, Blazer DG. The epidemiology of social phobia: findings from the Duke Epidemiological Catchment Area Study. Psychol Med 1993;23(3):709-18.
• 2. Katzelnick DJ, Kobak KA, DeLeire T, Henk HJ, Greist JH, Davidson JR, Schneier FR, Stein MB, Helstad CP. Impact of generalized social anxiety disorder in managed care. Am J Psychiatry 2001;158(12):1999-2007.
• 3. Nardi AE. O tratamento farmacológico da fobia social. Rev Bras Psiquiatr 1999;21(4):249-57.
• 4. Tharwani HM, Davidson RT. Symptomatic and functional assessment of social anxiety disorder in adults. Psychiatric Clin North Am 2001;24(4):643-59.
• 5. Osorio FL, Crippa JA, Loureiro SR. Instrumentos de avaliação do transtorno de ansiedade social. Rev Psiq Clin 2005;32(2):73-83.
• 6. Paes de Barros T. Perfil psicopatológico e transtorno de personalidade em uma amostra de pacientes com diagnóstico de Fobia Social [dissertação]. São Paulo: Faculdade de Medicina, Universidade de São Paulo; 1996.
• 7. Vilete LM, Coutinho ES, Figueira IL. Confiabilidade da versão em Português do Inventário de Fobia Social (SPIN) entre adolescentes estudantes do município do Rio de Janeiro. Cad Saude Publica 2004;20(1):89-99.
• 8. Osorio FL, Graeff F, Busato G, De Pinho M, Mazza M, Crippa JAS, Chaves M, Loureiro SR. Inventário de Fobia Social (SPIN): validação para o Brasil. Rev Bras Psiquiatr 2004;26 (Supl II):6.
• 9. Osorio FL, Loureiro SR, Zuardi AW., Graeff F, De Pinho M, Mazza M, Chaves M, Crippa JAS. Escala de auto-avaliação ao falar em público (SSPS): validação transcultural para o Brasil. Rev Bras Psiquiatr 2004;26(Supl II):21.
• 10. Picon P, Gauer GJ, Hirakata VN, Haggstram LM, Beidel DC, Turner SM, Manfro GG. Reliability of the Social Phobia and Anxiety Inventory (SPAI) Portuguese version in a heterogeneous sample of Brazilian university students. Rev Bras Psiquiatr 2005;27(2):124-30.
• 11. Davidson JR, Potts NL, Richichi EA, Ford SM, Krishnan KR, Smith RD, Wilson W. The Brief Social Phobia Scale. J Clin Psychiatry 1991;52(Suppl):48-51.
• 12. Davidson JR, Miner CM, De Veaugh-Geiss J, Tupler LA, Colket JT, Potts NL. The brief social phobia scale: a psychometric evaluation. Psycol Med 1997;27(1):161-7.
• 13. Crippa JA, Sanches RF, Hallak JE, Loureiro SR, Zuardi AW. A structured interview guide increases Brief Psychiatric Rating Scale reliability in raters with low clinical experience. Acta Psychistr Scand 2001;103(6):465-70.
• 14. Williams JB. A structured interview guide for the Hamilton Depression Rating Scale. Arch Gen Psychiatric 1988;45(8):742-7.
• 15. Vilela JA, Crippa JA, Del-Ben CM, Loureiro SR. Reliability and validity of a Portuguese version of the Young Mania Rating Scale. Braz J Med Biol Res 2005;38(9):1429-39.

Annex 1

Annex 2

Publication Dates

• Publication in this collection
  28 Sept 2006
• Date of issue
  Sept 2006

History

• Accepted
  17 May 2006
• Received
  20 Mar 2006