Validation of the Mental Illness Sexual Stigma Questionnaire (MISS-Q) in a sample of Brazilian adults in psychiatric care

Objective: We evaluated the psychometric properties of a new instrument “Mental Illness Sexual Stigma Questionnaire” (MISS-Q). Methods: We interviewed 641 sexually active adults (ages 18-80) attending public outpatient psychiatric clinics in Rio de Janeiro about their stigma experiences. Results: Nine factors were extracted through exploratory factor analysis (EFA) and labeled: ‘individual discrimination by others’; ‘staff willingness to talk about sexuality’; ‘staff and family prohibitions’; ‘sexual devaluation of self’; ‘perceived attractiveness’; ‘mental illness concealment’; ‘perceived sexual role competence’; ‘withdrawal’; and ‘locus of socialsexual control’. ‘Withdrawal’ and ‘locus of social-sexual control’ showed poor psychometric properties and were excluded from further analysis. The remaining seven factors had high factorial loadings (.39 to .86), varying from sufficient to optimal reliability (Ordinal α ranged from .57 to .88), and good convergent and discriminant validity. Conclusions: The resulting MISS-Q is the first instrument assessing mental illness sexual stigma with demonstrated psychometric properties. It may prove useful in reducing stigma, protecting sexual health, and promoting recovery.


INTRODUCTION
Nearly twenty years ago, the Surgeon General of the United States identified stigma as "the most formidable obstacle to future progress in the arena of mental illness and health" 1 .Psychiatric patients continue to be stigmatized 2,3 and experience detrimental effects and discrimination in multiple systems (e.g., education, housing, work-force, health, mental health, judicial) and in their social networks 3,4 .
Evidence suggests that labeling someone with mental illness influences sexuality and sexual behaviors as a contributor either to social and sexual isolation 5,6 or to increased sexual risk behaviors [7][8][9] .Expectations of romantic and sexual rejection by the person labeled with mental illness can lead to reduced confidence, constricted social networks, depression, and low self-esteem 3,[9][10][11] and may interfere with achieving full potential for recovery 12,13 .
In a recent review of the existing instruments assessing mental illness stigma 14 , none measured mental illness stigma related to patients' romantic and sexual relationships.Avoidance of patient sexuality by mental health providers and researchers is not only a manifestation of stigma but also includes legitimate worries about the vulnerability of people with mental illness to such adverse outcomes such as coerced sex, nonconsensual sex, unplanned pregnancy, sexually transmitted infections and painful experiences of rejection by potential sexual partners.However, ignoring the sexual lives of people in psychiatric care is not only inconsistent with recovery but contributes to adverse health outcomes, including the sharply elevated rates of HIV infection found among people with mental illness compared to the general population in most regions where rates have been examined 15,16 .
Our investigation into mental illness sexual stigma originated with psychiatric outpatients in the U.S. 6,7 .It was explored through ethnographic and other developmental work in Brazil 9,17,18 , resulting in a questionnaire with good testretest reliability (k =.75) 17 , that has been used in a U.S. pilot study of adolescents with psychiatric disorders 19 .However, the factorial structure and validity of the instrument have not been determined.The Mental Illness Sexual Stigma Questionnaire (MISS-Q) applies modified labeling theory 20 , which postulates that stigma influences behavior through social environmental and social psychological processes.Once labeled with mental illness (societal stigma), the person experiences stigma via three mechanisms: 1) individual discrimination -other individuals engage in overt practices of discrimination against the stigmatized individual; 2) structural discrimination -institutional practices and policies work against the stigmatized group; and 3) social psychological processes -the stigmatized person's own perceptions of the negative stereotypes attributed to the undesirable trait (sometimes referred to as internalized stigma or self-stigma) -and expected discrimination 10 .The MISS-Q was developed to measure these three mechanisms but its items have not yet been empirically confirmed through exploratory factor analysis.
The aim of this study was to identify latent factors underlying the MISS-Q and to test their psychometric properties (i.e., reliability and validity) in a sample of Brazilian adults in psychiatric care.

Sample and design
A sample (N = 641) of sexually active adults in psychiatric care was recruited from eight public outpatient psychiatric clinics in Rio de Janeiro between June 2007 and November 2009 as part of an HIV prevention trial (R01 MH65163: Wainberg).To evaluate the MISS-Q, we analyzed data collected at baseline by trained interviewers.

MISS-Q
The MISS-Q is a face-to-face interview containing 33 items.The first six items were constructed to assess Individual discrimination experiences including five general mental illness individual discrimination items (e.g."How often have you been made fun of because you have a mental illness?"; "Has anyone ever called you 'crazy,' 'loca,' or 'nuts'?"); and one sexual relationship-specific item.
Ten items refer to structural discrimination, including the willingness of mental health workers to discuss sexuality with patients (e.g."Staff members make patients feel comfortable to talk about sexuality and sex issues."),and messages of prohibition about sexual activity from staff and family members (e.g."How often has someone in your family ever said that since you are a user of mental health services you should not have sex?").
Seventeen items assess Social psychological processes, which comprises further sub-dimensions including
J Bras Psiquiatr.2017;66(3):131-8 Mental Illness Sexual Stigma Questionnaire devaluation, coping, attractiveness and locus of social-sexual control.'Devaluation' captures perceptions those with mental illness have of others' disregard and rejection of their sexuality (e.g."Most people don't show interest in having a romantic or sexual relationship with someone who has a mental illness").'Coping' focuses on avoidance of or willingness to directly deal with mental illness stigma (e.g."You hide the fact that you have been diagnosed with a mental illness from people you are interested in having a romantic or sexual relationship with").'Attractiveness' elicits internalized perceptions (e.g."Having a mental illness makes you feel less attractive than other women/men").'Locus of social-sexual control' prompts for choice in sexual relationships (e.g."In order to be sexually active, you always do what other people ask of you").All MISS-Q items had 4-point Likert-type response options (0never; 1 -rarely; 2 -sometimes; 3 -often).The one sexual item in the Individual discrimination component of the interview "How many of the people you have wanted to have a romantic or sexual relationship with said they didn't want to be involved with you because you were a user of mental health services?"was rated on 4-point Likert-type scale (0none; 1 -some; 2 -most; 3 -all).

Divergent validity measure
HIV knowledge was assessed using the item "Pulling out the penis before a man climaxes/cums keeps a woman from getting HIV during sex", rated on a Yes -1 / No -0 scale 21 .

Data analysis
Descriptive statistics (i.e.mean and standard deviation [SD], and skewness, kurtosis, and Shapiro-Wilks test to assess variables' distribution) and correlational analyses were performed with SPSS 23 22 .Exploratory Factor Analysis (EFA -Quartimin oblique rotation) was conducted in Mplus 7.3 23 to assess the dimensionality of the MISS-Q.Weighted Least Squares adjusted Mean and Variance (WLSMV) estimation was used because the MISS-Q items were analyzed as ordinal variables 23,24 .First, we determined the maximum number of factors to retain using the Kaiser criterion (eigenvalue > 1) 25 .Second, the factorial models were compared through the Root Mean Square Error of Approximation (RMSEA), Comparative Fit Index (CFI) and Tucker-Lewis Index (TLI).RMSEA ≤ .08 or ≤ .05,and CFI and TLI values ≥ .90 or ≥ .95reflect acceptable and optimal fit, respectively 26,27 .Items with low loadings (< .30)or statistically significant (p < .05)crossloadings greater than .30were considered weak indicators.Factor reliability was tested through ordinal alpha 28 computed in R (http://www.R-project.org)psych package 29 .Similarly to Cronbach's alpha, ordinal alpha coefficients greater than .70indicate good internal reliability 30 .
Convergent validity was assessed by testing the association (Pearson's r coefficient) between the extracted factors with sexual-or romantic-related items and the extracted factor with non-sexual/romantic items.Divergent validity was evaluated by testing the association (Pearson's point-biserial coefficients) between the extracted sexual/ romantic MISS-Q factors with one item assessing HIV knowledge, which was not expected to be associated with mental illness sexual stigma experiences.

Sample characteristics
The sample (Table 1) consisted of women (58%) and men (42%) with a mean age of 42.54 (SD = 10.32,range = 18-76) and was racially diverse (19.50% black, 32.76% white, and 47.74% multiracial).Overall, 32.92% had a diagnosis of schizophrenia; 21.68% of bipolar disorder; 20.28% of nonpsychotic depression; 10.30% of depression with psychosis; 6.24% of anxiety disorders; 4.84% of psychosis not otherwise specified; 3.12% of schizoaffective disorder; and 0.62% had other diagnoses.More than half of the participants (66.61%) reported being currently involved in a relationship, with 46.80% reporting being married.¥ 4 patients did not disclose their marital status; § 2 patients did not disclose their sexual orientation and 4 did not answer the question.

Exploratory Factor Analysis (EFA)
Nine factors with eigenvalue greater than 1 and good indices of fit were initially extracted (see Table 2).Two

MISS-Q factors inter-correlation
The results of the correlational analysis (Table 4) showed that patients with lower perceived attractiveness reported lower positive experiences of staff willingness to talk about sexuality, lower perceived sexual role competence, and greater mental illness concealment.Lower perceived attractiveness correlated with greater sexual devaluation of self and with experiences of rules that limit their sexual relationships from mental health professionals and family members.A more positive experience of staff willingness to talk about sexuality corresponded to greater perceived sexual role competence and disclosure of psychiatric diagnosis.Staff and family prohibitions were associated with lower sexual role competence.

Convergent validity
Convergent validity was investigated correlating the MISS-Q sexual/romantic factors with the individual discrimination factor (e.g., "How often have you been made fun of because you have a mental illness?"; "Has anyone ever called you 'crazy', 'loca', or 'nuts'?").Four MISS-Q factors were significantly correlated with individual discrimination (Table 4): prohibitions by staff and family members, perceived attractiveness, perceived sexual role competence, and sexual devaluation of self.

Divergent validity
Divergent validity was evaluated correlating the seven MISS-Q factors with a single item assessing HIV knowledge.None of the MISS-Q factors showed a statistically significant correlation (Table 4).

DISCUSSION
We conducted the first exploratory factor analysis to empirically evaluate the latent factors of the MISS-Q.We developed this instrument using a theoretical 3-factor model corresponding to the stigma mechanisms described by Link and Phelan 20 but this factoral structure was not previously investigated 6,7   The two factors 'perceived attractiveness' and 'mental illness concealment', each composed of two items, showed sufficient psychometric properties.Despite their low internal consistency, which was likely influenced by the small number of items comprising them, 'perceived attractiveness' and 'mental illness concealment' had high factor loadings (from .44 to .77).In order to improve the psychometric properties of these two sub-scales, more items need to be developed.
The two factors 'withdrawal' and 'locus of social-sexual control', both composed of two items, showed poor psychometric properties (i.e.low internal consistency and factor loadings) and were not further analyzed.Qualitative and quantitative investigations should be conducted to determine if refinement of these sub-scales could improve their psychometric properties.
The seven MISS-Q factors (27 items) extracted were significantly inter-correlated, with good divergent validity, and most with good convergent validity.Staff and family prohibitions, sexual devaluation of self, perceived sexual role competence, and perceived attractiveness correlated with individual discrimination, which was used as a criterion variable.Overall, the results of the current study suggest that the MISS-Q has good psychometric properties.
Staff willingness to talk with patients about their sexuality was not associated with individual discrimination, though patients with lower perceived attractiveness reported less-positive experiences of staff willingness to talk about sexuality and with experiences of rules from mental health professionals and family members that limit their sexual relationships.A more positive experience of staff willingness to talk about sexuality corresponded to greater perceived sexual role competence and disclosure of psychiatric diagnosis.Staff and family prohibitions were associated with lower sexual role competence.Staff willingness to have nonjudgmental conversations about sexual aspects of patients' lives is associated with patients' sense of competence in potential sexual or romantic encounters and reduced concern about disclosure of a mental illness diagnosis to a potential sexual/romantic partner.
It is interesting that patients' willingness to confront mental illness stigma as part of a sexual relationship was independent of how much stigma patients had experienced in other contexts (individual discrimination).Having the courage to reveal having mental illness in a sexual relationship despite discrimination in other domains may be a marker of resilience to the stigmatizing beliefs of others.Additional items in the MISS-Q about managing stigma in sexual and romantic relationships might shed light on patients' successful strategies for coping with mental illness stigma more generally.
A recent review identified two approaches for reducing general internalized stigma (i.e., not specifically sexualor romantic-related): interventions aimed at changing stigmatizing beliefs and attitudes about mental illness, and interventions that do not challenge stereotypes but rather improve stigma-coping skills by enhancing self-esteem, empowerment, and help-seeking behavior 31 .Even though tackling stigmatizing beliefs might seem a more direct and logical way to reduce internalized stigma, an important number of stigma experts seem to favor the coping training approach 31 .Future research should explore if reducing internalized stigma has, in turn, a positive effect on wellbeing, including sexual-or romantic-related aspects, or whether mental illness sexual stigma must be the direct focus of an intervention to reduce its negative impact.Creating interventions specific to reducing internalized sexual stigma would enable clinicians to identify critical windows during which intimate relationships and sexual health could become positive aspects of recovery.These interventions might also improve patients' ability to protect themselves from adverse sexual health outcomes by promoting a sense of competence in sexual and romantic encounters.Having a tool -the MISS-Q -to quantify mental illness sexual stigma is a first step toward identifying when it is hindering recovery and contributing to unsafe sexual behaviors.
Very few studies have been conducted examining how stigma affects the sexual lives of psychiatric patients.We believe the time has come to change this pattern.

Limitations
This study contained some methodological limitations.Despite the thorough psychometric analysis, we did not test the stability of the seven-factor model over time using confirmatory factor analysis (CFA).Additionally, validity ultimately must be established by use of the instrument across diverse age groups (including youth), socioeconomic strata (those receiving care in private and public treatment settings), and geographic regions (taking into account culture, context, and meanings of stigma and related behaviors).
Refinement of the MISS-Q sub-scales is necessary to improve their reliability, e.g., building additional items to better capture the constructs underlying mental illness sexual stigma.Lastly, the psychometric properties of the questionnaire should be examined in samples of people receiving care in other settings and geographic regions.For example, in this Brazilian outpatient sample, nearly half of patients reported being married; by contrast, in studies examining sexual behavior in the context of HIV risk, the vast majority of sexually active participants were single 16 .Our own measures were informed by carefully conducted ethnographic and other developmental work 9,17,18 .Other aspects of patients' stigma experiences may impinge upon their sexual relationships and these are worth examining in future studies.Given the cultural underpinnings of stigma about mental illness, the salience and generalizability of MISS-Q items, factors, and their psychometric properties may require additional refinements when used in new settings.

CONCLUSIONS
In conclusion, the MISS-Q is the first instrument with demonstrated psychometric properties to assess mental illness sexual stigma among sexually active adults in psychiatric care.Though some modifications are required to improve the measurement of perceived attractiveness, mental illness concealment, withdrawal, and locus of social-sexual control as they relate to sexual and romantic relationships, the instrument possesses strong psychometric properties and may be useful to investigators working with those in psychiatric care.This instrument is ready to use bearing in mind that internalized mental illness sexual stigma is very complex, multi-factorial, and probably not limited to what we measured and what showed good psychometric properties in this version.
The MISS-Q captures diverse aspects of mental illness sexual stigma, with potential applicability to test its association with sexual risk behaviors, to evaluate the outcome of stigma-reduction interventions, and to broaden recovery approaches so that people with mental illness receive more support to pursue the very ordinary goal of having safe intimate relationships as part of leading a full life.

Table 1 .
Characteristics of 641 psychiatric outpatients whose mental illness sexual stigma was assessed

Table 3 .
Mental Illness Sexual Stigma Questionnaire Mental Illness Sexual Stigma (MISS-Q) factor loadings and reliability You avoid approaching someone you are interested in having a romantic or sexual relationship with if you think he/she has negative attitudes about users of mental health services.
sound psychometric properties as demonstrated by high factor loadings and good internal reliability.The first factor, 'individual discrimination by others' assesses non-sexual/ romantic-related stigma, while the remaining six specifically assess sexual-or romantic-related stigma experiences of adults in psychiatric care.J Bras Psiquiatr.2017;66(3):131-8 How often has someone in your family ever said that since you are a user of mental health services you should not have a romantic or sexual relationship with people who do not have a mental illness?α = ordinal alpha; MH mental health; ¥ reversed score; F1: Staff willingness to talk about sex; F2: Sexual devaluation of self; F3: Staff and family prohibitions; F4: Individual discrimination by others; F5 Perceived attractiveness; F6: Mental illness concealment; F7: Perceived sexual role competence; F8: Withdrawal; and F9: Locus of social-sexual control.J Bras Psiquiatr.2017;66(3):131-8 Wainberg M, et al.