Rated pleasantness and relevance of Brazilian-Portuguese adjectives for future assessment instruments

CLINICAL AND EXPERIMENTAL NEUROPSYCHOLOGY

Rated pleasantness and relevance of Brazilian-Portuguese adjectives for future assessment instruments

Maria Clara Pinheiro de Paula Couto^I, Rochele Paz Fonseca^II, Sílvia Helena Koller^I

^IUniversidade Federal do Rio Grande do Sul, Porto Alegre, RS, Brasil

^IIPontifícia Universidade Católica do Rio Grande do Sul, Porto Alegre, RS, Brasil

^{Correspondence}

ABSTRACT

In the present study we present Brazilian normative ratings of pleasantness and relevance for a sample of German adjectives, with the aim of contributing to the development of psychosocial and neuropsychological instruments, among other related fields. Three judges validated a preliminary list of 186 German adjectives translated to Brazilian Portuguese. A coefficient of content validity (CCV) was obtained to evaluate the degree of agreement among the judges in two dimensions: equivalence and clearness. The final list contained 136 items. The CCV for the aggregate of the 136 adjectives was .90 for equivalence and .91 for clearness. These adjectives were subsequently rated for pleasantness and relevance by 385 Brazilian participants aged 17 to 68 years. The Brazilian ratings were compatible with the original German ratings. Agreement frequency was 73-99% for pleasantness and 44-96% for relevance. These norms may be useful for both national and cross-cultural studies between Germany and Brazil and can be used in a wide range of fields including social cognition, cognitive psychology, and clinical neuropsychology. Importantly, items from this list of adjectives can be used to develop instruments to assess memory, language, and mental representations, among other cognitive processes.

Keywords: pleasantness, relevance, word lists, cross-cultural research, neuropsychological assessment.

Introduction

Neuropsychological assessment can be conducted based on a set of clinical and experimental procedures such as interviews, observations, clinical or ecological tasks and standardized performance, and functional cognition tests (for review, see Lezak, Howieson, & Loring, 2004; Strauss, Sherman, & Spreen, 2006). To accurately adapt verbal stimuli to assess cognitive processes or social cognitive abilities, obtaining norms for word and sentence-level lists is essential. In Brazil, words for which norms have been obtained have been rated with regard to semantic association in children (Salles, Holderbaum, & Machado, 2009) and college students (Salles et al., 2008). In these specific studies, the authors considered semantic association as (1) association strength (i.e., which word is retrieved from memory after presentation of a target word) and (2) the number of words generated for each previously presented target word. Normative data have also been established for concreteness with words evaluated according to levels of abstraction or concreteness (Janczura, Castilho, Rocha, van Erven, & Huang, 2007). These norms are especially important for researchers who work with memory and language. Concrete words are theoretically more easily retrieved and recognized in lexical decision tasks (see Janczura et al., 2007). Semantic association norms are also useful when working with the semantic priming paradigm in which processing the target is facilitated by the prime when prime and target are semantically related. In addition to linguistic and mnemonic processes, Santos, Silveira, Gomes, and Stein (2009) recently provided affective norms for the verbal material included in the Brazilian version of the Deese-Roediger-McDermott (DRM) paradigm (Stein, Feix, & Rohenkohl, 2006). To the best of our knowledge, this was the first study to provide such normative ratings of pleasantness for Brazilian-Portuguese words, thus accomplishing emotional processes.

Considering the scarce norms for emotional verbal stimuli mainly with regard to social and emotional aspects, the present study sought to provide normative ratings of pleasantness and relevance for a set of adjectives used in Germany in experimental studies of social cognition. This interdisciplinary field combines methods and theoretical models from cognitive psychology, cognitive neuroscience, and social psychology. One important issue in this field concerns the dual-process models of automatic versus controlled components (for review, see Lieberman, 2007). In this study, automatic processes were the main focus and represented the thoughts and concepts that people cannot control and generally are not aware of feeling or thinking.

Within these automatic social-cognitive processes, emotional connotations of words are an important domain to be considered when planning experimental or clinical tasks that investigate cognitive processes such as attention, memory, mental representations, lexical abilities, executive function, and emotional processing. Pleasantness and relevance are aspects related to emotional connotations of words, which are briefly reviewed below.

Pleasantness

Pleasantness refers to the domain of emotionality related to verbal material (e.g., words). Therefore, one may categorize words as positive/pleasant, neutral, or negative/unpleasant. Studies have demonstrated that the affective connotations of words may differentially influence word processing. For example, for researchers in the field of neuropsychology, the emotional valence of a word is related to enhanced memory recovery (e.g., Kensinger & Corkin, 2003). Individuals remembered more negative words than neutral words mainly when the valence was associated with arousal.

In the field of social cognition, words with affective connotations have been used in experiments that explored social attitudes such as prejudice. A popular task used to investigate implicit social attitudes is the affective priming or evaluative priming (Fazio, Jackson, Dunton, & Williams, 1995). In this task, participants work through a sequence of trials in which two stimuli are presented: a prime and a target. The participants must evaluate the valence of the target (i.e., whether it is a positive or negative word). In the evaluative priming paradigm, the prime and target are either evaluatively congruent or incongruent. The prime is irrelevant to the evaluation task but is said to be automatically processed, thus influencing the participant's decision regarding the target's pleasantness. Therefore, if the prime and target share the same valence (i.e., they are congruent), then the participant's response latency is faster because of facilitation than when they do not share the same valence (i.e., they are incongruent). Fazio et al. (1995) examined the automatic activation of racial attitudes from memory. Photographs of white and black individuals were used as primes and positive and negative adjectives were used as targets. Among the white participants, white faces further facilitated the evaluation of positive targets, whereas black faces facilitated the evaluation of negative targets. The authors concluded that the participants' attitudes toward black people were negative. Many other studies tested the affective priming with social attitudes (Degner & Wentura, 2011; Livingston & Brewer, 2002; Lowery, Hardin, & Sinclair, 2001; Wittenbrink, Judd, & Park, 2001). Norms of pleasantness for adjectives are then important for researchers in the social cognition field.

International and national studies have generated pleasantness (affective) norms for verbal materials. Bradley and Lang (1999) provided emotional norms for a large sample of English nouns rated in the dimensions of pleasantness, arousal, and dominance. Bellezza, Greenwald, and Banaji (1986) provided normative data for 399 English words rated by male and female students according to their pleasantness, imagery, and frequency. Gender differences with regard to pleasantness and association norms for 101 English words were explored by Silverstein and Dienstbier (1968). In a recent study, Eilola and Havelka (2010) provided affective norms for a list of 210 English and Finnish nouns. Ratings for pleasantness, concreteness, and familiarity were collected from English and Finnish participants. In addition to pleasantness norms, the authors conducted cross-linguistic comparisons among British English, Finnish, and American ratings. In Brazil, Santos et al. (2009) established affective norms for the Brazilian version of the Deese-Roediger-McDermott (DRM) paradigm (Stein at al., 2006). This paradigm includes 44 lists of 15 words that are semantically related to a critical word. Later, Stein and Gomes (2009) complemented the affective norms with norms of concreteness, word frequency, and semantic association. Word lists generally include mainly nouns and do not consider adjectives.

Relevance

Relevance is a dimension associated with words that have been recently explored in studies of social attitudes. Specifically, prejudice is a research area in which the relevance domain has been explored. The idea is that prejudice is better defined as a complex social phenomenon that encompasses a wide range of negative reactions toward out-groups than as a one-dimensional attitude like positive vs. negative or I like it vs. I do not like it (Degner, Wentura, Gniewosz, & Noack, 2007; Degner & Wentura, 2011; Mackie, Devos, & Smith, 2000; Smith, Seger, & Mackie, 2007). Researchers debate that beyond valence, different negativity types may be activated at the automatic processing level. Degner et al. (2007) and Degner and Wentura (2011) proposed a differentiation known as the possessor-relevance vs. other-relevance distinction. This differentiation is based on the distinction between other vs. self profitability, which was first introduced by Peeters (1983; see also Peeters & Czapinski, 1990). Peeters theorized that the evaluative meaning of a trait does not depend solely on its valence (i.e., positive vs. negative) but also on the trait's adaptive value for humans in general. A trait can be considered positive or negative while having adaptive value for the trait holder or social context. Within this perspective, prejudice is characterized as a negative attitude in two domains: (1) possessor-relevant domain (i.e., negativity associated with traits that are related to the target person such as being depressive and lonely) and (2) other-relevant domain (i.e., negativity associated with traits that are related to the target person and harmful to others who interact with such a person such as aggression and dishonesty). Other-relevance is related to derogation and hostility toward social groups that are negatively perceived as threatening to the social context (e.g., Turkish people in Germany who are socially perceived as hostile; Degner et al., 2007; Degner & Wentura, 2011). Possessor-relevance is associated with feelings of pity toward social groups such as older persons who are perceived as worthless, weak, or incompetent (Degner & Wentura, 2011).

The possessor-relevance vs. other-relevance distinction has been tested in studies that assessed the differentiation of implicit prejudice by using the Affective Priming paradigm (Fazio et al., 1995). In Germany, for example, prejudice toward Turks, elderly persons, and homeless people has been assessed through the evaluative priming task (Degner & Wentura, 2011). The main goal of the study was to determine whether the attitudes toward these social groups were negative and, if so, what relevance type would be automatically activated in response to group exemplars. The findings confirmed that negativity was associated with elderly persons and Turks. Importantly, the results indicated that this negativity was reflected in the priming effects based on possessor-relevant targets (i.e., older persons) and other-relevant targets (i.e., Turks). This means that as assessed implicitly, attitudes match how elderly persons and Turks are socially perceived (i.e., weak, lonely, worthless, nonthreatening with former, and hostile and threatening with the latter).

Materials used in affective priming experiments that evaluated prejudice differentiation include frontal portraits as primes (e.g., portraits of older persons vs. portraits of young persons) and words as targets. The words are positive and negative adjectives that can be further classified as either possessor-relevant or other-relevant (Degner et al., 2007; Degner & Wentura, 2011). Such a target set contains 20 German adjectives with five to eight letters and was developed by Wentura, Kulfanek, and Greve (2005) who selected the adjectives from a normative list of 908 adjectives (Hager, Mecklenbräuker, Möller, & Westermann, 1985; Möller & Hager, 1991) based on their pleasantness values. The absolute values were 50 or more on a scale that ranged from –100 to +100. They later generated normative data (Wentura, Rothermund, & Bak, 1998) in which in addition to being positive and negative, the adjectives had possessor-relevant and other-relevant meanings.

One can establish relevance norms by training a small group of participants with clear and detailed explanations regarding the possessor-relevance vs. other-relevance distinction. Ensuring that the participants recognize the distinction without any doubts is important. The reliability of categorizations may be estimated with Cronbach's alpha by taking the raters as items. The other possibility is to collect normative data using untrained participants such as in large-scale data collection studies that utilize questionnaires. In this case, one must provide explanations of the possessor-relevance vs. other-relevance distinction using a questionnaire instruction and observe how categorization is carried out. Although clearly in this case, error variance will occur, this is a valuable way of establishing relevance norms because it provides evidence of how ingrained the distinction is in ordinary language psychology. However, because of the resulting error variance, categorizations should be checked for non-randomness (i.e., participants understand the distinction and securely categorize items). Another aspect that must be emphasized is that beyond the way of collecting normative data, the instructions provided to the participants may also influence the participants' categorizations. Developing instructions that communicate well the relevance distinction is crucial. In the present study we obtained normative relevance data by using questionnaires.

Although ratings of pleasantness for Brazilian-Portuguese nouns are available (Santos et al., 2009; Stein & Gomes, 2009), Brazil still lacks norms for relevance. Therefore, conducting studies on the assessment of implicit prejudice and its differentiation that are comparable to studies performed in Germany is impossible. Establishing Brazilian norms of relevance for the words is clearly needed, specifically for adjectives that have been used in German experiments (Degner et al., 2007; Degner & Wentura, 2011; Wentura & Degner, 2010; Wentura et al., 2005). Thus, experiments on prejudice and its differentiation can be executed in Brazil, allowing for comparisons between Germany and Brazil.

Considering the need for normative data for adjectives regarding pleasantness and relevance within the same investigation, the present study sought to (a) generate a valid list of Brazilian-Portuguese adjectives (based on the German list by Wentura, Rothermund, & Bak, 1998), (b) establish Brazilian norms of pleasantness and relevance for the generated list of valid Brazilian-Portuguese adjectives, and (c) compare the Brazilian norms of the words included in the target set of Wentura et al. (2005) with the original German norms.

Overview

In the present study we sought to establish Brazilian norms for ratings of pleasantness and relevance for a sample of adjectives. These adjectives comprise an existing German list that includes the target set by Wentura et al. (2005). The final norms were generated in two phases. In Phase 1 we translated and validated to Brazilian-Portuguese an existing set of 186 German adjectives. In Phase 2 we generated the Brazilian norms of pleasantness and relevance. Finally, we compared the Brazilian norms with the German norms to generate a list of emotional adjectives that clinicians and researchers can use as standardized stimuli when developing social cognitive or neuropsychological tasks.

Methods

Participants

The sample included 385 participants. Because we were interested in college students and adults, we decided on two criteria to calculate the sample size: (1) participants should be from Porto Alegre (Rio Grande do Sul, Brazil) and (2) have studied for at least 11 years. We considered a 5% error rate¹. The participants consisted of 270 women and 111 men (four missing cases) with a median age of 25 years (range, 17–68 years). They were either college students from public and private universities and different faculties (i.e., Psychology, Nutrition, Nursing, Dentistry, Publicity, Journalism, Public Relations, Linguistics, Philosophy, Sports, Law, Sociology, and Agronomy) or adults with at least an undergraduate level of education (i.e., psychologists, engineers, journalists, historians, biologists, and nurses). Median years of education of participants was 16 (range, 11–24 years).

Materials

In Phase 1 we used a list of 186 German adjectives elaborated by Wentura et al. (1998) based on the 908 norm list by Hager et al. (1985) and Möller and Hager (1991). The list contained the 20 adjectives of the Wentura et al. (2005) target set. We initially translated the adjective list to Brazilian-Portuguese. Some words could not be translated in a one-to-one manner. Thus, some German words (n = 9) were translated into Brazilian-Portuguese as expressions (e.g., humorlos - sem senso de humor). We presented the translated list to be evaluated so that the German adjective was paired with its Brazilian-Portuguese translation (e.g., schön – bonito). Each adjective was rated on a 5-point scale according to two criteria, equivalence and clearness (Cassepp-Borges, Balbinotti, & Teodoro, 2010). Equivalence refers to the precision of the translation from German to Brazilian-Portuguese (i.e., the adjective is precisely translated from German to Brazilian-Portuguese). Clearness refers to how clear the adjective is when translated to Brazilian-Portuguese (i.e., the translated adjective is clear and understandable to the study population). A higher rating of the adjective indicates higher equivalence to the German version and a clearer translation to Brazilian-Portuguese. We provided space at the end of the list for the judges to write their suggestions.

Three bilingual judges² rated the translated adjectives on a 5-point scale according to the two aforementioned criteria, equivalence and clearness³. The judges evaluated the list in two rounds. They first rated all 186 translated adjectives. Their evaluations were then analyzed, and suggestions were made for 40 listed adjectives. Based on these suggestions, a second list that contained only those 40 adjectives was generated, which the judges then reevaluated.

To validate the German list of adjectives translated to Brazilian-Portuguese, we followed the procedure suggested by Hernández-Nieto (2002) and described by Cassepp-Borges et al. (2010). These authors recommended using a coefficient of content validity (CCV), which evaluates the amount of agreement among the judges. The judges must rate a series of items on a 5-point scale according to three areas: language clearness, item adequacy, and theoretical relevance. These recommendations best suit validating psychological instruments such as scales and questionnaires. However, because the aim of the present study was to validate a list of words, we opted for the two areas of equivalence and clearness. The judges thus did not rate the adjective list according to theoretical relevance. We accepted only items with a mean above 4 (on a scale from 1 to 5) for equivalence and clearness (i.e., CCV > .80). To avoid evaluation bias, we also calculated an error⁴.

After completing the analysis of the CCV, we eliminated 30 adjectives because their CCV did not reach .80 in either equivalence or clearness. We excluded another five adjectives, three because they were missing responses from one of the three judges and the other two because they lacked a proper translation into Brazilian-Portuguese. The list then consisted of 151 adjectives. After controlling for synonyms, the final list for Phase 2 contained 136 valid Brazilian-Portuguese adjectives. The CCV for the aggregate of the 136 adjectives was .90 for equivalence and .91 for clearness.

Procedure

We used the list of 136 adjectives validated for Brazilian-Portuguese from Phase 1 in Phase 2. Each listed adjective was rated according to its pleasantness and relevance. Pleasantness was rated on a 7-point scale, from –3 for "very negative" to +3 for "very positive." Relevance was evaluated according to three categories: "possessor-relevant," "0," and "other-relevant." Option "0" was used when participants could not decide between possessor-relevance and other-relevance. However, we instructed the participants to avoid as much as possible answering with "0" and label each adjective as either possessor-relevant or other-relevant. We included questions in the list that asked for demographic information such as gender, age, years of education, faculty, and profession. We randomized the final list of 136 adjectives four times to create four different test forms.

In groups of approximately 20 persons each, the participants rated the list of 136 Brazilian-Portuguese adjectives in two steps. First, they rated the adjectives according to their pleasantness. We instructed them to read each adjective and then decide whether it was negative or positive⁵. Once they finished this step, they rated the list according to the relevance associated with the adjectives. We told the participants to first think about the pleasantness of each adjective. If the adjective was positive, they should then consider which of the following would be better: if they have the trait or if someone with whom they have a relationship has the trait. For example, with intelligence, which of the following is best for you: that you yourself are intelligent or that someone with whom you have a relationship is intelligent? If the adjective was negative, then the participants should think of which of the following would be worse for them: that they have the trait or that someone with whom they have a relationship has the trait. With aggression, for example, which of the following is worse for you: that you yourself are aggressive or that someone with whom you have a relationship is aggressive toward you? If they thought that it would be better/worse to have the trait, then they should rate the adjective as possessor-relevant. If instead they thought it would be better/worse that someone with whom they have a relationship has the trait, then they should rate the adjective as other-relevant.

The Universidade Federal do Rio Grande do Sul Ethics Committee (Institutional Review Board) approved the present study. Participants provided written informed consent prior to data collection.

Results

Prior to the analysis, we calculated an estimate index of sensitivity separately for pleasantness and relevance based on Signal Detection Theory (SDT). This index indicates the proportion of correct answers that the participants gave when evaluating the adjectives by considering the German norms. Hits were correctly categorized positive or other-relevant words, respectively, and false alarms were incorrectly categorized negative or possessor-relevant words. The index varied from –1 (100% of errors) to 1 (100% of hits). A higher value indicates a higher proportion of hits.

We expected the sensitivity index to be lower for the relevance domain. Pleasantness is a clearly defined domain for evaluation. However, we assumed that relevance would not be as clear a domain for the participants to evaluate because they may not have previously heard of relevance associated with adjectives. For this reason, we established a cut-off point of .20 for selecting participants with regard to the relevance domain. We thought that due to the difficulty associated with this domain, the participants with less than 20% correct answers probably did not fully understand the instructions. Once we selected the individuals who answered at least 20% of the items correctly, the final sample for the relevance domain consisted of 210 participants. We did not use such a cut-off for the pleasantness domain; thus, its final sample included 385 participants. The mean of the sensitivity index for pleasantness was .86 (SD = .10) and for relevance was .42 (SD = .16).

After this analysis, we calculated the Brazilian means (and standard deviations/proportions) for ratings of pleasantness and relevance for each of the 136 adjectives. For this, we recoded the answer categories for the items' relevance into "1" for other-relevance, "1.5" for neither, and "2" for possessor-relevance. The complete rating set for the 136 words is presented in ^{Appendix A}. To determine whether the categorization of words was non-random, we performed a binomial test (disregarding the null-ratings and with Bonferroni-Holm adjustment) for each of the 136 adjectives within the two dimensions of pleasantness and relevance. With regard to pleasantness, the binomial tests indicated non-randomness of categorizations for all 136 adjectives. With regard to relevance, we found random categorizations for 34/136 adjectives (see ^{Appendix A}). Among these random categorized items, 12 were misclassified: just (justo), humane (humano), compassionate (compassivo), loving (amoroso), nice (simpático), delicate (delicado), beloved (querido), amiable (adorável), interesting (interessante), cowardly (covarde), slow (preguiçoso), and lame (letárgico). Interestingly, these adjectives are either other-relevant-positive or possessor-relevant-negative. To verify whether the categorizations of these types of adjectives were associated with more noise, we subjected the means of relevance for possessor-relevant-positive, possessor-relevant-negative, other-relevant-positive, and other-relevant-negative adjectives to a repeated-measures analysis of variance (ANOVA). We found an adjective type effect (F_2.05,429.18 = 246.30, p < .001, η² = .54). To further explore the main effect of adjective type, we performed the following planned contrasts: possessor-relevant-positive vs. possessor-relevant-negative and other-relevant-positive vs. other-relevant-negative. The participants categorized possessor-relevant-positive adjectives less ambiguously than possessor-relevant-negative adjectives (t₂₀₉ = 9.70, p < .001, d = 1.02). With regard to other-relevant adjectives, negative adjectives were less ambiguously categorized than positive adjectives (t₂₀₉ = –8.37, p < .001, d = –.85).

The next step was to compare the German norms for the 20 adjectives on the Wentura et al. (2005) target set with the Brazilian norms of the same target set. In Germany, valence norm values vary from –100 (very negative) to 100 (very positive), and relevance norm values vary from 1 (other-relevance) to 2 (possessor-relevance). The Brazilian norms for valence vary from –3 (very negative) to 3 (very positive) and for relevance vary from 1 (other-relevance) to 2 (possessor-relevance). Two adjectives from the German target set, "humane" and "slow," were evaluated differently in Brazil in terms of their relevance. "Humane" was evaluated as other-relevant in Germany (1.00) and possessor-relevant in Brazil (1.54). "Slow" was evaluated as possessor-relevant in Germany (1.85) and other-relevant in Brazil (1.41). Because of such incompatibilities, we chose two other adjectives to include in the target set: "faithful" (other-relevant; Germany: 1.05; Brazil: 1.27) and "depressed" (possessor-relevant; Germany: 1.98; Brazil: 1.66). Despite the incompatibility in the evaluation of "just" (gerecht/justo) between the German and Brazilian samples, we opted to retain this word because we believe it is an example of a prototypical other-relevant item. Table 1 presents the German and Brazilian relevance norm values for the Wentura et al. (2005) 20-item target set. Despite being correct, categorizations for the adjectives "kind," "honest," "listless," and "lifeless" were random (i.e., with considerable error variance). Table 2 depicts the German and Brazilian pleasantness norm values for the same target set with categorizations for all of the set being correct and non-random. Finally, Table 3 shows the frequency of agreement of Brazilians with the German norms of pleasantness and relevance. The data demonstrate that the values for the Brazilian and German norms were overall compatible. The pleasantness and relevance means for all of the adjectives, with the exception of "just," were within our expected range, with agreement frequencies that ranged from 44% to 99%. Again, the cells of other-relevant-positive and possessor-relevant-negative adjectives were noisier than the cells of other-relevant-negative and possessor-relevant-positive adjectives.

Discussion

We structured this study into two phases. The main goal of Phase 1 was to develop a valid list of Brazilian adjectives based on an existing list of 186 German adjectives. From the original list of 186 German adjectives, 136 were correctly validated and thus comprised the final list of Brazilian-Portuguese adjectives to be evaluated in Phase 2. The exclusion of 50 adjectives highlights the importance of this adaptation process. The common practice of only translating foreign stimuli and applying them to tasks that assess social cognition, memory, language, and other neurocognitive abilities can be harmful to an accurate evaluation process (Acevedo et al., 2009; Fonseca, Parente, Côté, & Joanette, 2007) even when they are simple words (e.g., in the case of this study, adjectives with emotional connotation). Validity evidence obtained through the equivalence between Brazilian and German adjectives is another index of the high quality of the adaptation process, allowing for cross-cultural studies.

Phase 2 aimed to establish Brazilian norms for two dimensions associated with the adjectives: pleasantness (positive vs. negative) and relevance (possessor-relevance vs. other-relevance). In Phase 2 we made available the pleasantness and relevance norms for the list of 136 German adjectives validated to Brazilian-Portuguese during Phase 1. These norms were also presented for the 20 adjectives of the investigated German target set (Wentura et al., 2005). Compared with the original target set, we retained 18 adjectives and modified two (i.e., "slow" and "humane") because of incompatibilities in their relevance evaluation by Brazilians and Germans. The two adjectives added to the target set—"faithful" and "depressed"—obtained the same evaluation by the participants. These differences in emotional judgment between Brazilian and German participants through linguistic stimuli may show that even paralinguistic aspects are culturally dependent.

The mode of establishing ratings of relevance did appear to play a role in how our untrained participants categorized the adjectives. This is reflected by the random categorizations observed for 34 adjectives from the set of 136. However, having chosen to test untrained participants with questionnaire instructions, error variance in categorizations was expected as a result of such a choice. Interestingly, noisy categorizations were mainly common for possessor-relevant-negative and other-relevant-positive adjectives (see ^{Appendix A} and Table 3). In this case we asked whether the participants' categorizations were biased by the specific instructions we gave them concerning relevance. We instructed the participants to first decide on the pleasantness of the adjectives and then categorize them as either possessor-relevant or other-relevant by answering the following question: What is best/worst for you: that you have the trait or that someone with whom you have a relationship has the trait? The expression "with whom you have a relationship" may have evoked a more concrete representation of others with whom they relate (e.g., parents, superiors, and teachers). Instructing the participants to think of people they "potentially have to interact with" may have been more useful. This could have evoked a more abstract idea of a wider range of potential interactions and a larger number of social roles. For example, the adjective "just" was categorized as possessor-relevant instead of other-relevant. "Just" is a clear example of an other-relevant adjective. It is good when people are just toward us. However, emphasizing the relationship domain may have elicited ambiguity. Although we want others to be just with us, dealing with just decisions that are not necessarily good for us can be difficult, such as when a parent makes a just but severe decision. The same may have happened with the categorizations of possessor-relevant-negative adjectives. It is indeed bad if I am lifeless, but it may be equally bad if someone close to me is lifeless. Thus, having instructed the participants to think of people with whom they had a relationship may have created some ambiguity regarding their categorizations of other-relevant-positive and possessor-relevant-negative adjectives. This is interesting, especially in the domain of the automatic evaluation process. In this case, one could ask what really drives such a process (i.e., the clear classification of a trait derived from an abstract decomposing of its semantic meaning or the ambiguity of the trait caused by spontaneous associations).

The list of adjectives generated can be considered valid and standardized for healthy adults in southern Brazil. As described by some international studies, some stimuli can be used to build experiments using behavioral or neuroimaging methods. For example, functional or emotional assessment scales can be developed in Brazil such as an adaptation of the Automated Neuropsychological Assessment Metrics (Johnson, Vincent, Johnson, Gilliland, & Schlegel, 2008), which includes dimensions of mood (vigor, depression, anxiety, happiness, among others). Additionally, the provided list of 136 Brazilian-Portuguese adjectives and their Brazilian norms of pleasantness and relevance may be useful for future Brazilian researchers who may conduct priming studies in which the targets are positive (e.g., feliZ, adorável) and negative (e.g., mandão, agressivo) adjectives. With regard to social cognition within the field of automatic prejudice, the list offers targets suited for investigating the other vs. possessor distinction. Theory of mind abilities, which can be considered an awareness of the relationship between two or more people's mental state that can lead behavior (van Buijsen, Hendriks, Ketelaars, & Verhoeven, 2011), could also be evaluated through these judgments of the pleasantness and relevance of words that qualify nouns.

In addition to the possibility of contributing to the development of scales, performance tests could be constructed or adapted. For example, assessment of facial expressions and emotions, a task that demands perceptual, emotional, and memory abilities, may benefit from the provided list. Adjectives such as alegre (serene), apático (listless), and colérico (angry) from this new Brazilian list may be associated with facial expressions depicted in photographs (e.g., study by Girardi, MacPherson, & Abrahams, 2011, in amyotrophic lateral sclerosis patients). Language aspects as lexical-semantic components and metacognitive abilities that represent one of the executive functions could be evaluated using a semantic judgment task with pairs of adjectives as stimuli. In this case, patients with aphasia or right-hemisphere communication impairment could judge whether a relationship exists between the words egoísta (selfish) and avarento (stingy).

Functional neuroimaging experiments could also utilize tasks with adjectives to verify activation and deactivation during the reading of pleasant and unpleasant emotional words (e.g., Herbert et al., 2009). These authors found a preference for processing pleasant content, followed by stronger activation in prefrontal regions. Crosson et al. (2002) showed specific neural connections between anterior frontal lobe and limbic regions where the semantic processing of words with emotional connotations occurs.

The applicability of the stimuli provided by this newly available list of adjectives with emotional content can be estimated. The present study was performed to provide Brazilian normative data for a list of German adjectives for only two aspects: emotional and social. Therefore, this list can still be validated and standardized with regard to other criteria such as arousal and the level of semantic association with nouns, famous faces, and familiarity, among others. We suggest that more studies should be conducted to standardize lists of stimuli for neuropsychological instruments in Brazil and Latin America, contributing to the diagnostic process in neurological and psychiatric populations.

Acknowledgments

This study was supported by a grant from the National Council for Scientific and Technological Development (CNPq). We thank Dirk Wentura for his valuable comments on an earlier version of this manuscript.

Received 3 June 2011

Received in revised form 12 August 2011

Accepted 12 August 2011

Available online 29 December 2011

• Acevedo, A., Krueger, K. R., Navarro, E., Ortiz, F., Manly, J. J., Padilla-Vélez, M. M., ... Mungas, D. (2009). The Spanish translation and adaptation of the Uniform Data Set of the National Institute on Aging Alzheimer's Disease Centers. Alzheimer Disease and Associated Disorders, 23, 102-109.
• Bellezza, F. S., Greenwald, A. G., & Banaji, M. R. (1986). Words high and low in pleasantness as rated by male and female college students. Behavior Research Methods, Instruments, & Computers, 18, 299-303.
• Bradley, M. M., & Lang, P. J. (1999). Affective Norms for English Words (ANEW): Instruction manual and affective ratings (Technical Report C-1). Gainesville, FL: Center for Research in Psychophysiology, University of Florida.
• Cassepp-Borges, V., Balbinotti, M. A. A., & Teodoro, M. L. M. (2010). Tradução e validação de conteúdo: Uma proposta para a adaptação de instrumentos [Translation and content validity: a proposal for the instrument adaptation]. In L. Pasquali (Ed.), Instrumentação psicológica: Fundamentos e práticas (pp. 506-520). Porto Alegre: Artmed.
• Crosson, B., Cato, M. A., Sadek, J. R., Gökçay, D., Bauer, R. M., Fischler, I. S., ... Briggs, R. W. (2002). Semantic monitoring of words with emotional connotation during fMRI: Contribution of anterior left frontal cortex. Journal of the International Neuropsychological Society, 8, 607-622.
• Degner, J., & Wentura, D. (2011). Types of automatically activated prejudice: Assessing possessor- versus other-relevant valence in the evaluative priming task. Social Cognition, 29, 182-209.
• Degner, J., Wentura, D., Gniewosz, B., & Noack, P. (2007). Hostility-related prejudice against Turks in adolescents: Masked affective priming allows for a differentiation of automatic prejudice. Basic and Applied Social Psychology, 29, 245-256.
• Eilola, T. M., & Havelka, J. (2010). Affective norms for 210 British English and Finnish nouns. Behavior Research Methods, 42, 134-140.
• Fazio, R. H., Jackson, J. R., Dunton, B. C., & Williams, C. J. (1995). Variability in automatic activation as an unobtrusive measure of racial attitudes: A bona fide pipeline? Journal of Personality and Social Psychology, 69, 1013-1027.
• Fonseca, R. P., Parente, M. A .M. P., Côté, H., & Joanette, Y. (2007). Processo de adaptação da Bateria Montreal de Avaliação da Comunicação - Bateria MAC ao Português Brasileiro [Adaptation process to Brazilian-Portuguese of the Montreal Communication Evaluation Battery MAC battery]. Psicologia: Reflexão e Crítica, 20, 259-267.
• Girardi, A., MacPherson, S. E., & Abrahams, S. (2011). Deficits in emotional and social cognition in amyotrophic lateral sclerosis. Neuropsychology, 25, 53-65.
• Hager, W., Mecklenbräuker, S., Möller, H., & Westermann, R. (1985). Emotionsgehalt, bildhaftigkeit, konkretheit und bedeutungshaltigkeit von 580 adjektiven: ein beitrag zur normierung und zur prüfung einiger zusammenhangshypothesen [Emotionality, imageability, concreteness, and meaningfulness of 580 adjectives: A contribution to norming and test of correlations]. Archiv für Psychologie, 137, 75-97.
• Herbert, C., Ethofer, T., Anders, S., Junghofer, M., Wildgruber, D., Grodd, W., & Kissler, J. (2009). Amygdala activation during reading of emotional adjectives: An advantage for pleasant content. Social Cognitive and Affective Neuroscience, 4, 35-49.
• Hernández-Nieto, R. A. (2002). Contribuciones al análisis estadístico [Contributions to statistical analyses]. Mérida, Venezuela: Universidad de Los Andes/IESINFO.
• Janczura, G. A., Castilho, G. M., Rocha, N. O., van Erven, T. J. C., & Huang, T. P. (2007). Normas de concretude para 909 palavras da língua Portuguesa [Concreteness norms for 909 Portuguese words]. Psicologia: Teoria e Pesquisa, 23, 195-204.
• Johnson, D. R., Vincent, A. S., Johnson, A. E., Gilliland, K., & Schlegel, R. E. (2008). Reliability and construct validity of the Automated Neuropsychological Assessment Metrics (ANAM) mood scale. Archives of Clinical Neuropsychology, 23, 73-85.
• Kensinger, E. A., & Corkin, S. (2003). Memory enhancement for emotional words: Are emotional words more vividly remembered than neutral words? Memory and Cognition, 31, 1169-1180.
• Lezak, M. D., Howieson, D. B., & Loring, D. W. (2004). Neuropsychological assessment. New York: Oxford University Press.
• Lieberman, M. D. (2007). Social cognitive neuroscience: A review of core processes. Annual Review of Psychology, 58, 259-289.
• Livingston, R. W., & Brewer, M. B. (2002). What are we really priming? Cue-based versus category-based processing of facial stimuli. Journal of Personality and Social Psychology, 82, 5-18.
• Lowery, B. S., Hardin, C. D., & Sinclair, S. (2001). Social influence effects on automatic racial prejudice. Journal of Personality and Social Psychology, 81, 842-855.
• Mackie, D. M., Devos, T., & Smith, E. R. (2000). Intergroup emotions: Explaining offensive action tendencies in an intergroup context. Journal of Personality and Social Psychology, 79, 602-616.
• Möller, H., & Hager, W. (1991). Angenehmheit, bedeutungshaltigkeit, bildhaftigkeit und konkretheit-abstraktheit von 452 adjektiven [Pleasantness, meaningfulness, imageability and concreteness of 452 adjectives]. Sprache und Kognition, 10, 39-51.
• Peeters, G. (1983). Relational and informational patterns in social cognition. In W. Doise & S. Moscovici (Eds.), Current issues in European social psychology (vol. 1, pp. 201-237). Cambridge: Cambridge University Press.
• Peeters, G., & Czapinski, J. (1990). Positive-negative asymmetry in evaluations: The distinction between affective and informational negativity effects. European Review of Social Psychology, 1, 33-60.
• Salles, J. F., Holderbaum, C. S., & Machado, C. S. (2009). Normas de associação semântica de 50 palavras do português brasileiro para crianças: Tipo, força de associação e set size [Semantic association norms of 50 Brazilian-Portuguese words for children: Type, association strength and target's set size]. Revista Interamericana de Psicologia, 43, 57-67.
• Salles, J. F., Holderbaum, C. S., Becker, N., Rodrigues, J. D., Liedtke, F. V., Zibetti, M. R., & Piccoli, L. F. (2008). Normas de associação semântica para 88 palavras do português brasileiro [Norms of semantic association for 88 Portuguese words]. Revista PSICO, 39, 362-370.
• Santos, R. F., Silveira, R. A. T., Gomes, C. F. A., & Stein, L.M. (2009). Normas de emocionalidade para a versão Brasileira do paradigma Deese-Roediger-McDermott (DRM) [Emotionality norms for the Brazilian version of the Deese-Roediger-McDermott (DRM) Paradigm]. Psicologia: Teoria e Pesquisa, 25, 387-394.
• Silverstein, A., & Dienstbier, R. A. (1968). Rated pleasantness and association value of 101 English nouns. Journal of Verbal Learning and Verbal Behavior, 7, 81-86.
• Smith, E. R., Seger, C. R., & Mackie, D. M. (2007) Can emotions be truly group level? Evidence for four conceptual criteria. Journal of Personality and Social Psychology, 93, 431-446.
• Stein, L. M., Feix, L. F., & Rohenkohl, G. (2006). Avanços metodológicos no estudo das falsas memórias: Construção e normatização do procedimento de palavras associadas [Methodological advances in false memory research: Development and norming of DRM procedure]. Psicologia: Reflexão e Crítica, 19, 166-176.
• Stein, L. M., & Gomes, C. F. A. (2009). Normas Brasileiras para listas de palavras associadas: Associação semântica, concretude, frequência e emocionalidade [Brazilian norms for word lists: Semantic association, concreteness, word frequency, and emotionality]. Psicologia: Teoria e Pesquisa, 25, 537-546.
• Strauss, E., Sherman, E. M. S., & Spreen, O. (2006). A compendium of neuropsychological tests: Administration, norms, and commentary, 3rd edition. New York: Oxford University Press.
• van Buijsen, M., Hendriks, A., Ketelaars, M., & Verhoeven, L. (2011). Assessment of theory of mind in children with communication disorders: Role of presentation mode. Research in Developmental Disabilities, 32, 1038-1045.
• Wentura, D., & Degner, J. (2010). Automatic evaluation isn't that crude! Moderation of masked affective priming by type of valence. Cognition and Emotion, 24, 609-628.
• Wentura, D., Kulfanek, M., & Greve, W. (2005). Masked affective priming by name letters: Evidence for a correspondence of explicit and implicit self-esteem. Journal of Experimental Social Psychology, 41, 657-663.
• Wentura, D., Rothermund, K., & Bak, P. (1998). [Ratings for possessor- vs. other-relevance of adjectives]. Unpublished raw data.
• Wittenbrink, B., Judd, C. M., & Park, B. (2001). Spontaneous prejudice in context: Variability in automatically activated attitudes. Journal of Personality and Social Psychology, 81, 815-827.

Appendix A

1

According to the data provided by the Brazilian Institute of Geography and Statistics (IBGE) – Results of the Demographic Census, 2000 - Malha municipal digital do Brasil: 2001 (available at Cidades@/IBGE -

www.ibge.gov.br/cidadesat). In Porto Alegre there are 303,476 inhabitants who are more than 10 years old and have studied between 11 and 14 years. There are also 164,151 inhabitants who are 10 years old or more and have studied for 15 years or more.

2

The judges were three bilingual Brazilians (two men and one woman) who were fluent in German and Brazilian Portuguese. They were 25, 39, and 60 years old. They all had graduated and studied for at least 17 years. Two of the judges were German teachers, and one was a native German speaker.

3

The instructions given to judges included the following: (1) Equivalence, referring to how accurate the translation is from German to Portuguese (i.e., was the adjective translated accurately [is equivalent] from German into Portuguese? At what level? (2) Clearness, referring to how clear each adjective was translated (i.e., is the translated adjective sufficiently clear, understandable, and appropriate for the study population? At what level?).

4

The error is calculated with the following formula in which J represents the number of judges who evaluated the item (Cassepp-Borges et al., 2010): The calculated error was .04 (i.e., 1/27).

5

The instructions given to the participants were the following: We are interested in knowing whether the words convey either positive or negative reactions or whether they describe either positive or negative objects/contexts.

Publication Dates

• Publication in this collection
  06 Mar 2012
• Date of issue
  Dec 2011

History

• Received
  03 June 2011
• Reviewed
  12 Aug 2011
• Accepted
  12 Aug 2011