Construction and evidence of validity of the Objective Test of Reaction to Frustration

Abstract

The aim of this article is to present the process of construction and initial validity evidence of the Brazilian Objective Test of Reaction to Frustation - TRFO. Constructed from a projective version, the instrument consists of 31 situations that pictorially portray different reactions to frustration. Each situation has 11 possible answers, which represent different possibilities of reactions to frustration. The items were constructed from the analysis of free responses of 112 participants. These responses were transformed into phrases that represent each of the 11 possible reactions to frustration. For the validity evidence analysis, based on the internal structure, the TRFO was applied to 1766 participants from different Brazilian states. All the reactions were co-related to each other with low to moderate variations, corroborating the findings in the literature. The data suggest that the TRFO is a promising instrument in the evaluation of reactions to frustration.

Keywords:
frustration; reaction to frustration; scale; validation; construction of instrument

Resumo

O objetivo deste artigo é apresentar o processo de construção e evidências iniciais de validade do Teste de Reação à Frustração Objetivo (TRFO). Construído a partir de uma versão projetiva da mesma medida, o instrumento consta de 31 situações pictóricas, consideradas frustrantes. Cada situação possui 11 possíveis respostas, que representam diferentes possibilidades de reação à frustração. Os itens foram construídos a partir da análise de respostas livres de 112 participantes. Essas respostas foram transformadas em frases que representam cada uma das 11 possíveis reações à frustração. Para análise de evidência de validade, baseado na estrutura interna, o TRFO foi aplicado em 1.766 participantes, de diferentes estados brasileiros. Todas as reações foram correlacionadas entre si, com variações de baixas a moderadas, corroborando os achados na literatura. Os dados sugerem que o TRFO é um instrumento promissor na avaliação de reações a frustração.

Palavras-chave:
frustração; reação à frustração; teste; validação; construção de instrumento

In our modern society the complexity and demands required from individuals are sometimes considered to be exorbitant. These conditions can be considered continuous sources of negative feelings such as outrage, despair and depression. If on one hand they entail huge opportunities and possibilities in challenging situations, on the other hand they may fail in satisfactorily meeting these needs. These feelings are considered to be universal or inherent to human nature. They depend on specific contexts like the life history of an individual and the context in which she is inserted, and can be merged into the concept of frustration (Moura, 2004). However, individuals are expected to overcome their difficulties and obstacles in the best possible way, in a consistent search for their survival and adaptability to the environment.

Literature provides a wide range of definitions for the frustration construct. However, there is a consensus that the existence of an obstacle or an interference on the process of trying to meet a need (Parrek, 1964), causing some reaction in the organism, is a frustration process. This study understands frustration as a negative emotional state rather than an obstacle or event. This emotional state or negative feeling is originated from the failure in fulfilling something or a need that is important to the individual, regardless if it is a real or a fictitious object (Moura & Pasquali, 2006; Rosenzweig, 1976; Roudinesco & Plon, 1944).

There are few recent studies on this topic registered in literature. However, when it is related to the phenomenon of aggression, this concept becomes more meaningful and present in scientific studies. Until the 1960s, the studies about frustrations were almost exclusively focused on identification of processes involved in aggressive responses ensuing from situations of frustration (Anderson & Bushman, 2002; Baron & Richardson, 1994; Berkowitz, 1989; 1990; 2001; Moura & Pasquali, 2006a; Rodriguez, 2015; Tice, Bratslavsky & Baumeister, 2001). However, in the last decades, despite the decreased causative emphasis between frustration-aggression, this this relationship still prevails in the scientific literature on frustration and aggression, but now related to other variables (Moura, 2004).

Literature depicts frustration as an individual trait, which is linked to aspects of the individual's health, labor and organizations (Beghi, Spagnoli, Airoldi, Fiordelli, Appollonio, Bogliun, Zardi, Paleari, Gamba, Frattola & Da Prada, 2002; Callejon, 2011; Dyer, Abrahams, Mokoena, Lombard & van der Spuy, 2005; Ferreira & Capitão, 2010; Harlos, 2001; Krejèí, Kvapil & Semrád, 1996; Moura & Pasquali, 2006a; Schaubroeck, Jones & Xie, 2001; Shirayama, Shirayama, Iida, Kato, Kajimura, Watanabe, Sekimoto, Shirakawa, Okawa & Takahashi, 2003; Susskind, 2004).

In the labor and health fields, the way people deal or cope with frustrations is related with workers’ performance and well-being (Cebulak, 2001; Fox & Spector, 1999; Reis & Faiad, 2014; Swan, 1972; Vasilopoulos, Cucina & Hunter, 2007; Yates & Pilai, 1992). In the context of organizations, being resistant to frustration is a profile required in many selection processes. That is why it is relevant, mainly in the Brazilian context where studies on this topic are scarce.

Although it is considered an important phenomenon to be investigated, there is no valid measure to analyze frustration. This scenario is worsened in the context of public security organizations, for example, where there is a greater need to evaluate the police officers' frustration. Considering the level of demand in the evaluation in selective processes and the lack of measurement instruments to meet these demands (Santos, 2016), psychological tests should ideally gather the wide range of spontaneous manifestations, typical of the projective tests (Formiga & Mello, 2000; Miguel, 2014; Pinto, 2014), which would allow collective application, as well as the required objectivity and standardization. Thus, this article aims to present the process of construction and the validity evidence of the Objective Test of Reaction to Frustration. This instrument was built from a project version named Projective Test of Reaction to Frustration (TRFP).

In this concept, frustration refers to a negative feeling and the reaction or reactions resulting from frustration are part of the individual's process of adaptation to the situation experienced. To propose, evaluate or measure frustration is to understand the potential reactions to it.

For the instrument building process we employed the model proposed by Pasquali (2010) which is divided into three pillars identified as theoretical, empirical and analytical poles. The first pole comprises the theoretical proposition of the instrument and the construction of items. The second pole comprises the process of applying the instrument, while the third represents, in this study, the indication of the test’s validity evidences analysis.

Theoretical Procedures

Firstly, the constitutive definition and the operationalization on the frustration construct were surveyed. This construct has been defined usually as a phenomenon that occurs when an individual is prevented from satisfying a need or a desire in face of some sort of obstacle or problem. This generates an affective reaction that is known as reaction to frustration or resistance to frustration. Thus, frustration takes the form of a negative affection reaction that occurs when the individual undergoes a situation considered to be frustrating, stressing, a problem or impairment that entails the non-fulfillment of something (Moura & Pasquali, 2006; Rosenzweig, 1976; Roudinesco & Plon, 1944).

The affective reaction of frustration manifests itself in several behavioral actions in the search for the solution of the problem that opposes to meeting the individual's needs. These reactions can be manifested in behaviors that attempt to solve the problem. These behaviors could be: a) elimination of the problem (attack, pummel, destroy, harass); b) avoidance of the problem (escape, ignore, deviate, sublimate, overrule); c) dialogue with the problem (agree, try alternative solutions, beg).

Construction of the Objective Test of Reaction to Frustration - TRFO

The Objective Test of Reaction to Frustration (TRFO) derives from its Projective version - TBRFP (Moura, 2008). The 2008 version was based on the Picture Frustration Test (PFT) built by Rosenzweig in the 1930s (LaVoie, 1986; Nick, n.d.; Rosenzweig, 1963; 1976; 1978a; Rosenzweig, Ludwig & Adelman, 1975) which proposed to evaluate what the author called "Resistance to Frustration". The PFT was part of a battery of tests of aggression. The adult version was created in 1944, further adjusted to the child version (Rosenzweig, 1978b) and both were published in several countries. In Brazil, however, there is no record of the date when the test was translated and adapted to Portuguese by Eva Nick (n.d.). The PFT also has an objective format proposed by Moura and Pasquali in a Brazilian study of 2006.

A critical analysis of the PFT, proposed by Moura (2004), unveiled the following limitations of the instrument: a) the issue of the standard used in studies of standardization for different countries, analysis also performed by Rauchfle (1971); b) the outdated features of the drawings based on the reality of the 1930s; c) thenon-feasible inquiry procedure for collective applications; d) the lack of consensus among evaluators in the correction procedure; e) the theory considered outdated for the understanding of the phenomenon; and, f) the issue of evaluating the individuals' consciousness when answering the test. This analysis gave rise to the proposal of the TBRFP (Moura, 2008) which grounded the instrument presented herein.

Characterization of the TBRFP

The Brazilian Projective Test of Reaction to Frustration (TBRFP) is composed of 31 drawings. Each drawing features two or more characters in situations considered frustrating. In each of the proposed situations, there is always one character who addresses another, saying something to the remainder characters that describes or verbalizes what is happening in the situation. The fact is always verbalized by the character to the left of the drawing. To respond to the situation, the test respondent is expected to put him/herself in the shoes of the character who must respond to the person that verbalized something, according to the model shown in Figure 1. The subject’s response is written in a blank space. In other words, when the subject puts him/herself in the place of the character who must respond to the situation, he/she defines how he/she responds or reacts to similar situations, defining their tendency of reaction to frustrating situations.

Figure 1
Reproduction of situation 1 of the TBRFP test

Construction of the Objective Test of Reaction to Frustration

The TBRFP was used as the starting point for the construction of an objective version of the test of reaction to frustration. An objective test would be more responsive to the strategic demands of a collective evaluation to be used in selective processes, for example. In this sense, this instrument to measure frustration was designed based on the proposal by Moura (2004) and Moura & Pasquali (2006a; 2006b) of objectification of a projective test, and on Bjerstedt’s (1965) proposal of efficaciously working on different PFT-based forms of administration and correction.

The Brazilian Objective Test of Reaction to Frustration (TBRFO) aims the objectification of the TRRFP test retaining only the projective stimulus elicited by the drawings. The format of free answers was replaced by close-ended items or statements that represent, in advance, each of the 11 likely reactions to frustration. The 11 reactions were investigated in a previous study, based on an analysis of the content of answers to the projective test. The analysis resulted in a sample of oral responses representative of these actions.

Participants. The study on instrument construction comprised 112 students of high school, graduate and undergraduate psychology degree course at the Universidade de Brasília and at a training course delivered by a federal public security institution. Among them, 61 (54.5%) were from the city of Brasília and 51 (45.5%) from the city of Porto Alegre. Participants’ age ranged from 16 to 66 years (A = 29.87 and SD = 11.79), being 68 (60.7%) men and 44 (39.3%) women Regarding education, nine (8%) had not concluded high school, 19 (17%) concluded high school, 47 (42%) had not concluded higher education, 27 (24.1%) had a degree, six (5.4%) were taking Master's degree, and two (1.8%) were taking Doctor's Degree. Participants were pooled in eight groups of 14 individuals each, randomly selected.

Initial instrument. To design the TRFO, eight answer books were assembled and named “Cadernos de TRFO” (TRFO Notebooks), based on the 31 situations proposed in the TBRFP projective test. Each answer book comprised three to four test situations. For each situation, statements were elaborated representing each of the 11 likely reactions to frustration.

Statements were based on three sources: a) the Objective Test of Resistance to Frustration - TRFO (Moura & Pasquali, 2006a); b) the Brazilian version of the PFT test handbook, by Eva Nick (n.d.); b) analysis of responses to the Rosenzweig's PFT answered by applicants undergoing a selective process; c) phrases prepared by two researchers based on their knowledge about the test.

Each answer book corresponded to the following test situations: book 1 (situations 1, 2, 3 and 4); book 2 (situations 5, 6, 7 and 8); book 3 (situations 9, 10, 11 and 12); book 4 (situations 13, 14, 15 and 16); book 5 (situations 17, 18, 19 and 20); book 6 (situations 21, 22, 23 and 24); book 7 (situations 25, 26, 27 and 28) and book 8 (situations 29, 30 and 31). All statements were coded and grouped to identify, to each of the 31 test situations, a range of responses comprising the TBRFP coding proposal. The reaction categories were indicated as: Ex, Ex’, Ex/, ex, In, In’, In/, in, Na, Na’, na, as described in Table 1.

Table 1
Identification and description of the 11 reactions to frustration proposed in the TRFO

#	Category or Reaction	Symbol	Description of the reaction
1	Extraceptive with emphasis on the obstacle	Ex’	The individual seeks the problem solution and makes it clear, in the response, that the situation is indeed frustrating. The individual gets irritated by the frustrating object or situation and does not know what to do. He/she may become hostile or aggressive, but always focusing on the situation and not on another person.
2	Extraceptive against a focus	Ex	The individual seeks the problem solution and, for that, offends a person or something that has caused frustration, or is in a situation of frustration. It is characterized as classic aggressiveness, hostility.
3	Extraceptive with apology	Ex/	The individual seeks the problem solution and ends up assaulting the other, although this aggressive behavior is followed by an apology for being aggressive. The individual is aware that he/she is being aggressive, and apologizes for that.
4	Extraceptive for resolution	ex	The individual seeks the problem solution and, for that, asks or demands that someone take action on the frustrating situation, or propose a solution for it.
5	Intraceptive with emphasis on the obstacle	In’	The individual seeks the problem solution and, for that, may react by blaming him/herself for the event, emphasizing in his/her response that he/she may have effectively caused a frustrating situation. At some moments, the subject gets angry with him/herself, with self-criticism.
6	Intraceptive against a focus	In	The individual seeks the problem solution and, for that, takes on the responsibility for the frustrating situation, apologizes and regrets what happened. He/she recognizes he/she is the cause of the frustrating situation.
7	Intraceptive with apology	In/	The individual apologizes for what happened, recognizes his/her responsibility, but tries to excuse him/herself for the fact as a way of defending him/herself. The person describes the reason why he/she took on the blame.
8	Intraceptive for resolution	in	The individual seeks the problem solution and offers to remedy the situation, i.e., he/she him/herself undertakes to solve the problem. The person takes the lead to help somebody or him/herself ir order to solve the situation; it refers to promptness.
9	Acceptive with emphasis on the obstacle	Na’	The individual seeks the problem solution and, for that, tries to minimize the frustrating situation. The individual tries to perceive the frustrating situation as a favorable, positive and provident event.
10	Acceptive against a focus	Na	The individual seeks the problem solution, although for that he/she avoids or escapes from the problem. He/she takes the attitude of exempting the other, the environment or him/herself from any responsibility on the frustrating situation.
11	Acceptive for resolution	na	The individual seeks the problem solution and, for that, affirms that there is no problem about the fact occurred because, the situation will either resolve itself or get solved over time.

The eight books contained the 31 situations of the original TBRFP test, followed by statements representing each test category. There was a statement to each category asking the participant to select, among thee options in the form of phrases (a, b, c), the one that best represented their way of expressing (aggressive, angry, tranquil) when reacting to a situation considered to be frustrating. In addition to three options, there was a fourth possibility (d) for the participant to suggest another possible speech. For example, to situation 1 of the instrument, there were up to 11 possible answers representing each of the following categories: Ex, Ex’ (please see an example on Figure 2), Ex, ex, In, In’, In, in, Na, Na’, na, where each one presented four options (ways of speaking). The participant should select one of the four options to each situation. Therefore, sets of objective responses were defined to each of the 31 situations. Each situation had 11 options of responses, totaling 341 items or variables.

Figure 2
Representation of Situation 1, Book 1, to the Ex’ reaction

Procedures. Respondents were approached in the classroom, upon prior permission of each instructor/teacher. The test was collectively applied by a psychology intern, following an explanatory sheet with all instructions to be given prior to application. To the graduation students, tests were applied individually. It met the ethical requirements, and the Free and Informed Consent Term was previously signed by all participants. The eight books were randomly distributed in the classrooms. Participants took 10 to 30 minutes to answer.

Results and Discussion. The subjects’ responses were analyzed regarding the occurrence of each option selected. The criteria for selecting the options included in the final instrument were as follows: (1) the most frequent response selected by respondents; (2) in the case of a tie regarding the frequency of responses, a judge defined the option. Therefore, only one of the four options of responses to each category was selected to better define each situation and be part of the final instrument, named TRFO.

Regarding the participants' decisions on each situation, responses widely varied. The options for each category were analyzed as a way of adopting the one that seemed to have better represented their way of speaking in situations 1 to 31 of the test. Then, this option was included in the final instrument, as shown in Table 2.

Table 2
Percentage of selection of options in situations 1, 2 and 3 of the TRFO book

Categories/dimensions	Phrase of Situation 1				Phrase of Situation 2				Phrase of Situation 3
	a	b	c	d	A	B	C	d	a	B	c	D
Ex’	7.1	21.4	64.3	7.1	28.6	28.6	42.9	0	35.7	21.4	42.9	0
Ex	14.3	21.4	64.3	0	42.9	35.7	21.4	0	42.9	28.6	28.6	0
Ex/	42.9	42.9	14.3	0	7.1	71.4	14.3	7.1	46.2	30.8	23.1	0
ex	14.3	35.7	50.0	0	42.9	7.1	50.0	0	21.4	28.6	50.0	0
In’	76.9	0	23.1	0	0	28.6	64.3	7.1	15.4	46.2	38.5	0
In	35.7	42.9	21.4	0	35.7	14.3	50.0	0	21.4	78.6	0	0
In/	64.3	7.1	28.6	0	35.7	28.6	28.6	7.1	46.2	30.8	15.4	7.7
in	64.3	14.3	14.3	7.1	35.7	21.4	35.7	7.1	21.4	21.4	50.0	7.1
Na’	14.3	50.0	35.7	0	35.7	0	64.3	0	35.7	28.6	28.6	7.1
Na/	30.8	38.5	30.8	0	50.0	28.6	14.3	7.1	30.8	53.8	7.7	7.7
Na	46.2	46.2	0	7.7	28.6	42.9	28.6	0	23.1	46.2	23.1	7.7

• Note. This table presents only part of the three first situations of the test.

Regarding the selection of the most adequate option, 16 items tied, and two researchers experts in the test decided on it. Moreover, the use of phrases to define the kinds of reaction showed that the difference between percentages regarding the selection of the phases that best defined each reaction were minimal in some cases. This suggests the possibility of building different versions of the instrument in further surveys. Another point is that the selection of phrases could be ensured by a larger sample than the one used, providing greater representativeness in the selection.

After being defined, the options were randomly distributed to avoid having the same order of categories. The answers gave rise to the Objective Test of Reaction to Frustration - TRFO (please see Figure 3).

Figure 3
Reproduction of item 1 of the TBRFP test

Figure 3 depicts the situation proposed by item 1 of the test. This figure shows that among each phrase representing a reaction or category of reaction, the most frequent in Book 1 was considered, as identified in Table 2. For options E/ and na, letter “a” was selected because researchers considered it clearer for tiebreaker purposes.

One can conclude that the TRFO presents options of response that represent a significant sample of reactions to frustration typically identified. The TRFO evaluation was defined with the same categories proposed in the TBRFP, according to the methodology developed by Moura (2004) and Moura & Pasquali (2006a). These categories have been previously presented and defined. A correction template was designed based on the 11 test options to evaluate the likely reactions to frustration. Therefore, the TRFO comprises 31 situations represented by drawings, each one of them with 11 possible answers. After the evaluation of the instrument with the template, the individual’s scores are calculated by summing the occurrences of each of the 11 reactions in the 31 situations.

Initial studies of evidence of validity of the TRFO

This article aims to present the initial evidences of validity based on the content (theoretical representation of the construct) and internal structure (AERA, 2014), and on the analysis of correlations between the test items (Primi, Muniz & Hutz, 2009). For that, the structure of correlation between the instrument's reactions was analyzed. Initially, the database was cleaned using the frequency analysis and variables distribution.

Participants. The study on initial evidence of validity of the TRFO was responded by 1,766 participants from the Mid-West (62.5%), Southeast (29.84%), North (4.64%), South (1.02%), Northeast (Salvador) (0.23%) regions. The remainder respondents did not identify their region of origin. Of the participants, 1,282 (72.6%) were men and 455 (25.8%) were women, with average age of 29.77 years (SD = 6.84).

Instrument. The TRFO instrument was applied, as previously described. The instrument is structured in 31 situations considered to be frustrating, represented by drawings that allow the analysis of 11 likely reactions to frustration.

Procedure. Psychology interns, duly trained, collectively applied the test in the classrooms. The ethical process was met, considering only the protocols with the Free and Informed Consent Term signed.

Results. Eleven individuals’ scores, representing the categories or reaction to frustration based on the 31 TRFO situations were created to analyze the TRFO test. The 11 scores were defined by the direction of the problem resolution (Extraceptive, Intraceptive or Acceptive) and how the individual tried to solve the problem (action with emphasis on the obstacle, reaction against a focus, reaction with apology, reaction for solving). Scores were designed using the test correction template. After submitting the answers to the correction template, the frequency of occurrence of each of the 11 reactions to the 31 situations of the instrument was analyzed, reaching a final score. Then, the measures of central tendency and dispersion were calculated to the 11 scores, as shown in Table 3.

Table 3
Descriptive statistics of the 11 reactions in the 31 situations proposed in the TRFO (N = 1,766)

Statistics	Ex’	Ex	ex	Ex/	In’	In	in	In/	Na’	Na	na
N	1457	903	1526	1407	1401	1628	1744	1615	1520	1675	1708
NR	309	863	240	359	365	138	22	151	246	91	58
Average	2.55	1.03	2.23	1.84	1.65	2.88	6.52	2.44	2.13	3.47	3.74
DP	2.07	1.48	1.65	1.56	1.38	1.89	2.91	1.55	1.60	2.06	1.98
Median	2.0	1.0	2.0	2.0	1.0	3.0	7.0	2.0	2.0	3.0	4.0
Asymmetry	0.85	2.59	0.76	1.06	1.02	0.72	0.16	0.57	0.96	0.69	0.39
EP Asymmetry	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06
Kurtosis	0.55	11.21	0.30	1.36	1.43	0.67	-0.23	0.03	1.23	0.70	-0.08
EP Short	0.12	0.12	0.12	0.12	0.12	0.12	0.12	0.12	0.12	0.12	0.12
Minimum	1	1	1	1	1	1	1	1	1	1	1
Maximum	11	13	9	10	9	13	18	8	9	12	11
P	0.08	0.03	0.07	0.05	0.05	0.09	0.21	0.08	0.07	0.11	0.12

• Datum. P: proportion of occurrence = Average/31. NR: No Reaction

The Data show that the Extraceptive reaction to solve the problem (in) had the highest occurrence, followed by the Acceptive to resolution (na) and the Acceptive against a focus (Na). The highest number of non-responses is found in the Extraceptive reaction against a focus (Ex), while the lowest number is for Intraceptive reaction for resolution (in). In these cases, data represent the quantity of non-occurrence of reaction throughout the instrument. The analysis of averages and ratio of occurrence shows that reactions appear in the following order: in (6.52; 21%), na (3.74; 12%), Na (3.47; 11%), In (2.88; 9%), Ex’ (2.55; 8%), In/ (2.44; 8%), ex (2.23; 7%), Na’ (2.13; 7%), Ex/ (1.84; 5%), In’ (1.65; 5%), Ex (1.03; 3%).

According to the data found, the sample presented occurrence in all reactions proposed. This is an indication that these are likely responses to frustrating situations. According to Rosenzweig (1949) one of the limitations to the evolution of reactions to frustration was that when an individual presents higher score in a reaction, for example, this would imply that the remainder scores in other reactions would be lower. In principle, this fact would be a limitation to the statistical interpretation of data. However, this finding led the author to consider the importance of investments in experimental and clinical studies to achieve more critical analysis of the categories proposed in the instrument. Further studies of validity (Rosenzweig, 1963; 1976; 1978b; Parrek, 1964), just like this study, showed a standard response in the test, i.e., that individuals effectively present a reaction that prevails in their way of reacting to frustrating situations, but other reactions are also part of this way of reacting, even in lesser degree. In this sense, there is a situation that prevails in the way an individual reacts, while the remainder appears as potential responses that emerge depending on the frustrating situation.

Regarding the analysis of dispersion of reactions, although distribution was not regular in all reactions, the large number of subjects in the sample softens the impact of this parameter on the analyses. The best normal distribution was found in the reaction in.

Analysis of Correlation between the TRFO reactions

Data in Table 4 show the correlations between the TRFO reactions. The Extraceptive reactions (Ex’, Ex, ex, Ex/) were positively correlated to each other; the Extraceptive reactions were negatively correlated with the Intraceptive reactions (In’, IN, in, In/), except for Ex and In/. There was a negative correlation between the Extraceptive and the Intraceptive reactions with Acceptive reactions (Na’, Na, na), thus corroborating the theoretical proposal of the instrument.

Table 4
Matrix of the TRFO (N=1,766) correlations

Reaction	Ex’	Ex	ex	Ex/	In’	In	in	In/	Na’	Na	na
Ex’	-
Ex	0.34***	-
ex	0.13***	0.20***	-
Ex/	0.32***	0.32***	0.16***	-
In’			-0.07***		-
In	-0.16***	-0.12***	-0.20***	-0.15***	0.12***	-
in	-0.28***	-0.38***	-0.07***	-0.28***	-0.22***	-0.19***	-
In/		0.08***	-0.10***			-0.07***	-0.12***	-
Na’	-0.29***	-0.15***	-0.15***	-0.12***			-0.10***	-	-
Na	-0.29***	-0.26***	-0.28***	-0.35***	-0.11***			-0.08***	-	-
na	-0.38***	-0.32***	-0.26***	-0.32***	-0.10***		0.12***	-0.10***	0.12***	0.19***	-

• *** Significant value p < 0,001

Table 4 also shows the negative correlation between Acceptive reactions (Na’) and Extraceptive (Ex’), both with emphasis on the situation (r = - 0.29). This shows that the more individuals tend to consider the frustrating situation as something favorable or provident, the less they seem to be irritated with facts. Regarding the Acceptive reaction against a focus (na), data show that the more individuals expect things to go well over time, higher is their tendencies to be irritated (Ex’) with the means (r = 0.38).

Final Remarks about the TRFO

Firstly, we could say that the objective instrument presents evidence of validity of content. The survey on all possible reactions to frustration in the test is a representative sample of behaviors (Pacico & Hutz, 2015) that express the frustration construction, according to the universe of behaviors delimited by Rosenzweig’s (1945; 1976) proposal. Still according to the AERA (2014) this validity is evidenced in the logical or empirical analysis when the test content represents the content of the domain assessed.

The TRFO proposal showed that this instrument can meet most of the demands for evaluation of this measure, mainly for use in collective applications (Moura, 2007; Moura & Pasquali, 2006a). Therefore, the objective measure emerges as a differential in comparison with the projective proposal, because it optimizes the correction time using the template, ensures more objectivity and makes clear the selective processes, for example. Based on the reformulation of the proposal on categorization of the responses to the PFT (Rosenzweig, 1945; 1976) and the adequacy of drawings and structure, one can say that the TRFO is a promising measure of reactions to frustration since, although being based on the PFT (Rosenzweig, 1945), it refers to an innovative proposal, with new theoretical, structural and evaluative framework.

For the psychological evaluation practice, the proposal of evaluating frustration through reactions shows that high or low resistance to frustration (terminologies largely used, mainly in the field of personnel selection) is not the only possible action of the individual. Based on the 11 possibilities of reaction, we should evaluate which highest tendency (highest score) we could expect among the individuals’ reactions.

Further studies could contribute to the analysis of evidences of validity of the instrument. The validity of criterion can be analyzed by comparing the reactions of frustration with data on work performance, mainly in the context of selective processes. Another important investment could be the identification of reactions that would be more or less appropriate for an applicant in the working context, considering that this measure has been required in selective processes, but misses scientific grounds.

Further research could also invest in the relationship between reactions to frustration and the emotions regulation process. Gondim and cols. (2015), referring to the emotions regulation process, say that different strategies are mobilized in the pursuing of fitting an emotional response to a positive or negative social situation (Aldao, 2013). This way, when one identifies frustration as a negative emotion subject to different kinds of reactions, the possibility of down regulation, as stated by Santana and Gondim (2016) would be an important contribution to the context of work.

Data of evidences based on the correlations between reactions corroborate Moura’s (2004) findings when comparing categories in the TORF test, and are an indicator of evidence of validity based on the internal structure analysis. The validity is based on the correspondence between the structure of indicators’ correlations (in this case, the reactions), and the structure foreseen by the theory was confirmed. Initial psychometric data proved to be a promising instrument to be further explored in other surveys that present new evidences of validity of the instrument, mainly applied to different contexts of psychological evaluation.

Referências

• Aldao, A. (2013). The future of emotion regulation research: Capturing context. Perspectives on Psychological Science, 8(2), 155-172. doi: 10.1177/1745691612459518
  » https://doi.org/10.1177/1745691612459518
• American Educational Research Association, American Psychological Association & National Council on Measurement in Education - AERA (2014). Standards for Educational and Psychological Testing. Washington: AERA.
• Anderson, C. A. & Bushman, B. J. (2002). Human aggression. Annual Review Psychology, 53(1), 27-51. doi: 10.1037/0033-2909.106.1.74
  » https://doi.org/10.1037/0033-2909.106.1.74
• Baron, R. A. & Richardson, D. R. (1994). Human aggression. New York: Plenun Press. doi: 10.1521/jscp.1983.1.2.97
  » https://doi.org/10.1521/jscp.1983.1.2.97
• Beghi, E., Spagnoli, P., Airoldi, L., Fiordelli, E., Appollonio, I., Bogliun, G., Zardi, A., Paleari, F., Gamba, P., Frattola, L. & Da Prada, L. (2002). Emotional and affective disturbances in patients with epilepsy. Epilepsy & Behavior,3(3), 255-261. doi: 10.1016/S1525-5050(02)00008-2
  » https://doi.org/10.1016/S1525-5050(02)00008-2
• Berkowitz, L. (1989). The frustration-aggression hypothesis: Examination and reformulation. Psychological Bulletin, 106(1), 59-73. doi: 10.1037/0033-2909.106.1.59
  » https://doi.org/10.1037/0033-2909.106.1.59
• Berkowitz, L. (1990). On the formation and regulation of anger and aggression: A cognitive-neoassociationistic analysis. American Psychologist, 45(4), 494-503. doi: 10.1037/0003-066X.45.4.494
  » https://doi.org/10.1037/0003-066X.45.4.494
• Berkowitz, L. (2001). On the formation and regulation of anger and aggression: A cognitive-neoassociationistic analysis. Em W. G. Parrot (Ed.), Emotions in social psychology: Essential readings (pp. 325-335). New York: MacGraw-Hill. doi: 10.1037/0003-066X.45.4.494
  » https://doi.org/10.1037/0003-066X.45.4.494
• Bjerstedt, A. (1965). Rosenzweig Picture-Frustration Study. In O. K. Buros (Ed.), The sixth mental measurements yearbook (pp. 511-515). Highland Park, NJ: The Gryphn Press.
• Callejon, C. (2013). Impacto emocional nos profissionais que atuam em funções de risco na atualidade. XXXIV Congresso Interamericano de Psicologia. Brasil: Brasília.
• Cebulak, W. (2001). Fairness, job frustration, and moral dilemmas in policing that impact police effectiveness. Journal of Police and criminal Psychology, 16(2), 48-57.
• Dill, J. C. & Anderson, C. A. (1995). Effects of frustration justification on hostile aggression. Aggressive Behaviour, 21(5), 359-369. doi: 10.1002/1098-2337(1995)21:5<359::AID-AB2480210505>3.0.CO;2-6
  » https://doi.org/10.1002/1098-2337(1995)21:5<359::AID-AB2480210505>3.0.CO;2-6
• Dyer, S. J., Abrahams, N., Mokoena, N. E., Lombard, C. J. & van der Spuy, Z. M. (2005). Psychological distress among women suffering from couple infertility in South Africa: a quantitative assessment. Human Reproduction, 20(7), 1938-1943. doi: 10.1093/humrep/deh845
  » https://doi.org/10.1093/humrep/deh845
• Ferreira, E. O & Capitão, C. G. (2010). Investigação do grau de tolerância à frustração em presidiários. Aletheia, 31, 91-110.
• Formiga, N. S. & Mello, I. (2000). Testes psicológicos e técnicas projetivas: uma integração para um desenvolvimento da interação interpretativa indivíduo-psicólogo. Psicologia Ciência e Profissão, 20(2), 12-19. doi: 10.1590/S1414-98932000000200004
  » https://doi.org/10.1590/S1414-98932000000200004
• Fox, S. & Spector, P. E. (1999). A model of work frustration-aggression. Journal of Organization Behavior, 20(6), 915-931. doi: 10.1002/(SICI)1099-1379(199911)20:6<915::AID-JOB918>3.0.CO;2-6
  » https://doi.org/10.1002/(SICI)1099-1379(199911)20:6<915::AID-JOB918>3.0.CO;2-6
• Gondim, S. M. G., Pereira, C. R., Hirschle, A. L. T., Palma, E. M. S., Alberton, G. D., Paranhos, J., Santana, V., & Ribeiro, W. R. B. (2015). Evidências de validação de uma medida de características pessoais de regulação das emoções. Psicologia: Reflexão e Crítica, 28(4), 659-667. doi: 10.1590/1678-7153.201528403
  » https://doi.org/10.1590/1678-7153.201528403
• Hair, J. F., Anderson, R. E., Tatham, R. L., & Black, W. C. (2005). Análise multivariada de dados. Porto Alegre: Bookman Editora.
• Harlos, K. P. (2001). When organizational voice systems fail: more on the Deaf-Ear Syndrome and frustration effects. The Journal of Applied Behavioral Science, 37(3), 324-342. doi: 10.1177/0021886301373005
  » https://doi.org/10.1177/0021886301373005
• Krejèí, P., Kvapil, J. & Semrád, J. (1996). The relation between job satisfaction, job frustration and narcissism and attitudes towards professional ethical behavior among police officers. Policing in Central and Eastern Europe: College of Police and Security Studies, Slovein. Acessado em http://www.unojust.org
  » http://www.unojust.org
• LaVoie, A. L. (1986). Rosenzweig Picture-Frustration Study. In D.J. Keyser & R.C. Sweetland (Eds.), Test Critiques, 14, 388-395. Kansas: Test Corporation of America.
• Miguel, F. K. (2014). Mitos e verdades no ensino de técnicas projetivas. Psico-USF, 19 (1), 97-106. doi: 10.1590/S1413-82712014000100010
  » https://doi.org/10.1590/S1413-82712014000100010
• Moura, C. F. M. (2004). Construção e validação de um teste de resistência à frustração. (Dissertação de Mestrado). Universidade de Brasília, Brasília.
• Moura, C. F. (2007). Teste Brasileiro de Resistência à Frustração: uma nova proposta. Trabalho apresentado no VI Congresso Iberoamericano de Evaluación Psicológica, Ciudad de México.
• Moura, C. F. (2008). Reação à frustração: construção e validação da medida e proposta de um perfil de reação. (Tese de doutorado). Universidade de Brasília, Brasília.
• Moura, C. F. & Pasquali, L. (2006a). Construção de um teste objetivo de resistência à frustração. Psico-USF, 11(2), 137-146.
• Moura, C. F. & Pasquali, L. (2006b). O Teste Brasileiro Projetivo de resistência à frustração. Trabalho apresentado no IV Congresso Nacional da Associação Brasileira de Rorschach e Métodos projetivos, Brasília, DF.
• Nick, E. (n.d.). Manual do teste de frustração - Forma para adultos. (S. Rosenzweig, Trad.) Rio de Janeiro: CEPA. (Trabalho original sem identificação de data).
• Pacico, J. C. & Hutz, C. S. (2015). Validade. Em C.S. Hutz, D.R. Bandeira & C. M. Trentini (Orgs.). Psicometria (pp. 71-95). Porto Alegre: Artmed.
• Parrek, U. N. (1964). Developmental patterns in reactions to frustration. Londres: Asia Publishing House.
• Pasquali, L. (2010). Testes referentes a construto: Teoria e modelo de construção. Em L. Pasquali (Org.). Instrumentação Psicológica: Fundamentos e Práticas (pp. 165 - 198). Porto Alegre: Artmed.
• Pastore, N. (1952). The role of arbitrariness in the frustration-aggression hypothesis. Journal of Abnormal Psychology, 47, 728-731. doi: 10.1037/h0060884
  » https://doi.org/10.1037/h0060884
• Pinto, E. R. (2014). Conceitos fundamentais dos métodos projetivos. Ágora 17 (1), 135-153. http://dx.doi.org/10.1590/S1516-14982014000100009
  » http://dx.doi.org/10.1590/S1516-14982014000100009
• Rauchfle, U. (1971). New possibilities for interpreting Rosenzweig Picture Frustration Test via use of indices. Psychologie, 30(4), 299-311.
• Reis, M. S & Faiad, C. (2014). Escala de Instabilidade Emocional para a Segurança Pública. Psico-USF, 19 (1), 87-96. http://dx.doi.org/10.1590/S1413-82712014000100009
  » http://dx.doi.org/10.1590/S1413-82712014000100009
• Rodriguez, C. (2015). Analog assessment of frustration tolerance: Association with self-reported child abuse risk and physiological reactivity. Child abuse & neglect, 46, 121-131.
• Rosenzweig, S. (1945). The picture-association method and its application in a study of reactions to frustration. Journal of Personality, 14(1), 3-23. doi: 10.1111/j.1467-6494.1945.tb01036.x
  » https://doi.org/10.1111/j.1467-6494.1945.tb01036.x
• Rosenzweig, S. (1963). Validity of the Rosenzweig Picture-Frustration Study with felons and deliquents. Journal of Consulting Psychology, 27(6), 535-536. doi: 10.1037/h0039628
  » https://doi.org/10.1037/h0039628
• Rosenzweig, S. (1976). Aggressive behavior and the Rosenzweig Picture-Frustration (P-F) Study. Journal of Clinical Psychology,32(4), 885-891. doi: 10.1002/1097-4679(197610)32:4<885::AID-JCLP2270320434>3.0.CO;2-R
  » https://doi.org/10.1002/1097-4679(197610)32:4<885::AID-JCLP2270320434>3.0.CO;2-R
• Rosenzweig, S. (1978a). An investigation of the realiability of the Rosenzweig Picture-Frustration (P-F) study children’s form. Journal of Personality Assessment, 42, 483-488. doi: 10.1207/s15327752jpa4205_7
  » https://doi.org/10.1207/s15327752jpa4205_7
• Rosenzweig, S. (1978b). Revised norms for the children’s form of the Rosenzweig Picture-Frustration (P-F) Study, with update P-F reference list. Journal of Clinical Child Psychology, 17(4), 326-328. doi: 10.1207/s15374424jccp1704_5
  » https://doi.org/10.1207/s15374424jccp1704_5
• Rosenzweig, S., Ludwig, D. J. & Adelman, S. (1975). Retest reliability of the Rosenzweig picture-frustration study and similar semiprojective techniques. Journal of Personality Assessment, 39,3-12. doi: 10.1207/s15327752jpa3901_1
  » https://doi.org/10.1207/s15327752jpa3901_1
• Roudinesco, E. & Plon, M. (1944). Dicionário de Psicanálise. Rio de Janeiro: Jorge Zahar Editor.
• Santana, V. S., & Gondim, S. M. G. (2016). Regulação emocional, bem-estar psicológico e bem-estar subjetivo. Estudos de Psicologia (Natal), 21(1), 58-68. doi: 10.5935/1678-4669.20160007
  » https://doi.org/10.5935/1678-4669.20160007
• Santos, A. P. (2016). The effects of legal judgment on psychological evaluations in Brazilian Federal Police. Police, 10(2), 113-120. doi: 10.1093/police/pav053
  » https://doi.org/10.1093/police/pav053
• Schaubroeck, J., Jones, J. R. & Xie, J. L. (2001). Individual Differences in Utilizing Control to Cope With Job Demands: Effects on Susceptibility to Infectious Disease. Journal of Applied Psychology, 86(2), 265-278. doi: 10.1037/0021-9010.86.2.265
  » https://doi.org/10.1037/0021-9010.86.2.265
• Shirayama, M., Shirayama, Y., Iida, H., Kato, M., Kajimura, N., Watanabe, T., Sekimoto, M., Shirakawa, S., Okawa, M. & Takahashi, K. (2003). The psychological aspects of patients with delayed sleep phase syndrome (DSPS). Sleep Medicine,4(5), 427-433. doi: 10.1016/S1389-9457(03)00101-1
  » https://doi.org/10.1016/S1389-9457(03)00101-1
• Susskind, A. M. (2004). Consumer Frustration in the Customer-Server Exchange: The Role of Attitudes Toward Complaining and Information Inadequacy Related to Service Failures. Journal of Hospitality & Tourism Research, 28(1), 21-43. doi: 10.1177/1096348003257328
  » https://doi.org/10.1177/1096348003257328
• Swan, H. F. (1972). Personnel induced frustration. The Canadian Chartered Accountant, 100(4), 48. doi: 10.1111/j.1754-7121.1971.tb00302.x
  » https://doi.org/10.1111/j.1754-7121.1971.tb00302.x
• Tice, D. M., Bratslavsky, E. & Baumeister, R. F. (2001). Emotional distress regulation takes precedence over impulse control: If you feel bad, do it! Journal of Personality and Social Psychology, 80(1), 53-67. doi: 10.1037/0022-3514.80.1.53
  » https://doi.org/10.1037/0022-3514.80.1.53
• Vasilopoulos, N. L., Cucina, J. M. & Hunter, A. E. (2007). Personality and training proficiency: issues of bandwidth fidelity and curvilinearity. Journal of Occupational and Organizational Psychology, 80(1), 109-131. doi: 10.1348/096317906X102114
  » https://doi.org/10.1348/096317906X102114
• Yates, D. L & Pilai, V. K. (1992). Frustration and strain among fort worth police officers. Sociology and Social Research, 76(3), 145-149.

Publication Dates

• Publication in this collection
  2016

History

• Received
  08 Sept 2016
• Accepted
  13 Dec 2016