Incremental Validity between the Wartegg and Rorschach tests (R-PAS)

Validade Incremental entre o Teste de Wartegg e o Rorschach (R-PAS)

Validad Incremental entre la Prueba de Wartegg y Rorschach (R-PAS)

Abstract

Incremental validity indicates how much a measure can add prevision to a criterion, more than what can be previewed by other sources of data. In other words, it means how an instrument can complement and aid on information comprehension derived from another. The objective of the study was to verify evidence of incremental validity between the Wartegg and the Rorschach tests (R-PAS). A total of 40 subjects with ages varying between 21 to 70 years participated, divided into two groups, one composed by schizophrenia diagnosis and another, by subjects with a history of psychiatric diseases. Everybody responded to the Rorschach and Wartegg tests. The results indicated predictive capacity among the instruments of 75% for the variable Formal Quality, 98% for Movement and 100% for Content. New studies are suggested about validity evidences with larger samples as well as the analysis of other variables, not explored in this study.

Keywords:
psychological assessment; schizophrenia; linear regression; projective techniques

Resumo

Validade incremental diz respeito ao quanto uma medida pode adicionar à previsão de um critério, acima do que pode ser previsto por outras fontes de dados, ou seja, de que forma um instrumento pode complementar e auxiliar na compreensão de informações obtidas por outro. O objetivo do estudo foi verificar evidências de validade incremental entre o Teste de Wartegg e o Rorschach (R-PAS). Participaram 40 sujeitos, com idades entre 21 a 70 anos, divididos em dois grupos, um composto por pacientes com diagnóstico de esquizofrenia e outro por sujeitos sem histórico de doença psiquiátrica. Todos responderam o Rorschach e o Teste de Wartegg. Os resultados indicaram capacidade preditiva entre os instrumentos de 75% para a variável Qualidade Formal, 98% para Movimento e 100% para Conteúdo. Sugere-se novos estudos acerca das evidências de validade, com amostras maiores e também análise de outras variáveis não exploradas no presente estudo.

Palavras-chave:
avaliação psicológica; esquizofrenia; regressão linear; técnicas projetivas

Resumen

Validad incremental dice respecto a lo cuanto una medida puede añadir a la previsión de un criterio, más de lo que puede ser previsto por otras fuentes de datos, o sea, de cual manera un instrumento puede complementar y auxiliar en la comprensión de informaciones obtenidas por otro. El objetivo del estudio fue verificar la evidencia de validez incremental entre la Prueba de Wartegg y lo Rorschach (R-PAS). Participado 40 sujetos con edades entre 21 y 70 años, divididos en dos grupos, un compuesto por pacientes con diagnóstico de esquizofrenia y otro, por sujetos sin histórico de enfermedad psiquiátrica. Todos respondieron a lo Roraschach y a la Prueba de Wartegg. Los resultados indicaron capacidad predictiva entre los instrumentos de 75% para la variable Calidad formal, 98% para Movimiento y 100% para Contenido. Se sugieren nuevos estudios acerca de las evidencias de validad, con amuestras mayores y también análisis de otras variables no exploradas en el presente estudio.

Palabras clave:
evaluación psicológica; esquisofrenia; regresión lineal; técnicas proyectivas

Psychological evaluation is defined as a process that aims to understand instances of the psychological functioning of individuals, in order to make possible understandings, thus operating scientifically based interventions (Cunha, 2000; Noronha & Alchieri, 2004). In this process, psychological tests stand out for their great usefulness in identifying nuances of psychological functioning, but they are not the only source of information available, other sources such as interviews, anamneses or even the synthesis of various instruments is necessary (Campos, 2013; Vieira, 2017).

Although there are objective parameters for the use of the tests, it is important to understand the social, clinical and theoretical context that underlies each decision making, considered as the main objective of the evaluation process (Villemor-Amaral & Pasqualini-Casado, 2006). To be used, the tests must present minimum acceptable parameters such as evidence of validity, reliability and standardization so that they can support decision making (Urbina, 2007). However, although these parameters give reliability to the data found, they do not guarantee that the process will be developed in a satisfactory manner, and may even be limiting to a comprehensive understanding when faced in a dichotomous manner (Campos, 2013; Cunha, 2000).

In this sense, some authors (Brackett & Mayer, 2003; Capitão & Cardoso, 2012; Franco & Villemor-Amaral, 2012; Haynes & Lench, 2003; Merino Soto, 2014; Silveira, Oliveira, & Bandeira, 2018) have used the proposal of incremental validity in the sense of complementing the data during a psychological evaluation process in different contexts. Broadly speaking, incremental validity refers to how much a measure can add to the prediction of a criterion, above what can be predicted by other data sources, that is, how one instrument can complement and assist in understanding information obtained by another (Hunsley & Meyer, 2003).

Andrei, Siegling, Aloe, Baldaro and Petrides (2016) and Haynes and Lench (2003) point out that there are some different objectives and models in the verification of incremental validity, being linked to the validity of the instrument’s predictive power, content and criteria, the effectiveness of treatments, monitoring of changes in individuals, force of discrimination of sensitivity and specificity in addition to the ecological validity linked to different contexts. However, the authors synthesize that, even with particularities, they all seek to verify the predictive efficacy of a particular phenomenon.

The operationalization in the search for incremental validity is done by a regression analysis in order to verify the predictive value of the instruments. The prediction is observed as the data sources are responsible for the variation of the observed criterion. Thus, they can contribute to the improvement in the prediction of important criteria such as diagnosis, treatment design or even future performance (Hunsley & Meyer, 2003; Ng & Feldman, 2015).

Authors like Brackett and Mayer (2003), Capitão and Cardoso (2012) and Ng (2015), employed incremental validity seeking to understand how the complementarity of data can assist in the process of evaluation, understanding and consequent treatment of an individual. These authors conclude that low correlations do not indicate a problem about evidence of validity. Even if low, they can have a great impact on the interpretations performed in clinical practice, even more when reinforced by other techniques and sources of information. These indicators work as points of convergence between the instruments, which in turn, access information in a differentiated way, thus involving different forms of mediation, as Schmitt, Hofmann, Gschwendner, Gerstenberg and Zinkernagel comment on (2015).

Regarding the two techniques used in this study, about the Rorschach test, it is possible to verify a large number of studies, which gives the technique the accumulation of favorable evidence as well as the constant development of new interpretations, which can be verified with the recent systematization performed by Meyer, Viglione, Mihura, Erard and Erdberg (2011), proposing the Rorschach Perfomance Assessment System (R-PAS), with the objective of improving the use of some variables used as well as optimizing the application (Mihura & Meyer, 2018; Pianowski, Meyer, Villemor-Amaral, Zuanazzi,l& Nascimento, 2019).

The Wartegg Test, on the other hand, has not been the object of many studies, disfavoring its practical applicability. Considering publications from 2015 onwards, only five works were found in international literature. Rizzo, Della Villa and Crisi (2015) conducted a case study with a 17-year-old boy with anxious symptoms and possible delusions. The Watergg Test was used in the search of clinical/qualitative indicators being possible to identify predominantly psychological discomfort and depressive indicators, according to the authors.

Crisi and Dentale (2016), conducted a study with the objective of verifying three classification categories of the Wartegg Test, namely evocative character (EC), formal quality (FQ) and affective quality (AQ). For this, 564 subjects participated, 290 male, with a mean age of 24.6 years (SD = 3.54) divided into three groups being anxious, psychotic and control group. The authors used the difference between means to verify significant differences between the groups. They also performed a precision analysis between judges, and 30 protocols of the study were evaluated by two judges finding good concordance rates (EC = 0.74; FQ = 0.92; AQ = 0.71).

In the same year, Engelman et al. (2016), conducted a case study of a 49-year-old woman with suspected attention deficit and hyperactivity (ADHD), with the objective of the clinical understanding of the patient from MMPI-2 data, the Rorschach (SC) and the Wartegg Tests. The authors observed that the Wartegg Test showed indicators of sadness and depression, emphasizing that it does not present specific indicators for ADHD.

Later, Vari et al. (2017) conducted a study to verify the correlation between skin diseases and depression, anxiety and suicidal ideation. For this they used the Wartegg Test, the Millon Clinical Multiaxial Inventory-III, in addition to a questionnaire on personality indicators. A total of 94 subjects participated in the study, 52 of whom were diagnosed with psoriasis and 42 were from the control group. The ages varied between 18 and 70 years (M = 42.05; SD = 11.21). The results for Wartegg indicated a difference between the groups for the variables affective quality (AQ) and global rejection (GR), and the authors concluded that the indicators favorable to the use of the tool were.

Finally, Yuana, Harjunowibowo, Karyanta and Budiyanto (2018) sought to verify the effectiveness of a computerized system, using similarity of cosine, for interpretation of content indicators for the Wartegg Test. In this study, they used five protocols of the test. The results indicated that the interpretations made from the software analyses, proved to be consistent.

In summary, in the last five years, five studies have been published with the Wartegg Test, being two case studies, one in which the test was used for information gathering, without, however, being the focus of the study and two in which its psychometric parameters were studied. This scenario motivated studies of the system proposed by Pessotto (2018a), based on the Rorschach test (R-PAS), and was later studied by Pessotto and Primi (2018c, 2018b). It is believed that studies involving these two instruments can help in the understanding of psychological instances through their incremental validity and clinical applicability. Therefore, the objective of this study was to verify evidence of incremental validity between the Wartegg and the Rorschach (R-PAS) tests based on indicators of formal quality, movement and content of the two tests.

Method

Participants

Forty subjects participated in this study and they were divided into two groups. Group 1 consisted of 20 patients diagnosed with schizophrenia from two institutions, a psychiatric hospital located in the interior of São Paulo and a public institution with care for this population in the capital. For group 2, the subjects were paired, with no history of psychiatric diagnoses, considering age (SD = 5), ethnicity, marital status, gender and schooling.

For all participants, age ranged from 21 to 70 (M = 40; SD = 12.8), 90% being male. About ethnicity, 70% declared themselves white, 15% black and 15% from other ethnic groups. Regarding schooling, 35% had incomplete elementary schooling, 25% complete elementary schooling, 20% incomplete high schooling, 5% complete high schooling, 5% incomplete high schooling, 5% complete high schooling and 5% post-graduation. The procedures were performed by the first author of this study, in two institutions in the interior of São Paulo, which offer psychiatric care, being the subjects of the clinical group approached directly in these institutions. In the control group, the subjects were contacted via internet, due to the statistical pairing and approached in their residences.

Instruments

Wartegg Test. Graphic self-expression test, which aims to identify aspects of personality. The method uses a stimulus sheet composed of eight squares of 4 cm X 4 cm. Each one contains a printed stimulus, and the subject is asked to continue the drawing, from these printed stimuli. The application lasts approximately 15 minutes. In this study the Pessotto system (2018a) was used. For this study, the variables formal quality (FQu and FQ-), movement (M, FM, m, a and p) and content (H, (H), Hd, A, Ad, (Ad), Na, Art, Ay, Cg, Fi and NC) were used, as they already present favorable evidence in previous studies (Pessotto & Primi, 2018a, 2018b). The formal quality indicates whether or not the design performed is adequate to the initial stimulus of the frame. Movement is used when the drawing indicates action of any nature and finally, the content refers to the elements present in the subject’s drawing. A deepening of the codes and their indicators presented in this system, can be verified in Pessotto and Primi (2018b, 2018c).

Rorschach Perfomance Assessment System (R-PAS). A technique by Meyer et al. (2011) consists in sequentially evaluating ten boards containing paint stains, asking him to answer the question “what could this be? In this system (R-PAS) the subject must provide two to three answers per board. This phase is called association. It is followed by the clarification phase in which each answer is taken up, seeking to identify two questions, namely, what made the subject see a certain object and where the stain is located. The R-PAS presents concordance rates between judges ranging from 0.86 to 0.89 in a set of interpretations and also presents concordance rates ranging from 0.77 to 0.96 in response coding. In addition, it offers several indexes of evidence of validity, such as the ability to perceive emotions correlated to the proper perception of reality ([FQ]r = 0.30) and generate emotional states to facilitate the work, correlated to lower levels of stressors ([m]r = 0.25).

In this study formal quality variables (FQo, FQu, FQ- and FQn) were used, which indicates the suitability of the figure seen to the contours of the stain, movement (M, FM, m, a and p), characterizing figures seen with some type of action and finally, contents (H, (H), Hd, A, (A), Ad, (Ad), Na, Art, Ay, Bl, Cg, Ex, Fi and NC), indicating the elements present in the response of the subject, such as human and animal figures, for example.

Procedures

Data collection. The applications were made individually in places made available by the collection fields. All subjects responded to the Wartegg and the Rorschach Tests with an average total duration of 40 minutes.

Data analysis. For the analyses, Pearson’s correlation was initiated, followed by Student’s t test in order to identify the differences between the variables for the two groups, followed by Cohen’s d test in order to verify the size of the effect of the expressive differences for each one of them, in the common variables between the techniques. Finally, hierarchical logistic regression was used to verify the incremental validity between the instruments. The Statistical Package for Social Sciences (SPSS) in version 21 was used for the analysis.

Ethical Considerations

The project was approved by the Research Ethics Committee of the Universidade São Francisco (Opinion No. 408,154, CAAE 20552313.6.0000.5514). After this approval, the subjects were invited to participate in the study by signing the Free and Informed Consent Term (FICT). According to resolution 196/96 of the National Health Council, the participant himself may sign the FICT, except in situations where it presents reduced capacity for self-determination. In these cases, a legal representative has been requested to sign the FICT.

Results

For the analyses carried out in order to achieve the objectives of this study, a set of variables was selected, namely formal quality, movement and contents, as described in the method. For the formal quality variables (FQ) besides using them separately, the Rorschach test FQo and FQu codes were added, because in the Wartegg Test there is still no frequency data referring to this variable, presenting only FQu. Pearson’s correlation between the variables was then employed in order to observe similarities between them. The results can be seen in Table 1.

Table 1
Pearson’s correlation between the Rorschach and Wartegg Tests

		Wartegg
		H	(H)	Hd	(Hd)	A	Ad	(Ad)	An	Art	Ay
Rorschach	H	0.60**	-0.11	0.12	0.01	0.02	0.08	-0.08	-0.08	-0.08	0.08
	(H)	-0.04	-0.05	-0.09	0.13	0.36*	0.04	-0.15	-0.02	-0.15	0.11
	Hd	0.2	-0.15	0.23	-0.09	-0.07	0.04	-0.01	-0.11	-0.01	0.1
	(Hd)	-0.04	-0.1	0.08	-0.18	-0.2	0.01	-0.1	0.21	-0.1	0.01
	A	0.04	0.09	0.06	0.07	0.50**	-0.12	0.01	-0.27	0.01	-0.01
	(A)	-0.07	0.19	0.09	0.12	0.40*	-0.03	-0.11	-0.11	-0.11	0.29
	Ad	-0.11	-0.13	0.14	-0.2	-0.16	-0.07	0.44**	0.25	0.44**	0.04
	(Ad)	-0.09	-0.08	-0.03	0.33*	-0.16	0.22	-0.05	-0.05	-0.05	-0.1
	An	-0.05	-0.12	0.06	-0.01	-0.01	-0.08	-0.12	0.26	-0.12	-0.13
	Art	0	-0.13	-0.15	0.09	-0.03	-0.17	-0.1	-0.1	-0.1	0.06
	Ay	-0.02	-0.06	-0.09	0.06	-0.12	0	-0.06	-0.06	-0.06	-0.11
	Bl	-0.08	-0.06	-0.18	-0.12	-0.14	0.28	-0.05	-0.05	-0.05	-0.08
	Cg	0.06	-0.12	0.16	-0.09	0.39*	-0.03	0.03	0.18	0.03	0.23
	Ex	-0.04	-0.04	-0.1	-0.07	-0.08	-0.05	-0.03	-0.03	-0.03	-0.05
	Fi	0.18	-0.11	-0.02	0.08	-0.16	0.3	-0.08	-0.08	-0.08	0
	NC	-0.07	0.05	-0.31	0.03	0.12	0.24	-0.14	-0.14	-0.14	-0.07
	FQu	0.12	0	0.22	0.04	0.26	-0.05	-0.01	0.25	-0.01	-0.01
	FQ-	0.07	0.01	-0.08	-0.2	0.19	0.07	-0.17	-0.17	-0.17	0.28
	FQn	-0.08	-0.03	-0.23	-0.03	-0.13	-0.08	-0.06	-0.06	-0.06	-0.04
	FQo	0.02	-0.07	0.21	0.19	-0.04	0.09	0.07	-0.16	0.07	-0.12
	FQo/u	-0.07	-0.08	0.26	0.23	0.05	0.02	0.01	0.14	0.01	-0.11
	M	0.45**	-0.14	0.27	-0.16	0.57**	0.02	-0.2	0.01	-0.2	0.19
	FM	0.21	-0.1	0.03	0	0.49**	0	0.04	-0.21	0.04	0.23
	m	0.05	0.05	-0.16	0.15	0.23	0.14	0.27	-0.12	0.27	-0.21
	a	0.32*	-0.06	-0.01	-0.01	0.59**	0.15	-0.12	-0.16	-0.12	0.22
	p	0.16	-0.19	0.40*	-0.14	0.18	-0.31*	0.24	0.01	0.24	0.02
		Wartegg
		Cg	Fi	NC	FQu	FQ-	M	FM	m	a	p
Rorschach	H	0.07	-0.11	0	0.08	-0.05	0.32*	0.23	0.02	0.31	-0.04
	(H)	-0.22	-0.13	0.15	-0.09	0.1	0.13	0.22	0	0.07	0.13
	Hd	0.28	-0.16	-0.02	-0.08	0.06	-0.02	-0.08	0.13	0.1	-0.09
	(Hd)	0.08	0.08	-0.02	0.13	-0.18	-0.09	-0.05	0.25	0.13	-0.1
	A	-0.05	0.12	-0.18	-0.03	0.04	0.33*	0.27	-0.01	0.14	0.27
	(A)	0.04	-0.16	-0.14	-0.19	0.22	-0.03	0.06	-0.2	-0.07	-0.09
	Ad	0.15	-0.06	-0.02	0.21	-0.22	-0.18	0.02	0.2	0.12	-0.2
	(Ad)	-0.08	-0.08	-0.01	-0.01	0.02	0.05	-0.11	-0.03	-0.07	0.1
	An	-0.18	0.04	0.14	0.26	-0.23	-0.01	-0.06	-0.02	-0.11	0.13
	Art	0	0	0.08	0.25	-0.3	-0.14	0	0.1	0.02	-0.08
	Ay	-0.09	0.05	0.16	-0.1	0.13	-0.01	-0.13	0.01	-0.07	0.04
	Bl	-0.07	-0.07	0.25	-0.34*	0.35*	-0.17	-0.1	0.02	-0.04	-0.16
	Cg	0.05	-0.06	-0.01	0.02	0.01	0.06	0.07	-0.05	0.01	0.04
	Ex	-0.04	-0.04	0.17	0.11	-0.11	-0.1	-0.05	-0.08	-0.07	-0.09
	Fi	-0.12	-0.12	0.01	-0.24	0.22	0.16	-0.04	-0.06	0.03	0.05
	NC	-0.1	-0.01	0.09	-0.44**	0.45**	-0.02	-0.15	-0.09	-0.13	0.03
	FQu	0.22	0.13	-0.25	0.26	-0.24	0.28	0.05	0.18	0.08	0.40*
	FQ-	-0.04	-0.19	0.18	-0.38*	0.41**	-0.28	0.09	-0.18	-0.06	-0.36*
	FQn	-0.09	-0.05	0.29	-0.1	0.12	-0.17	-0.1	-0.17	-0.17	-0.13
	FQo	-0.07	0.15	-0.15	0.25	-0.27	0.46**	0.03	0.16	0.23	0.32*
	Fqo/u	-0.05	0.27	-0.23	0.3	-0.31	0.47**	0	0.12	0.15	0.40**
	M	0.12	-0.09	-0.05	0.03	0	0.34*	0.39*	-0.02	0.31*	0.02
	FM	-0.08	-0.03	0.03	0.08	-0.05	0.45**	0.59**	0.04	0.41**	0.14
	m	-0.17	0.11	-0.16	-0.3	0.3	0.2	-0.04	0.09	0.11	0.12
	a	-0.03	-0.17	-0.02	-0.08	0.11	0.34*	0.50**	-0.03	-0.3	0.11
	p	0.02	0.33*	-0.08	0.26	-0.25	0.49**	0.2	0.12	0.41**	0.07

• Note. ** Significant correlation at 0.01 level; * Significant correlation at 0.05 level.

In Table 1 it is possible to observe a correlation of strong magnitude (0.60), 21 correlations ranging from 0.40 to 0.59 and 17 from 0.31 to 0.39. The greatest correlation found was verified between the H contents of the two techniques, signaling similarity in their variance. It is also possible to observe correlations between the content codes of both techniques comprising (Hd), A, Ad, (Ad) and Art from Wartegg and (H), A, Ad, (Ad), Cg from the Rorschach test varying between 0.33 and 0.50.

Another set of correlations were observed between the movement indicators ranging from 0.31 to 0.59 with this value of greater magnitude being verified between the FM codes of the two techniques. Other correlations can still be observed, but in general, indicate covariances between the indicators of the two tests.

Then the Student’s t test was applied in order to verify the differences between the variables for the two groups, as well as Cohen’s d, in order to verify the size of the effect of the differences between the groups. These indexes can be seen in Table 2.

Table 2
Descriptive statistics, Student’s t test and Cohen’s d test for groups with schizophrenia (Eqz) and normative (Nor) for Rorschach and Wartegg Tests

Rorschach									Wartegg
Var.	Group	M	SD	N	t	p	d		Var.	Group	M	SD	N	t	p	d
H	EQ	1.4	1.27	20	-1.102	0.277	0.35		H	EQ	0.05	0.22	20	-0.831	0.411	0.26
	N	1.85	1.31	20						N	0.15	0.49	20
(H)	EQ	1.05	1.32	20	-0.623	0.537	0.20		(H)	EQ	0.45	1.57	20	1.127	0.267	-0.36
	N	1.3	1.22	20						N	0.05	0.22	20
Hd	EQ	1.2	1.61	20	0.4	0.692	-0.13		Hd	EQ	0.45	0.76	20	-0.873	0.388	0.28
	N	1	1.56	20						N	0.7	1.03	20
(Hd)	EQ	0.4	0.60	20	0.9	0.374	-0.28		(Hd)	EQ	0.1	0.45	20	-1.515	0.138	0.48
	N	0.25	0.44	20						N	0.35	0.59	20
A	EQ	6.95	3.71	20	-1.501	0.142	0.47		A	EQ	0.2	0.52	20	-0.326	0.746	0.10
	N	8.55	3.00	20						N	0.25	0.44	20
(A)	EQ	0.4	0.60	20	0	1	0.00		(A)	EQ	0	0.00	20	-	-	-
	N	0.4	0.60	20						N	0	0.00	20
Ad	EQ	1.4	1.79	20	-0.094	0.926	0.03		Ad	EQ	0.15	0.37	20	1	0.324	-0.58
	N	1.45	1.57	20						N	0	0.00	20
(Ad)	EQ	0.1	0.31	20	0	1	0.00		(Ad)	EQ	0.05	0.22	20	1	0.324	-0.32
	N	0.1	0.31	20						N	0	0.00	20
An	EQ	1.45	1.93	20	-0.445	0.659	0.14		An	EQ	0.05	0.22	20	1	0.324	-0.32
	N	1.75	2.31	20						N	0	0.00	20
Art	EQ	0.45	1.00	20	-0.369	0.714	0.12		Art	EQ	0.05	0.22	20	1	0.324	-0.32
	N	0.55	0.69	20						N	0	0.00	20
Ay	EQ	0.45	1.10	20	0.953	0.347	-0.30		Ay	EQ	0	0.00	20	-1.831	0.075	0.58
	N	0.2	0.41	20						N	0.15	0.37	20
Bl	EQ	0.15	0.37	20	1.831	0.075	-0.58		Bl	EQ	0	0.00	20	-	-	-
	N	0	0.00	20						N	0	0.00	20
Cg	EQ	0.65	1.27	20	-0.716	0.478	0.23		Cg	EQ	0.05	0.22	20	0	1	0.00
	N	0.9	0.91	20						N	0.05	0.22	20
Ex	EQ	0	0.00	20	-1	0.324	0.32		Ex	EQ	0	0.00	20	-	-	-
	N	0.05	0.22	20						N	0	0.00	20
Fi	EQ	0.3	0.57	20	-0.238	0.813	0.08		Fi	EQ	0.05	0.22	20	0	1	0.00
	N	0.35	0.75	20						N	0.05	0.22	20
Sx	EQ	0	0.00	20	-	-	-		Sx	EQ	0	0.00	20	-	-	-
	N	0	0.00	20						N	0	0.00	20
NC	EQ	5.85	4.68	20	1.15	0.257	-0.36		NC	EQ	6.1	2.00	20	-0.673	0.505	0.21
	N	4.5	2.37	20						N	6.45	1.19	20
FQu	EQ	5.2	2.35	20	-2.37	0.023	0.75		FQu	EQ	4.15	3.53	20	-3.974	0.001	1.20
	N	7	2.45	20						N	7.45	1.64	20
FQ-	EQ	4.2	2.33	20	2.251	0.03	-0.71		FQ-	EQ	3.65	3.63	20	3.56	0.001	-1.13
	N	2.6	2.16	20						N	0.5	1.57	20
M	EQ	2.35	1.93	20	-1.41	0.167	0.45		M	EQ	0.1	0.45	20	-3.126	0.003	0.99
	N	3.35	2.52	20						N	0.7	0.73	20
FM	EQ	2.35	2.01	20	-2.631	0.012	0.83		FM	EQ	0	0.00	20	-2.179	0.036	0.69
	N	4.3	2.64	20						N	0.2	0.41	20
m	EQ	0.65	0.88	20	0.372	0.712	-0.12		m	EQ	0.35	0.67	20	0.515	0.609	-0.16
	N	0.55	0.83	20						N	0.25	0.55	20
a	EQ	3.8	3.94	20	-1.835	0.074	0.58		a	EQ	0.45	0.89	20	-0.15	0.881	0.05
	N	6.1	3.99	20						N	0.5	1.19	20
p	EQ	1.55	1.61	20	-1.558	0.128	0.49		p	EQ	0	0.00	20	-4.333	0	1.37
	N	2.25	1.21	20						N	0.65	0.67	20

The results of Table 2 are favorable to the objectives of this study. It is possible to observe that the variables related to formal quality (FQu and FQ-) in both techniques presented significant differences between the groups. In addition, the averages for the M and FM of the Wartegg and the Rorschach Tests, were also verified as different among the groups.

Finally, linear regression analysis was used to verify the incremental validity between the two techniques. For this analysis we used the same variables indicated above, but grouped in broad indicators, namely formal quality, movement and contents. The analysis was performed separately for each of the variables. The results can be seen in Table 3.

Table 3
Ranking table and predicted percentage for regression analysis

Formal quality			Predicted
			group		% correct
	Observed		EQ	Nor
	group	EQ	15	5	75
		Nor	5	15	75
	% global				75%
Movement			Predicted
			grupo		% correct
	Observed		EQ	Nor
	group	EQ	20	0	100
		Nor	1	19	95
	% global				98%
Content			Predicted
			group		% correct
	Observed		EQ	Nor
	group	EQ	20	0	100
		Nor	0	20	100
	% global				100%

In Table 3 it is possible to verify good predictive indices for the 3 variables. For the formal quality it is observed that the values for true positives and negatives were equal, reaching 75% of the cases (R2 Cox & Snell = 0.416 and R2 Nagelkerke = 0.554).

For movement, there are 20 true positives and 19 true negatives, predicting 98% of the cases (R2 Cox & Snell = 0.725 and R2 Nagelkerke = 0.967), Finally, for content it is possible to observe that all true positives and negatives were found, predicting 100% of the cases (R2 Cox & Snell = 0.750 and R2 Nagelkerke = 1).

Discussion

Initially, it is possible to verify that the correlations observed in Table 1 corroborate with the objective of this study going against the initial hypotheses about their interpretations. The correlation between the H codes in the two techniques indicates good similarity between them, that is, as the person demonstrates that he is able to perceive and perceive the other in a realistic and multifaceted way in the Rorschach test, tends to demonstrate the same in Wartegg (r = 60). Moore, Viglione, Rosenfarb, Patterson and Mausbach (2013) signal that this is an indicator of mental representations linked to interpersonal relations having observed correlation between the Rorschach (R-PAS) test and Social Skills Performance Assessment (SSPA).

Still considering indicators linked to the focus of attention of the individual [(Hd), A, Ad, (Ad) and Art of Wartegg and (H), A, Ad, (Ad), Cg of Rorschach], the correlations observed proved positive for the scope of this study. In this sense, there is similarity in the perception of details or even more simplified perception, not encompassing the whole, between the response process of the two tests. Exner and Sendin (1999), Mihura, Meyer, Dumitrascu and Bombel (2013) and Moore et al. (2013) point out that beyond the relationship, the focus of interest of the subject, can signal interest in the other, indicating questions about personal relationships. These results show good integration between the techniques, informing that to some degree, these indicators seem to access similar latent abilities, verified by the covariances observed.

The correlations observed between the movement indicators of the two techniques signal some level of participation of the perception/perception mechanisms linked to primary needs, especially for the variable FM (Mihura et al., 2013). Therefore, it is possible to understand that, in general, responses containing movement are linked to internal inferences, since effectively, the stains do not present movement. Thus, the effort to foresee an action or emotion in the Rorschach test, is in line with the same indicator in the Wartegg Test.

The same can be observed in formal quality variables. Several authors (Marques, Chaves, & Yazigi, 2012; Pianowski & Villemor-Amaral, 2010; Yazigi et al., 2016) also observed a relationship between formal quality and perceptual elements, usually with patients diagnosed with schizophrenia, this being a good indicator for the disorder. In this sense, observing the correlations between the formal quality codes, it is possible to notice that as the subject tends to distort his perception in the Rorschach test, providing poor responses to the contours of the stain, he also tends to complete the figures of the Wartegg Test without much reference to the initial stimulus of the frame, that is, with little integration in his/her perception between the initial stimulus and the final product.

Other correlations can still be observed, but in general, they signal covariances between the indicators of the two tests. It is important to consider the differences between the nature of the stimuli of the two techniques, making it necessary other studies for further inferences about the intepretations for the Wartegg Test. However, this analysis points to the existence of covariance, without however presenting high correlations, indicating that part of the latent ability accessed is shared in some way, but there is still part of the variance that is not accessed. In this sense, the Student’s t test was applied in order to verify the differences and Cohen’s d to find the size of the effect of the differences between the groups evaluated by both techniques. These indexes can be seen in Table 2.

The differences found in the variables linked to formal quality (FQu and FQ-) in both techniques were already expected for the Rorschach test (Pianowski, Meyer, & Villemor-Amaral, 2016; Pianowski & Villemor-Amaral, 2010; Vieira & Villemor-Amaral, 2015) and focuses on assigning the Wartegg Test the same property (Pacico, Hutz, Schneider, & Bandeira, 2015; Urbina, 2007), This presents even greater values for the Cohen d indicating greater impact on the variable. Alves, Quaglia, Bachett and Oliveira (2014) in a study investigating the depth perception in subjects with schizophrenia noted marked perceptual distortion, especially in subjects medicated over a period of less than four months, indicating distortion of the form observed. In another study, Amaro, Areco and Nascimento (2017) also found significant differences in a sample of 50 elderly people, corroborating the sensitivity of the variable for the assessment of perceptual elements.

There was also a significant difference for movement codes (M and FM), which corroborates the findings of some authors (Exner & Sendin, 1999; Pianowski & Villemor-Amaral, 2010) in studies that verified the relevance of these variables in the study of schizophrenia, especially for human movement, predominantly associated with QF-, indicating rupture with reality and possible psychotic thinking, which may be related to this variable having been observed as statistically different in the t-test.

Mihura et al. (2013) in a study on the indicators of the variables in the Rorschach test (R-PAS) it is emphasized that M is associated with mental abilities, including planning and empathy, which would justify the higher values of the normative group, since patients with schizophrenia may present deficits in social skills (Murta, 2005).

These analyses allow us to verify that the two techniques share aspects in which they are able to verify, through their variables, that is, to infer types of psychological functioning from the criteria used in the responses of the subject, as proposed by Urbina (2007). Even sharing these possibilities, it is important to understand that part of the variance of latent abilities are not measured by one technique or another, due to the different formats of the items, mode of data collection and, consequently, access to the latent trace (Schmitt et al., 2015). In this sense, as proposed by Captain and Cardoso (2012) the use of different techniques may favor the completion of information.

In view of this consideration, the observed results of linear regression indicate the importance of using techniques together in the use of psychological evaluation as indicated in the incremental validity (Capitão & Cardoso, 2012; Hunsley & Meyer, 2003). The levels of prediction evidenced by both techniques have proven effective in predicting schizophrenia. In the case of formal quality, together they predict correctly, 75% of the cases, this indicates that, designing first what the Rorschach test can predict, Wartegg is added for the rest of the data. Likewise, for movement and content, the hit percentage is high, reaching 100% in the second case.

Thus, the incremental validity evidenced from the two instruments may prove useful for the clinical evaluation context, as some researchers have also observed for other instruments (Brackett & Mayer, 2003; Capitão & Cardoso, 2012; Franco & Villemor-Amaral, 2012). In this sense, it is possible to observe through the analyses that the instruments, although they show convergence as to the latent traits evaluated, their joint use can help in the intervening process adding great preventive power to the evaluation process. Therefore, it is possible to conclude that the use of the Wartegg Test can increase evidence of validity to the Rorschach test by enriching clinical decisions in psychological evaluation processes, signaling important indicators about the perceptual distortion of the assessed subjects.

It is understood that the present study is an initial for the Pessotto system (2018a) for the Wartegg Test, conceiving the need to improve it, as well as the exploration of other variables. Another issue to be considered as a limitation may be the reduced number of subjects. Even this fact being observed in other studies with self-expression techniques (Scortegagna & Villemor-Amaral, 2013; Yazigi et al., 2013), it is important to enlarge the sample, as well as the evaluation in other contexts.

Referências

• Alves, A., Quaglia, M. A. C., Bachett, L. S., & Oliveira, M. S. (2014). Percepção monocular da profundidade ou relevo na ilusão da máscara côncava na esquizofrenia [Monocular depth perception or relief in the hollow-face illusion in the schizophrenia]. Estudos de Psicologia (Natal), 19(1), 40-47. doi:10.1590/S1413-294X2014000100006
  » https://doi.org/10.1590/S1413-294X2014000100006
• Amaro, T. A. C., Areco, K. N., & Nascimento, R. S. G. (2017). Avaliação dos aspectos da personalidade em pessoas idosas na cidade de São Paulo por meio do Rorschach Performance System (R-PAS) [Evaluation of personality aspects in elderly people in Sao Paulo city using Rorschach Performance System (R-PAS)]. Revista Kairós Gerontologia, 20(2), 221-228. doi:10.23925/2176-901X.2017v20i2p211-228
  » https://doi.org/10.23925/2176-901X.2017v20i2p211-228
• Andrei, F., Siegling, A. B., Aloe, A. M., Baldaro, B., & Petrides, K. V. (2016). The incremental validity of the Trait Emotional Intelligence Questionnaire (TEIQue): A systematic review and meta-analysis. Journal of Personality Assessment, 98(3), 261-276. doi:10.1080/00223891.2015.1084630
  » https://doi.org/10.1080/00223891.2015.1084630
• Brackett, M. A., & Mayer, J. D. (2003). Convergent, discriminant, and incremental validity of competing measures of emotional intelligence. Personality and Social Psychology Bulletin, 29(9), 1147-1158. doi:10.1177/0146167203254596
  » https://doi.org/10.1177/0146167203254596
• Campos, R. C. (2013). Além dos números há uma pessoa: Sobre a utilização clínica de testes [Behind the numbers there is a person: About the clinical use of tests]. Avaliação Psicológica, 12(3), 291-298. Retrieved from http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712013000300003
  » http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712013000300003
• Capitão, C. G., & Cardoso, L. M. (2012). Evidence of incremental validity between pfister test and human figure drawing. International Journal of Psychology and Behavioral Sciences, 2(4), 120-129. doi:10.5923/j.ijpbs.20120204.07
  » https://doi.org/10.5923/j.ijpbs.20120204.07
• Crisi, A., & Dentale, F. (2016). The Wartegg Drawing Completion Test: Inter-rater agreement and criterion validity of three new scoring categories. International Journal of Psychology and Psychological Therapy, 16(1), 83-90.
• Cunha, J. A. (2000). Psicodiagnóstico-V [Psychodiagnosis-V]. Porto Alegre, RS: Artmed.
• Engelman, D. H., Allyn, J. B., Crisi, A., Finn, S. E., Fischer, C. T., & Nakamura, N. (2016). “Why am I so stuck?”: A collaborative/therapeutic assessment case discussion. Journal of Personality Assessment, 98(4), 360-373. doi:10.1080/00223891.2015.1119154
  » https://doi.org/10.1080/00223891.2015.1119154
• Exner, J. E. Jr., & Sendín, C. (1999). Manual de interpretação do Rorschach: Para o sistema compreensivo [Rorschach interpretation manual: For the comprehensive system] (L. Y. Massuh, Trans.). São Paulo, SP: Casa do Psicólogo.
• Franco, R. R. C., & Villemor-Amaral, A. E. (2012). Validade incremental do Zulliger e do Pfister no contexto da toxicomania [Incremental validity of the Zulliger and Pfister in the contexto of drug addiction]. Psico-USF, 17(1), 73-83. doi:10.1590/S1413-82712012000100009
  » https://doi.org/10.1590/S1413-82712012000100009
• Haynes, S. N., & Lench, H. C. (2003). Incremental validity of new clinical assessment measures. Psychological Assessment, 15(4), 456-466. doi:10.1037/1040-3590.15.4.456
  » https://doi.org/10.1037/1040-3590.15.4.456
• Hunsley, J., & Meyer, G. J. (2003). The incremental validity of psychological testing and assessment: Conceptual, methodological, and statistical issues. Psychological Assessment, 15(4), 446-455. doi:10.1037/1040-3590.15.4.446
  » https://doi.org/10.1037/1040-3590.15.4.446
• Marques, T. C., Chaves, A. C., & Yazigi, L. (2012). Estudo parcial da validação do Atlas do Rorschach Sistema Compreensivo em amostra de pacientes psiquiátricos de São Paulo. [Partial validation study of the Rorschach Comprehensive System atlas on a sample of psychiatric patients of São Paulo]. Psico-USF, 17(3), 417-416. doi:10.1590/S1413-82712012000300008
  » https://doi.org/10.1590/S1413-82712012000300008
• Merino Soto, C. A. (2014). Validez incremental del Test Gestáltico de Bender Modicado, en niños que inician el primer grado [Incremental validity for the Modified Bender Gestalt Test in children commencing first grade]. Avances en Psicología Latinoamericana, 32(2), 275-286. doi:10.12804/apl32.2.2014.07
  » https://doi.org/10.12804/apl32.2.2014.07
• Meyer, G. J., Viglione, D. J., Mihura, J. L., Erard, R. E., & Erdberg, P. (2011). Rorschach Performance Assessment System. Toledo, OH: Rorschach Performance Assessment System/LLC.
• Mihura, J. L., & Meyer, G. J. (2018). Using the Rorschach Performance Assessment System (R-PAS). New York, NY: Guilford.
• Mihura, J. L., Meyer, G. J., Dumitrascu, N., & Bombel, G. (2013). The validity of individual Rorschach variables: Systematic reviews and meta-analyses of the comprehensive system. Psychological Bulletin, 139(3), 548-605. doi:10.1037/a0029406
  » https://doi.org/10.1037/a0029406
• Moore, R. C., Viglione, D. J., Rosenfarb, I. S., Patterson, T. L., & Mausbach, B. T. (2013). Rorschach measures of cognition relate to everyday and social functioning in schizophrenia. Psychological Assessment, 25(1), 253-263. doi:10.1037/a0030546
  » https://doi.org/10.1037/a0030546
• Murta, S. G. (2005). Aplicações do treinamento em habilidades sociais: Análise da produção nacional [Applications of training in social skills: Analysis of national production]. Psicologia: Reflexão e Crítica, 18(2), 283-291. doi:10.1590/S0102-79722005000200017
  » https://doi.org/10.1590/S0102-79722005000200017
• Noronha, A. P. P., & Alchieri, J. C. (2004). Knowledge in psychological assessment. Estudos de Psicologia (Campinas), 21(1), 43-52. doi:10.1590/S0103-166X2004000100004
  » https://doi.org/10.1590/S0103-166X2004000100004
• Ng, T. W. H. (2015). The incremental validity of organizational commitment, organizational trust, and organizational identification. Journal of Vocational Behavior, 88, 154-163. doi:10.1016/j.jvb.2015.03.003
  » https://doi.org/10.1016/j.jvb.2015.03.003
• Ng, T. W. H., & Feldman, D. C. (2015). Ethical leadership: Meta-analytic evidence of criterion-related and incremental validity. Journal of Applied Psychology, 100(3), 948-965. doi:10.1037/a0038246
  » https://doi.org/10.1037/a0038246
• Pacico, J. C., Hutz, C. S., Schneider, A. M. A., & Bandeira, D. R. (2015). Validade [Validity]. In C. S. Hutz, D. R. Bandeira, & C. M. Trentini (Orgs.), Psicometria [Psychometry] (pp. 71-84). Porto Alegre, RS: Artmed .
• Pessotto, F., & Primi, R. (2018a). A study of Coding Systems for the Wartegg Test and their relations with the Rorschach (R- PAS). Trends in Psychology, 26(4), 1847-1860. doi:10.9788/TP2018.4-06pt
  » https://doi.org/10.9788/TP2018.4-06pt
• Pessotto, F., & Primi, R. (2018b). Evidências de validade de critério para o teste de Wartegg [Evidence of criterion validity for the Wartegg Test]. Avaliação Psicológica, 17(3), 279-291. doi:10.15689/ap.2018.1703.13941.01
  » https://doi.org/10.15689/ap.2018.1703.13941.01
• Pessotto, F., & Primi, R. (2018c). Evidências de validade convergente para o Teste de Wartegg [Convergent validity evidences for Wartegg Test]. Psico, 49(1), 73-80. doi:10.15448/1980-8623.2018.1.26935
  » https://doi.org/10.15448/1980-8623.2018.1.26935
• Pianowski, G., Meyer, G. J., & Villemor-Amaral, A. E. (2016). The impact of R-optimized administration modeling procedures on Brazilian normative reference values for Rorschach Scores. Journal of Personality Assessment, 98(4), 408-418. doi:10.1080/00223891.2016.1148701
  » https://doi.org/10.1080/00223891.2016.1148701
• Pianowski, G., Meyer, G. J., Villemor-Amaral, A. E., Zuanazzi, A. C., & Nascimento, R. S. G. F. (2019). Does the Rorschach Performance Assessment System (R-PAS) differ from the Comprehensive System (CS) on variables relevant to interpretation? Journal of Personality Assessment, 1-16. Advance online publication. doi:10.1080/00223891.2019.1677678
  » https://doi.org/10.1080/00223891.2019.1677678
• Pianowski, G., & Villemor-Amaral, A. E. (2010). Localização e qualidade formal do Rorschach-SC no Brasil: Validade com não-pacientes [Location and formal quality of the Rorschach-SC in Brazil: Validity with non-patient sample]. Psico-USF, 15(3), 333-343. doi:10.1590/S1413-82712010000300007
  » https://doi.org/10.1590/S1413-82712010000300007
• Rizzo, A., Della Villa, L., & Crisi, A. (2015). Can the problematic Internet use evolve in a pre-psychotic state? A single case study with the Wartegg. Computers in Human Behavior, 51(A), 532-538. doi:10.1016/j.chb.2015.04.063
  » https://doi.org/10.1016/j.chb.2015.04.063
• Schmitt, M., Hofmann, W., Gschwendner, T., Gerstenberg, F., & Zinkernagel, A. (2015). A model of moderated convergence between direct, indirect, and behavioral measures of personality traits. In T. M. Ortner & F. J. R. van de Vijver (Eds.), Behavior-based assessment in psychology: Going beyond self-report in the personality, affective, motivation, and social domains: Vol. 1. Psychological assessment - Science and practice (pp. 29-44). Göttingen, Germany: Hogrefe.
• Scortegagna, S. A., & Villemor-Amaral, A. E. (2013). Rorschach e pedofilia: A fidedignidade no Teste-Reteste [Rorschach and pedophilia: A reliability at Test-Retest]. Psico, 44(4), 508-517. Retrieved from https://revistaseletronicas.pucrs.br/ojs/index.php/revistapsico/article/view/11651
  » https://revistaseletronicas.pucrs.br/ojs/index.php/revistapsico/article/view/11651
• Silveira, L. B., Oliveira, S. E. S., & Bandeira, D. R. (2018). Validity evidences of the Inventory of Personality Organization - Brasil (IPO-Br): Its relation with the Five-Factor Model of Personality. Trends in Psychology , 26(4), 1875-1889. doi:10.9788/TP2018.4-07pt
  » https://doi.org/10.9788/TP2018.4-07pt
• Urbina, S. (2007). Fundamentos da testagem psicológica [Fundamentals of psychological testing] (C. Dornelles, Trans.). Porto Alegre, RS: Artmed .
• Vari, C., Velotti, P., Crisi, A., Carlesimo, S., Richetta, A. G., & Zavattini, G. C. (2017). Investigating personality and psychopathology in patients with psoriasis. Rorschachiana, 38(2), 87-107. doi:10.1027/1192-5604/a000092
  » https://doi.org/10.1027/1192-5604/a000092
• Vieira, P. G. (2017). Avaliação terapêutica: Uma nova abordagem para o psicodiagnóstico [Therapeutic assessment: A new approach to psychodiagnosis]. In P. Landin, R. Almeida, & A. Roma (Eds.), Avaliação psicológica: A atualidade da prática profissional (Vol. 1, pp. 21-31). São Paulo, SP: Leader.
• Vieira, P. G., & Villemor-Amaral, A. E. (2015). Evidências de validade do Rorschach Performance Assessment System no diagnóstico da esquizofrenia [Evidence of validity of Rorschach Performance Assessment System in the diagnosis of schizophrenia]. Avaliação Psicológica, 14(1), 53-62. Retrieved from http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712015000100007
  » http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712015000100007
• Villemor-Amaral, A. E., & Pasqualini-Casado, L. (2006). A cientificidade das técnicas projetivas em debate [The scientific status of projective techniques in discussion]. Psico-USF, 11(2), 185-193. doi:10.1590/S1413-82712006000200007
  » https://doi.org/10.1590/S1413-82712006000200007
• Yazigi, L., Areco, K. C. N., Fiore, M. L. M., Barros, R., Lerman, T. G., Abela, R. K., … Marques, T. C. (2013). Avaliação de processo psicoterápico por meio do Rorschach Performance Assessment System [Evaluation of the psychotherapeutic process through the Rorschach Performance Assessment System]. Fragmentos de Cultura, 23(4), 515-526. doi:10.18224/frag.v23i4.2983
  » https://doi.org/10.18224/frag.v23i4.2983
• Yazigi, L., Semer, N. L., Fiore, M. L. M., Abela, R. K., Lerman, T. G., & Marques, T. C. (2016). Form quality in Rorschach Comprehensive System and R-PAS: Sample of psychiatric cases. Paidéia (Ribeirão Preto), 26(63), 53-61. doi:10.1590/1982-43272663201607
  » https://doi.org/10.1590/1982-43272663201607
• Yuana, R. A., Harjunowibowo, D., Karyanta, N. A., & Budiyanto, C. W. (2018). Data similarity filtering of Wartegg Personality Test result using cosine-similarity. International Journal of Recent Contributions from Engineering, Science & IT (IJES), 6(3), 19-28. Retrieved from https://online-journals.org/index.php/i-jes/article/view/9413/5257
  » https://online-journals.org/index.php/i-jes/article/view/9413/5257

• 5
  Article derived from the PhD thesis of the first author under the supervision of the second, defended in 2015, in the Post graduate Program in Psychology with emphasis on Psychological Assessment from the Universidade São Francisco.

Edited by

Associate editor: Clarissa Mendonça Corradi-Webster

Publication Dates

• Publication in this collection
  08 Jan 2021
• Date of issue
  2021

History

• Received
  08 Nov 2019
• Reviewed
  09 July 2020
• Reviewed
  07 Sept 2020
• Accepted
  10 Sept 2020