Integrated assessment of children’s cognitive and creative abilities: Psychometric studies

Avaliação integrada das habilidades cognitivas e criativas de crianças: estudos psicométricos

Evaluación integrada de las habilidades cognitivas y creativas de los niños

Abstract

It is essential that intelligence assessment be integrated with creativity, although no instruments in Brazil do so. This research investigated the item difficulty and validity and reliability of the Bateria de Avaliação Intelectual e Criativa Infantil (BAICI) to address this gap. The first sample consisted of 612 children (54% M) aged 7 to 12 years, and the second sample consisted of 377 students (56% M), some of whom (N = 164) were already identified as exhibiting high skills/giftedness. Item analysis indicated the need to adjust the BAICI items. The results of a MANCOVA indicated that the BAICI exhibits evidence of validity with external variables because the group of gifted children was significantly distinguished from the group of students attending regular schools on tests of vocabulary, speed, logical thinking, and creativity. The study concludes that the BAICI has psychometric qualities that can be used in the psychological assessment of children.

Keywords:
Psychological assessment; creativity; intelligence; high abilities, giftedness

Resumo

É essencial que a avaliação da inteligência seja integrada com a criatividade embora não existam instrumentos no país para essa finalidade. Esta pesquisa investigou a dificuldade dos itens e as evidências de validade e precisão da Bateria de Avaliação Intelectual e Criativa Infantil (BAICI) para oferecer uma avaliação mais completa do potencial infantil. A primeira amostra foi composta por 612 crianças (54% M), idade sete a 12 anos e a segunda de 377 estudantes (56% M), uma parte (N = 164) já identificadas com altas habilidades/superdotação. A análise pela TRI indicou a necessidade de ajuste de itens da BAICI. Os resultados pela MANCOVA indicaram que a BAICI possui evidências de validade com variáveis externas, pois o grupo de crianças superdotadas se distinguiu significativamente de estudantes de escolas regulares nos testes de vocabulário, rapidez, pensamento lógico e criatividade. Conclui-se que a BAICI possui qualidades psicométricas para ser utilizada na avaliação psicológica infantil.

Palavras-chave:
avaliação psicológica; inteligência; criatividade; altas habilidades

Resumen

Es fundamental que la evaluación de la inteligencia se integre con la creatividad, aunque en el país no existen instrumentos para este fin. Esta investigación analizó las evidencias de validez y precisión de la Batería de Evaluación Intelectual y Creativa Infantil (BAICI) para proporcionar una evaluación más completa del potencial de los niños. La primera muestra estuvo compuesta por 612 niños (54% M), de 7 a 12 años, a su vez, la segunda por 377 estudiantes (56% M), una parte (N = 164) ya identificada con altas habilidades/superdotación. El análisis de la TRI indicó la necesidad de ajustar los ítems de la BAICI. Los resultados de MANCOVA indicaron que BAICI tiene evidencias de validez con variables externas, ya que el grupo de niños superdotados se distinguió significativamente de niños de escuelas regulares en las pruebas de vocabulario, velocidad, pensamiento lógico y creatividad. Se concluye que la BAICI tiene cualidades psicométricas para ser utilizada en la evaluación psicológica infantil.

Palabras clave:
evaluación psicológica; inteligencia; creatividad; altas habilidades; superdotación

Intelligence assessment is a well-known practice in the area of psychological evaluation, as can be observed from the number of international instruments developed to measure this construct (Ackerman, 2013; Benson et al., 2019; Sattler, 2018). Additionally, in Brazil, intelligence tests are predominant among instruments that have already been approved by the Federal Council of Psychology (2018) and listed in the Psychological Test Assessment System-SATEPSI (2021), thus demonstrating the importance of this construct in psychological evaluation.

Intelligence and creativity are distinct constructs, although they may also be interrelated (Karwowski et al., 2016; Kim et al., 2010). However, creativity has not been included in intelligence test batteries, either nationally or even internationally, thus limiting a more complete assessment of an individual’s cognitive potential (Kaufman, 2015; Pfeiffer, 2018). Therefore, considering that intelligence and creativity are considered the key skills for the 21st century (Organization for Economic Cooperation and Development [OECD], 2020), it is necessary to find ways to integrate both dimensions to gain a broader understanding of the cognitive potential of children.

Intelligence and Creativity: Relationship between constructs

Intelligence has been studied under several theoretical approaches, thus demonstrating the importance of this concept (Flanagan & McDonough, 2018). Among the most internationally recognized models that stands out is that of Cattell (1971), which was derived from a large number of factorial studies and defines intelligence as being composed of three strata in a hierarchical structure. His proposal was revised by McGrew (2009), who developed an integrative model known as Cattell-Horn-Carroll (CHC), with the first stratum composed of general intelligence (G), and the second stratum explained by 10 types of broad skills: fluid intelligence (Gf), crystallized intelligence (GC), short-term memory (Gsm),visual processing (Gv), auditory processing (Ga), long-term storage and recovery (Glr), processing speed (Gs), decision and reaction time (Gt), reading and writing (Grw) and quantitative knowledge (GQ). The third stratum features over 70 specific abilities. This model was recently expanded by Schneider and McGrew (2018) to include 16 broad skills, including sensory and psychomotor skills, that are still difficult to measure.

Creativity has been studied from different perspectives, indicating that it is a complex phenomenon that involves cognitive processes, personality characteristics, educational and social components, or their interaction (Kaufman, 2016; Pfeiffer, 2018). Creativity as a cognitive process is evaluated through the ability to generate ideas, which needs to follow some criteria; that is, the idea needs to be original but also relevant and useful (Runco & Pritzker, 2020). In turn, the characteristics of the creative person have been emphasized in the literature (Kaufman & Sternberg, 2019), as well as the environmental, educational, and cultural elements for stimulating creativity (Simonton, 2018), thus indicating the need for an integrated vision for understanding this construct.

The relationship between creativity and intelligence has been discussed for decades. In the first theories about intelligence, such as in the structure of the intellect by Guilford (1968), creativity is conceived as a component of intelligence, being represented as divergent thinking as opposed to convergent thinking. During studies on the subject (Runco & Pritzker 2020), it is observed that there is consensus that creativity is related to intelligence, but the question remains as to what extent these constructs overlap, the so-called threshold hypothesis (Kaufman et al., 2012; Kim et al., 2010). Some authors (Kim, 2005) perceive these constructs as independent because the correlations between creativity and IQ tests are very weak, without significant gender influence. In contrast, other authors (Silvia, 2015) argue that creativity and intelligence are intricately linked by comparing data from cognitive processes with neuroscience discoveries. Therefore, the nature of the relationship between intelligence and creativity has not yet been elucidated, as concluded by Plucker et al. (2015).

According to CHC theory, creativity is measured by Glr, that is, long-term memory and recovery (Flanagan et al., 2007; Schneider & McGrew, 2018). However, the association of creativity with memory is not clear because problem solving requires finding new relationships, which is usually more strongly related to fluid intelligence skills (Gf) than to memory (Silvia et al., 2013). It is also important to note that only the fluency of ideas is being evaluated in the main test batteries; thus, the question can be raised whether regular IQ tests also assess creativity, in addition to intelligence (Kaufman et al., 2012; Kaufman, 2012). Therefore, understanding intelligence is necessary but not sufficient to explain the different forms of creative expression (Jauk et al., 2013; Karwowski et al., 2016)

The theoretical basis of the CHC model influenced the revision of several existing test batteries, such as the Woodcock-Johnson Battery [WJ-III] (Woodcock et al., 2001). Due to the importance of the WJ-III battery, several Brazilian studies (Wechsler, 2007) were conducted to verify the evidence of its validity for the Brazilian population, and Brazilian items were developed for the verbal area, and a new version was developed for collective administration. Although evidence of the validity of the WJ-III was confirmed in Brazilian samples, the need for an adapted version of the battery for our population was also verified (Primi et al., 2012; Wechsler et al., 2010). This led to the proposal of a new battery that could integrate intelligence with creativity and could also be applied collectively, which was not the case with the WJ-III battery.

There are several debates about the methods for understanding and evaluating creativity, covering both subjective or qualitative measures, such as peer, teacher, or parent assessments, as well as quantitative ones, involving standardized tests and scales (Nakano, 2018). Among the most widely used tests internationally for evaluating creativity are Torrance’s figural and verbal creativity tests (Torrance, 1966), which assess divergent thinking in the dimensions of fluency, flexibility, originality, and elaboration (Kaufman, 2016). Evidence of the validity and reliability of these tests as indicators of creativity has been confirmed in Brazilian studies (Wechsler, 2004a, 2004b), thus indicating that it is possible to evaluate creativity objectively and scientifically.

The need to create a battery of tests for assessing intelligence combined with creativity is made quite clear when focusing on the need to understand the different areas in which a child may have greater or lesser difficulty, thus enabling better diagnoses than a total IQ result (Kaufman, 2015, 2016). This is highlighted, for example, in the area of special education and, more specifically, in the identification of students with high skills or giftedness, which requires students to have complete diagnoses in order to access specific educational programs (Alencar & Fleith, 2005; Renzulli & Renzulli, 2010). According to Brazilian legislation, students with high skills/giftedness are defined as those who exhibit high performance in the intellectual, creative, leadership, artistic or psychomotor areas, and these abilities may occur in isolation or in combination (Secretariat of Special Education, 1995). Federal law states that students identified as having high skills/giftedness care must be accommodated in special educational programs (Secretariat of Special Education, 2002).

The identification of students with high skills/giftedness has been performed mainly through intelligence tests and teachers’ observations (Almeida et al., 2019; Martins et al., 2016). This is the result of the scarcity of measures that allow a more integrated diagnosis of other skills, thus resulting in limitations in the detection of students with different types of potential (Virgolim & Konkiewitz, 2018). Therefore, these professionals use more subjective measures to make this identification instead of scientifically proven tests (Wechsler & Fleith, 2017), or they rely only on intelligence tests, disregarding the importance of creativity in cognitive functioning, the recognition of which would allow a broader view of the student’s potential (Nakano et al., 2016).

Due to the need to identify the cognitive and creative abilities of children with or without high abilities, through an integrated approach with tests that present evidence of validity and reliability, this study investigated the possibility of using the Battery of Intellectual and Creative Evaluation - child form (BAICI) to meet this objective. For this purpose, two studies were delineated. The first aimed to analyze the difficulty and reliability of the BAICI items for children from different regions of the country. The second aimed to verify the hypothesis that the BAICI could discriminate children already identified as highly skilled/gifted by their teachers from those not identified or not yet assessed, thus demonstrating evidence of validity.

Method

Study 1

Participants

The sample consisted of 612 children (54% F) aged seven to 12 years (Mean= 10.43; SD= 1.85). These children were studying in public schools (80%) located in cities in different Brazilian regions: South (Curitiba= 101), Southeast (Campinas= 206), Central (Brasília= 98, Rondonópolis= 55), and Northeast (João Pessoa= 51, Natal=62, São Luís do Maranhão= 39). A convenience sample was collected by researchers in these different regions .

Collaborators: Psychology students participated in the application and correction of the tests, supervised by the researchers from each region.

Instrument

Battery Of Intellectual and Creative Evaluation - Children’s Version-BAICI.

This battery consists of six subtests that measure the following areas: crystallized intelligence-Gc by verbal skill tests (vocabulary, 21 figures to be named), logical thinking-Gf (16 items representing a sequence of abstract figures), visual-spatial thinking -Gv (12 items in the form of puzzles), visual -auditory memory-Glr (11 items in the form of symbols that should be remembered after presentation), speed of thinking -Gs (48 pairs of letters to be identified within a certain time frame) and creative thinking (1 activity composed of symbols to be drawn and 2 verbal activities, with one composed of titles for a story and another of imagination of life within the oceans). These intellectual abilities were based on the WJ-III battery (Woodcock et al., 2001), except for creative thinking, which is not evaluated in this battery.

The BAICI was constructed from an analysis of the difficulty of the items on the adult version (Wechsler et al., 2010), called “Bateria de Avaliacao Intelectual de Adultos- BAIAD” (Battery for Assessing Adults’ Intelligence). The evidence of the reliability and validity of he BAIAD’s internal structure was investigated in two studies through confirmatory factor analysis (Wechsler et al., 2014b) and network analysis (Wechsler et al., 2019). The BAIAD was found to evaluate the following factors: Gf (fluid intelligence, composed of analogies, logical thinking and visual-spatial subtests) and Gc (crystallized intelligence, composed of vocabulary, synonyms and antonyms, both related to the G factor (general intelligence). The BAIAD’s vocabulary items were constructed from an analysis of textbooks and comparative studies with the standard battery of Woodcock-Johnson III -WJ III (Wechsler et al., 2007), which demonstrated that it needed several adjustments to be used in our country (Wechsler et al., 2010). Gender differences were also found in the studies (Wechsler et al., 2014 a). The BAIAD’s easiest items in vocabulary, logical thinking, visual-spatial thinking and memory tests were separated for the children’s version (BAICI), and items assessing speed and creativity were added. Therefore, a Brazilian battery was elaborated for collective administration. The evaluation of creativity was added to the BAICI to enable an integrative view of intelligence with creativity, as recommended in other studies on this topic (Milan & Wechsler, 2018).

Procedure

The project was submitted and approved by the Ethics Committee n.408.707. Subsequently, contacts were made with research professors from various institutions in the country requesting their collaboration to administer the BAICI to children between the ages of 7 and 12 years in public and private schools with the help of psychology students. The administration was collective and was separated into 2 sessions of 1.5 hours each to avoid fatiguing the children. The instructions were read by the examiners in each room, especially for those children with reading difficulties. For underage children (7-8 years) who could not write, especially those from some public schools, the instrument administration occurred individually. After the administration and correction of the subtests by each regional team, the results were sent to the central coordination group for the project, which elaborated the individual graphs. The graphs included the total results for each child, and were returned to the regional teams so that feedback could be given to parents and teachers to indicate each child’s stronger and weaker areas.

Data analysis

The analysis of the item difficulty analysis was performed for 4 subtests of the BAICI (vocabulary, logical thinking, visual-spatial thinking, and auditory visual memory). The speed of the thinking test does not allow this analysis because its objective is to evaluate speed, so an individual cannot respond to all the items within the allotted time (Boone et al., 2014). Additionally, the creative thinking test does not allow this analysis because it is composed of open-ended answers.

The infit and outfit measurements used had values ranging from .5 to 1.5. The item reduction process was performed through the following steps: a) infit-outfit values within the defined range; b) item-total correlation (biserial point), above .30. The final selection of the items was also analyzed according to the change in theta, according to the Rasch model, to evaluate whether there would be much discrepancy in the skill level (e.g., >5.00). Accuracy was evaluated using the Rasch and Cronbach’s alpha models. The WINSTEP program (Linacre, 2020) was used for these analyses.

Results

The analysis of difficulty, adjustment of items and reliability are presented in Table 1 (vocabulary and logical thinking subtests) and Table 2 (visual-spatial thinking and memory), which are analyzed according to the parameters described above.

Table 1
Difficulty index, item adjustment and accuracy of the vocabulary and logical thinking subtests

Subtest	Item	difficulty	Infit	Outfit	Correlation Item-theta
Vocabulary	21	2.82	.86	.58	. 43
	7	2.38	1.04	1.20	.35
	12	2.24	1.10	1.22	.33
	15	2.14	1.04	.95	.39
	18	1.73	.86	.69	.52
	11	1.46	.96	.86	.48
	14	.95	.88	.81	. 55
	06	.28	.91	.96	.54
	19	-.03	1.02	1.00	.51
	13	-.08	1.07	1.11	.48
	22	-.19	1.01	1.01	.52
	1	-.26	1.46	1.71	.27
	10	-.61	.99	0.93	.53
	20	-.72	.94	1.01	.55
	9	-.88	.91	0.83	.57
	5	-1.10	.87	0.76	.59
	8	-1.27	.93	0.78	.56
	4	-1.44	.90	0.85	.56
	17	-1.62	.99	1.10	.51
	2	-1.76	1.20	1.45	.44
	3	-1.91	.92	0.79	.55
	16	-2.15	1.12	1.33	.43
Logical Reasoning	15	6.34	1.07	.56	.10
	16	5.69	1.15	.96	.10
	14	1.08	1.20	1.39	.46
	13	.63	1.18	1.22	.49.
	12	.37	.95	.94	.59
	10	.02	1.04	.95	.57
	11	-.21	.84	.78	.64
	7	-.68	1.39	1.40	.44
	8	-.92	.79	.65	.66
	9	-1.07	.87	.80	.63
	5	-1.25	.91	86	.61
	4	-1.58	1.04	1.00	.56
	6	-1.72	.88	.78	.61
	3	-2.05	1.01	.89	.57
	1	-2.12	.87	.78	.60
	2	-2.52	.90	.61	.59

Table 2
Difficulty index. item adjustment and accuracy of the vocabulary and logical thinking subtests

Subtest	Item	difficulty	Infit	Outfit	Correlation Item-theta
Visual Spatial Thinking	8	2.12	.85	.82	.32
	12	1.66	.87	.88	.36
	9	1.31	1.02	1.15	.32
	10	.56	.90	.83	.47
	11	.46	1.06	1.12	.39
	7	.20	1.09	.99	.41
	5	- .36	1.00	.97	.50
	4	- .43	.95	.91	.53
	6	- .57	1.08	1.18	.46
	1	-1.20	1.19	1.21	.46
	3	-1.59	.89	.87	.63
	2	-2.17	.94	.85	.63
Auditory Visual Memory	9	.40	1.08	2.31	.55
	7	3.32	.89	2.30	.39
	6	.17	.92	1.26	.61
	11	3.91	1.14	1.07	.32
	5	-.69	.92	1.09	.65
	1	-4.43	1.08	.92	.59
	2	-4.02	.93	1.05	.62
	4	-1.78	.97	.91	.66
	10	3.65	.96	.96	.36
	8	1.97	.87	.50	.53
	3	-2.50	.86	.82	.68

As seen in Table 1, in the verbal ability subtest, item 1 of the vocabulary presents misfit problems (outfit = 1.71, item-theta correlation=.27). This item presents confusion possibly because the misfit may be due to the unclear intent of the item; that is, whether the child should name a part of the elephant with the arrow (in this case, the trunk) or whether it should name the entire figure. For the logical thinking subtest, items 15 and 16 present very high theta values of difficulty (5.34 and 5.69, respectively), probably because they require large changes in relation to the previous items, as they request 2 or more conditions to find a logical answer to the item (which criterion for the drawing outside the figure is required to enter in the frame presented, by the shape, position, quantity and/or color). The accuracy of the two subtests (verbal and logical thinking ability) reached .99 by the Rasch model, while in Cronbach’s model, it reached .82 for verbal ability and .94 for logical thinking, thus indicating that it meets the quality parameter in both models.

The analysis of the difficulty of the items and their accuracy for the tests of visual-spatial thinking and auditory visual memory is presented in Table 2. According to the results, the visual-spatial thinking test did not present misfits, and the difficulty level of its items was within the defined parameters. In turn, the auditory visual memory test presented outfit in items 7 and 9 (values ≥ 2.30). These items that have shapes shaded or filled with vertical lines bring greater difficulties in memorization and indicate that they should be modified. It can also be observed that the last 2 items of this subtest (10 and 11) are more difficult (theta= 3.65 and 3.91) because they have a greater number of stimuli to be remembered, while the initial items (1 and 2) are very easy (theta =-4.43 and -4.02), as expected. The accuracy of these subtests by the Rasch model reached 0.98 for the two subtests and a Cronbach’s alpha of .55 for the visual-spatial test and 0.73 for the memory test.

The results indicated the need to adjust items in the following subtests: 1 in vocabulary, 2 in logical reasoning and 2 in visual-auditory memory. The results also indicated that the elephant item needed to be redesigned due to confusing vocabulary. The results for logical reasoning demonstrated that 2 items should be repositioned due to the high level of difficulty, and the results for visual memory indicated the need to redesign 2 items. Only the results for visual-spatial test did not indicate any need for adjustments because the items were well understood. The accuracy of each of the subtests, calculated by the Rasch model, was shown to be quite high (≥ .98), thus indicating that it was adequate for these measurements. The accuracy estimated by Cronbach’s alpha index reached 0.9. indicating that the BAICI has excellent accuracy.

Study 2

Participants

The sample consisted of 377 children (49% F), with ages ranging from 7 to 12 years (M= 9.80; SD= 1.41), and all students from public schools in 6 cities located in 4 Brazilian states (Maranhão, Paraná, Distrito Federal, Mato Grosso do Sul, Rio Grande do Norte, São Paulo). In addition to attending regular school, 164 of these children (31% F) attended special programs for children with high skills/giftedness in the cities of Brasília (96), São Luís do Maranhão (38) and Londrina (30). These children were indicated as being gifted or having high skills by their teachers.

Instruments

1. Battery of Intellectual and Creative Skills (BAICI). This battery, previously described in study 1, was used with a reduced format, which involved removing the items that caused misfit in vocabulary, logical thinking, and memory.

2. The Scale for Assessment of Behavioral Characteristics of Students with Higher Skills - SCRBSS, was originally published by Renzulli et al. (1976). This scale is intended to help the teacher in the classroom evaluate the behavioral characteristics of his or her students in 10 areas: creativity, leadership, motivation, learning, performing and visual arts, music, planning and communication (expression and accuracy). This scale is the instrument most used by the High Skills/Gifted Centers (NAAHS) and in the resource rooms by teachers to identify children with high skills/giftedness for placement into specialized educational care. This scale was translated into Portuguese, and there are no data thus far on its validity.

Procedure

Researchers from different regions were contacted to verify whether they could participate in the project. In three cities (Brasília, São Luiz do Maranhão, Londrina) in which NAAHS or resource rooms existed, the researchers requested permission from the coordinators of these programs and from the parents of the children for BAICI group administration to those students attending the program. In other cities, the BAICI was administered only in regular public schools due to the lack of these special programs, and permission was requested from the school principals and parents of students in the selected age group (seven - 12 years).

The criterion used for the entry of children into the special programs for high skills/giftedness through the NAAHS and resource rooms was the SCRBSS scale (Renzulli et al., 1976). The researchers did not have access to specific data on the performance of each child on this scale, so they used only the fact that the children were attending these programs as a criterion.

Performance on the different BAICI tests was evaluated to verify whether this battery exhibited evidence of validity for external variables, thus making it possible to significantly distinguish (p≤0.05) children who participated in special programs for high skills/gifted children, based on a nomination from their teachers, from children in regular schools and those without a teacher’s nomination. Therefore, it was hypothesized that there were differences between the groups but not between genders. The variables analyzed were group type, gender and age (covariate) using multivariate covariance (MANCOVA) and univariate variance (ANOVA) analyses. Pearson’s correlation was also performed between the total score on the intelligence tests and total score for creativity to verify whether these constructs would have weak significant relationships (p≤0.05) with each other.

Results

The analysis of the means for the group of children participating in programs for high skills/giftedness compared to the means for the group of children from regular schools is presented in Table 3.

Table 3
Marginal means in the BAICI for the high skill/gifted and regular groups

BAICI	Group type	M	SD	Lower limit	Upper limit
Logical thinking	Regular	9.085	.521	8.057	10.113
	High skills	11.061	.295	10.479	11.643
Visual-spatial	Regular	2.297	.218	1.866	2.728
	High skills	1.475	.124	1.231	1.719
Vocabulary	Regular	8.134	.691	6.769	9.499
	High skills	13.767	.392	12.994	14.540
Memory	Regular	2.883	.367	2.159	3.608
	High skills	3.435	.208	3.025	3.846
Speed	Regular	18.18	1.771	14.688	21.678
	High skills	33.545	1.003	31.566	35.525
Creativity	Regular	30.681	3.561	23.652	37.710
	High skills	46.429	2.016	42.449	50.410

As seen in Table 3, children with high skills/giftedness obtained higher averages in the tests of logical thinking, vocabulary, auditory visual memory, speed of thinking and creativity. Only in the visual-spatial thinking test did the regular group obtain a higher mean than the group with high abilities. For gender, according to Table 4, it was observed that for vocabulary, males had higher means than females, while in visual-spatial thinking, females obtained better results. To analyze whether these differences between the means were significant, MANCOVA and ANOVA tests were performed, controlling for gender and group differences and using age as a covariate, as shown in Table 5.

Table 4
Marginal means by gender for the BAICI subtests

BAICI	Gender	M	DP	Lower limit	Upper limit
Logical thinking	female	9.494	.447	8.613	10.376
	male	10.652	.390	9.882	11.421
Visual-Spatial	female	2.051	.187	1.681	2.421
	male	1.721	.163	1.398	2.043
Vocabulary	female	9.890	.593	8.720	11.061
	male	12.011	.517	10.989	13.032
Memory	female	2.902	.315	2.281	3.523
	male	3.416	.275	2.874	3.959
Speed	female	26.071	1.518	23.075	29.067
	male	25.657	1.325	23.042	28.273
Creativity	female	41.414	3.053	35.388	47.440
	male	35.696	2.664	30.437	40.955

Table 5
Multivariate analysis of covariance of the BAICI by gender and group type. covariable age

Origin	Dependent Variable	Type III Sum of Squares	Gl	Mean Square	F	h2
Age	Logical thinking	12.067	1	12.067	1.199	.008
	Visual-Spatial	.070	1	.070	.038	.000
	Vocabulary	10.866	1	10.866	.623	.004
	Memory	960.202	1	960.202	4.618*	.028
	Speed	82.940	1	82.940	.732	.005
	Creativity	1504.878	1	1504.878	3.211	.020
Sex	Logical thinking	14.682	1	14.682	1.458	.009
	Visual-Spatial	3.352	1	3.352	1.816	.011
	Vocabulary	74.013	1	74.013	4.244*	.026
	Memory	.823	1	.823	.004	.000
	Speed	32.640	1	32.640	.288	.002
	Creativity	136.032	1	136.032	.290	.002
Group type	Logical thinking	72.269	1	72.269	7.179**	.043
	Visual-Spatial	27.023	1	27.023	14.642**	.085
	Vocabulary	746.696	1	746.696	42.813**	.213
	Memory	15.700	1	15.700	.076	.000
	Speed	5377.237	1	5377.237	47.430**	.231
	Creativity	2226.783	1	2226.783	4.752*	.029
Gender * Type group	Logical thinking	9.705	1	9.705	.964	.006
	Visual-Spatial	8.630	1	8.630	4.676*	.029
	Vocabulary	.070	1	.070	.004	.000
	Memory	22.431	1	22.431	.108	.001
	Speed	274.618	1	274.618	2.422	.015
	Creativity	16.575	1	16.575	.035	.000

• *p≤.05; ** p≤.01

According to Table 5, the results of the BAICI were able to significantly differentiate the group with high abilities from the regular group on all subtests except for memory. The tests that had the greatest effect (h²= square eta) distinguishing the groups were vocabulary (F= 42, 81; p≤ .01, η² =.21) and speed of thinking (F= 47.43, p≤ .01, η² = .23). The logical thinking (F=7.17, p≤ .01) and creativity (F=4.75, p≤0.05) tests also distinguished the group of high abilities from the regular group, but with less effect (η² = .02-.04). The visual-spatial thinking test made a significant distinction but in reverse, that is, the group of children from regular schools obtained a better result than the group with high skills.

For the other variables studied, gender showed significant results (F= 4.24, p≤ .05, η² = 0.02) on the vocabulary test, with superiority for males. There was an interaction of gender and type of group for the variable visual-spatial thinking (F=4.676, p≤ .05, η² = .02), with superiority for the female gender of the regular group. The covariate age influenced the result of the memory test (F=4.618, p≤.05, η² =.02), with students of the highest ages having the best results. Pearson’s correlation between the total score obtained in the cognitive skills tests and creativity tests was low (r= .358) and was also not significant at the p≤.05 level.

Discussion

The need for intelligence assessment has been greatly emphasized in psychology, as seen by the number of psychological tests developed internationally to measure it (Benson et al., 2019; Flanagan & McDonough, 2018). It is the same in our country, where intelligence tests are predominant in the list of instruments approved by SATEPSI (2020). However, the evaluation of creativity has not been included in the test batteries to allow a more complete and integrated cognitive assessment, either abroad or within Brazil (Kaufman, 2015; Pfeiffer, 2018), thus greatly limiting the ability to evaluate an individual’s potential.

There are debates about the relationship between intelligence and creativity to this day, questioning whether these constructs are independent or overlap (Kim et al., 2010; Plucker et al., 2015). According to CHC (Cattell-Horn-Carroll) theory, which proposes a hierarchical model of intelligence (McGrew, 2009), creativity is associated with long-term memory (Glr); however, this association is questioned by other scholars because it may limit the understanding of creativity (Jauk, 2013; Nusbaum & Silva, 2011, Silva, 2008). Therefore, understanding intelligence is important, but it is not enough to explain the different forms of expression of human potential, indicating the need for instruments that can evaluate, in an integrated way, intellectual and creative skills.

The need to verify the psychometric quality of instruments to be used by Brazilian psychologists is regulated by the Federal Council of Psychology (2018). However, the results of previous Brazilian studies conducted with the Woodcock-Johnson III (WJ-III) battery, which adopts the CHC model to evaluate intelligence, indicated the need to create a new battery (Wechsler et al, 2007; Wechsler et al., 2010a), and two batteries were then developed: the BAIAD (adult form) and the BAICI (child form). In this second instrument, creativity tests were added that did not exist in the WJ-III, and administration of the instrument was modified from an individual format to a collective format to allow the identification of cognitive potential on a broader scale. Therefore, the objective of the research was to investigate whether the Battery of Intellectual and Creative Evaluation (BAICI) had the psychometric qualities of validity and reliability that would allow it to be used to evaluate, in an integrated way, intelligence with creativity in order to improve child psychodiagnosis and provide a more complete cognitive assessment.

In the first study with the BAICI, we observed the need to review some items of vocabulary, logical thinking and memory tests, thus resulting in some changes to the instrument’s presentation. The study found that the drawings in the vocabulary and memory tests should be reviewed to enable better understanding and that 2 items on the logical thinking tests should be repositioned to the final part of the test because they presented greater difficulty. Therefore, considering that the difficulty of the items impacted the results, a review was performed and a second study was conducted. The accuracy of the BAICI subtests, in turn, was considered adequate because high reliability indexes were obtained from the two methods used (Rasch and Cronbach).

Interestingly, the visual-spatial thinking test was not adequate because it indicated superior results for the group of children from regular schools. It should be emphasized that the children with high abilities tended to observe more details in each figure, thus demonstrating a tendency toward perfectionism. This may have impaired their scores on the test, and this characteristic has been previously pointed out by other authors (Almeida &Reppold, 2019). Therefore, this test should be reviewed in the next version of the BAICI. Additionally, the differences obtained for gender on vocabulary test confirm what had already been observed in the study by Wechsler et al. (2014), which demonstrated that boys had higher vocabulary results on the WJ-III test, indicating the future need for norms separated by gender on the verbal ability test.

There is consensus that intelligence is related to creativity, but how much these constructs overlap remains a question (Kaufman & Plucker, 2011; Kim et al., 2010). Some authors (Kim, 2005) perceive these constructs as independent, while others (Silvia, 2015) maintain that creativity and intelligence are closely related. In this study, although there was also a relationship between intelligence and creativity, this association was not significant. Thus, the importance of evaluating both constructs to obtain a better understanding of human potential has been demonstrated (Barbot et al., 2019; Karwowski et al., 2016; Runco, 2014). Therefore, a battery of tests, such as the BAICI, that can provide an evaluation of these two important dimensions in an integrated way can contribute to child psychodiagnosis.

The limitations of this study should be considered, such as the small sample size in some regions and the fact that the research was conducted with certain age groups. The scarcity of special programs for children with high skills/giftedness also limited the scope of the study for comparison with other Brazilian cities. Future studies with the BAICI may expand to include samples from different regions and propose comparative standards by age. In turn, the results indicated that the BAICI can contribute to the broader cognitive assessment process, integrating intelligence with creativity, to provide a more complete diagnosis of child cognitive potential.

Acknowledgments:

This work was funded by the National Council for Technical and Scientific Development (CNPQ).

References

• Ackerman, P. (2013). Assessment of intellectual functioning in adults. In K. F. Geisinger (Eds.), APA handbook of testing and assessment in Psychology. Testing and Assessment in Clinical and Counseling Psychology (pp.119-132). American Psychological Association. https://doi.org/10.1037/14048-008
  » https://doi.org/10.1037/14048-008
• Alencar, E.M. S., & Fleith, D. (2005). Attention to the student who stands out for a higher potential. Special Education Journal, 27, 1-5. https://periodicos.ufsm.br/educacaoespecial/article/view/4346/2565
  » https://periodicos.ufsm.br/educacaoespecial/article/view/4346/2565
• Almeida, L. S., Wechsler, S.M., Oliveira, E. P., Cruz, J. F.A. (2019). Positive Psychology: an enriched look of overdom and excellence. In L. S. Almeida & C. T. Reppold (Eds). Positive psychology: Education, health and work (pp 171-188). Center for Studies and Resources in Psychology.
• Barbot, B., Reiter-Palmon, R., & Hass, R. W. (2019). Creativity assessment in psychological research: Re(setting) the standards. Psychology of Aesthetics, Creativity and the Arts, 15 (2) 233-240. https://doi.org/10.1037/aca0000233
  » https://doi.org/10.1037/aca0000233
• Benson, N. F., Floyd, R. G., Kranzlerc, J. H., Eckertd, T. E., Fefere, S. A., & Morgana, G.B.(2019). Test use and assessment practices of school psychologists in the United States: Findings from the 2017 National Survey. Journal of School Psychology, 72, 29-48. https://doi.org/10.1016/j.jsp.2018.12.004
  » https://doi.org/10.1016/j.jsp.2018.12.004
• Boone, W. J., Staver, J. R., & Yale, M. S. (2014). Rasch Analysis in the Human Sciences. Springer. https://doi.org/ 10.1007/978-94-007-6857-4
  » https://doi.org/ 10.1007/978-94-007-6857-4
• Cattell, R. B. (1971). Abilities: Their structure, growth and action. Houghton Mifflin.
• Federal Council of Psychology (2018). Federal Council of Psychology approves new resolution on psychological assessment. https://site.cfp.org.br/nova-esolucao-over-evaluation-psychologica/
  » https://site.cfp.org.br/nova-esolucao-over-evaluation-psychologica/
• Flanagan, D. P., & McDonough, E. M. (2018). Contemporary psychological assessment theories, tests and issues. The Guilford Press.
• Guilford, J. P. (1968). Intelligence, creativity, and their educational implications. Roberto R. Knarp.
• Jauk, E., Benedek, M., Dunst, B., & Neubauer, A. C. (2013). The relationship between intelligence and creativity: New support for the threshold hypothesis by means of empirical breakpoint detection. Intelligence, 41, 212-221. https://doi.org/10.1016/j.intell.2013.03.003
  » https://doi.org/10.1016/j.intell.2013.03.003
• Karwowski, M., Dul, J., Gralewski, J., Jauk, E., Dorota, M. Jankowska, D.M., Gajda, A., Chruszczewski, M. H., & Benedek, M. (2016). Is creativity without intelligence possible? A Necessary Condition Analysis. Intelligence, 57, 105-117. https://doi.org/10.1016/j.intell.2016.04.006
  » https://doi.org/10.1016/j.intell.2016.04.006
• Kaufman, J. C. (2015). Why creativity isn’t in IQ tests, why it matters and why it won’t change anytime soon probably. Journal of Intelligence, 3(2), 59-72. https://doi.org/10.3390/jintelligence3030059
  » https://doi.org/10.3390/jintelligence3030059
• Kaufman, J. C. (2016). Creativity 101. Springer
• Kaufman, J. C, Plucker, J. A, & Russell, C.M. (2012). Identifying and assessing creativity as a component of giftedness. Journal of Psychoeducational Assessment, 3. 60-73. https://doi.org/10.1177%2F0734282911428196
  » https://doi.org/10.1177%2F0734282911428196
• Kaufman, J. C. & Sternberg, R. (2019). The Cambridge handbook of creativity (2nd ed.). Cambridge University Press. https://doi.org/10.1017/9781316979839
  » https://doi.org/10.1017/9781316979839
• Kim, K. H. (2005). Can only intelligent people be creative? The Journal of Secondary Gifted Education,2(3), 57-66. https://doi.org/10.4219/jsge-2005-473
  » https://doi.org/10.4219/jsge-2005-473
• Kim, K. H., Cramond, B., & VanTassel-Baska, J. (2010). The relationship between creativity and intelligence. In J. C. Kaufman, & R. J. Sternberg (Eds.), The Cambridge handbook of creativity (pp. 395-412). Cambridge University Press
• Linacre, M. (2020). WIINSTEP help for Rasch analysis. https://www.winsteps.com/winman/index.htm
  » https://www.winsteps.com/winman/index.htm
• Martins, B. A., Pedro, K. M., & Ogeda, C.M.M. (2016). High incapacity/giftedness: say the research on these invisible children? School Psychology and ‘Educational, 20(3), 561-568. https://dx.doi.org/10.1590/2175-3539201502031046
  » https://dx.doi.org/10.1590/2175-3539201502031046
• McGrew, K. S. (2009). CHC theory and the human cognitive abilities project: Standing on the shoulders of the giants of psychometric intelligence research. Intelligence, 37, 1-10. http://dx.doi.org/10.1016/j.intell .2008.08.004
  » http://dx.doi.org/10.1016/j.intell .2008.08.004
• Milan, Q. G., & Wechsler, S.M. (2018). Integrated assessment of intelligence and creativity. Journal of Psychology, 36 (2), 525-548. https://dx doi.org/10.18800/psico.201802.005
  » https://dx doi.org/10.18800/psico.201802.005
• Nakano, T.C., Campos, C. R. & Santos, M. V. (2016). High rating scale skills / gifted - teacher version: content validity. Interdisciplinary Studies in Psychology, 7(1), 103-123.
• Nakano, T.C., (2018). Current Methods for Assessing Creativity: Advantages and Questions. Psychological Evaluation, 19(1), 97-105. http://dx.doi.org/10.15689/ap.2020.1901.14269.11
  » http://dx.doi.org/10.15689/ap.2020.1901.14269.11
• Organization for Economic Cooperation and Development- OECD (2020). Fostering students’ creativity and critical thinking. Retrieved from https://www.oecd-ilibrary.org/content/publication/62212c37-en
  » https://www.oecd-ilibrary.org/content/publication/62212c37-en
• Peiffer, S. I. (2018). APA handbook of giftedness and talent. American Psychological Association.
• Plucker, J. A., Esping, A., Kaufman, J. C., & Avitia, M. J. (2015). Creativity and intelligence. In S. Goldstein, D. Princiotta, & J. Naglieri (Eds). Handbook of intelligence. (pp.19-25). Springer. http://doi.org/10.1007/978-1-4939-1562-0_19,
  » http://doi.org/10.1007/978-1-4939-1562-0_19
• Primi, R., Nakano, T. C., & Wechsler, S. M. (2012). Cross-battery analysis of the Battery of Reasoning Abilities (BPR-5) with the Woodcock-Johnson Cognitive Tests (WJ-III). Temas em Psicologia, 20, 121-132. https://doi.org/10.1590/1413-82712019240414
  » https://doi.org/10.1590/1413-82712019240414
• Renzulli, J. S., Smith, L. H., White, A. J., Callahan, C. M., & Hartman, R. K. (1976). Scales for rating the behavioral characteristics of superior students. Manual. Creative Learning Press.
• Renzulli, J. S, Renzulli, S.R. (2010). The Schoolwide Enrichment Model: A Focus on Student Strengths and Interests. Gifted Education International, 26,140-156. http://10.1177/026142941002600303
  » http://10.1177/026142941002600303
• Runco, M. A. (2014). Creativity: theories and themes, research, development and practice (2nd. Ed.) . Elsevier Academic Press.
• Runco, M., Pritzker, S. (2020). Encyclopedia of creativity (3rd ed). Academic Press.
• Sattler, J. (2018). Assessment of children: Cognitive foundations, applications an resource guide (6th ed). Sattler.
• Department of Education Special (1995). General guidelines for the educational care for students with high skills/gifted and talents. Ministry of Education.
• Secretariat of Special Education (2002). Live School Project: Ensuring access and permanence of all students in school: Students with educational needs Special. Ministry of Education.
• Silvia, P. J., Beaty, R. E, & Nusbaum, E. C. (2013). Verbal fluency and creativity: General and specific contributions of broad retrieval ability factors to divergent thinking. Intelligence, 41, 328-340. https://doi.org/10.1016/j.intell.2013.05.004
  » https://doi.org/10.1016/j.intell.2013.05.004
• Silvia, P. J. (2015). Intelligence and creativity: Are pretty similar after all. EducationalPsychology, 27, 599-606. https://doi.org/10.1007/s10648-015-9299-1
  » https://doi.org/10.1007/s10648-015-9299-1
• Simonton, D.K. (2018), Defining Creativity: Don’t We Also Need to Define What Is Not Creative? Journal of Creative Behavior, 52: 80-90 https://doi.org/10.1002/jocb.137
  » https://doi.org/10.1002/jocb.137
• Schneider, W. J., & McGrew, K. S. (2018). The Cattell-Horn-Carroll theory of intelligence. In D. P. Flanagan, & E. M. McDonough (Eds.). Contemporary intellectual assessment (pp. 73-163). Guilford Press
• Psychological Tests Assessment System - SATEPSI (2021). Welcome to the Psychological Test Assessment System. https://satepsi.cfp.org.br/
  » https://satepsi.cfp.org.br/
• Torrance, E. P. (1966). Torrance tests of creative thinking-norms technical manual research edition. Personnel Pres
• Virgolim, A. M. R. & Konkiewitz, E.C. (2018). High skills/gifted, intelligence and creativity. Campinas; Papirus Publisher
• Wechsler, S.M. (2004a). Evaluation of creativity by figures. Test Torrance. Brazilian version. LAMP/PUC-Campinas.
• Wechsler, S.M. (2004b). Evaluation of creativity by words. Torrance test. Brazilian version. LAMP/PUC-Campinas
• Wechsler, S. M., & Fleith, D. (2017). The scenario of gifted education in Brazil. Cogent Education, 4(1), 1-4. http://dx.doi.org/q1
  » http://dx.doi.org/q1
• Wechsler, S. M., Machado, W. L., Ribeiro, R. K. S., Schneider, P., Goetz, E., Goetz, E., Costa W., & Costa, C. I. (2019); Analysis of the internal structure of the Battery Intellectual Evaluation of Young People . Psycho-USF, 24(4), 779-790. https://doi.org/10.1590/1413-82712019240414
  » https://doi.org/10.1590/1413-82712019240414
• Wechsler, S.M., Nakano, T.C., Domingues, S. F. s., Rosa, H. R., Silva Silva-Filho, J. H., & Minervino, C. A. S. M. (2014a). Gender differences on tests of crystallized intelligence. European Journal of Education and Psychology, 7(1), 57-70. https://doi.org/ 10.1989/ejep.v7i1.152
  » https://doi.org/ 10.1989/ejep.v7i1.152
• Wechsler, S. M., Nunes, C., Schelini, P. W., Pasian, S. R., Moretti, L, Anache, A.. (2010). Brazilian adaptation of the Woodcock-III tests. School Psychology International 31(4), 409-421. https://doi.org/10.1177%2F0143034310377165
  » https://doi.org/10.1177%2F0143034310377165
• Wechsler, S.M., Vendramini. C.M., & Schelini, P.W. (2007). Brazilian adaptation of the verbal tests of the Woodcock-Johnson Battery III. Inter-American Journal of Psychology , 41, 285-294.
• Wechsler, S.M., Vendramini, C.M.M., Schelini, P. W., Lourençoni, M. A., Souza, A.A., & Mundim, M. C. (2014b). Factorial structure of the Brazilian Adult Intelligence Battery. Psychology & Neuroscience, 7(4), 559-566. https://doi.org/10.3922/j.psns.2014.4.15
  » https://doi.org/10.3922/j.psns.2014.4.15
• Woodcock, R. W., McGrew, K. S., & Matter, N. (2001). Woodcock-Johnson III. Itasca, IL: Riverside Publishing

Publication Dates

• Publication in this collection
  06 Jan 2023
• Date of issue
  Oct-Dec 2022

History

• Received
  02 Dec 2020
• Reviewed
  28 Apr 2021
• Accepted
  06 July 2021