Validity Evidences for the Portuguese Version of the Problem Behavior Scale: Preschool and Kindergarten Behavior Scales-2¹

Evidências de Validade da Versão Portuguesa da Escala de Problemas do Comportamento: Preschool and Kindergarten Behavior Scales-2

Evidencias de Validez de la Versión Portuguesa de la Escala de Problemas de Conducta: Preschool and Kindergarten Behavior Scales-2

ABSTRACT

The early identification of problem behaviors is essential in preschool. This paper presents evidence of validity (confirmatory factor analysis) for the Problem Behavior scale of the Portuguese version of the Preschool and Kindergarten Behavior Scales - Second Edition (PKBS-2). Analyses were performed for the scale's 46 items, which were grouped into 16 item-parcels. Once it was verified that the model fit the total sample (N = 2000; CFI = .98; RMSEA = .06), analyses were replicated for the samples collected at home and at school (n = 1000 per setting). Results indicate a factor structure equivalent to the original version, with five supplemental subscales, distributed into two subscales (Externalizing and Internalizing), stable for the two subsamples, with high internal consistency levels (α = .78-.97). The discussion highlights the utility/validity of the Portuguese version of the Problem Behavior scale among preschoolers.

Keywords
factor analysis; behavior disorders scales; preschool students

RESUMO

A identificação precoce de problemas do comportamento é essencial na idade pré-escolar. Este artigo tem como objetivo apresentar evidências de validade (análise fatorial confirmatória) da escala de Problemas do Comportamento da versão portuguesa das Preschool and Kindergarten Behavior Scales - Second Edition (PKBS-2). Foram efetuadas análises para os 46 itens da escala, posteriormente reagrupados em 16 parcelas. Verificado o ajustamento do modelo para a totalidade da amostra (N = 2000; CFI = 0,98; RMSEA = 0,06), as análises foram replicadas para as subamostras recolhidas em casa e na escola (n = 1000 por contexto). Os resultados apontam para uma estrutura fatorial equivalente à da versão original, com cinco subescalas suplementares, repartidas por duas subescalas (Externalizante e Internalizante), e estável para as duas subamostras, com valores elevados de consistência interna (α = 0,78-0,97). A discussão enfatiza a utilidade/validade da versão portuguesa da escala de Problemas do Comportamento com pré-escolares.

Palavras-chave
análise fatorial; escalas distúrbios do comportamento; pré-escolares

RESUMEN

La identificación temprana de los problemas de conducta es esencial en el preescolar. Este artículo tiene como objetivo presentar evidencias de validez (análisis factorial confirmatorio) de la escala de Problemas de Conducta de la versión portuguesa de las Preschool and Kindergarten Behavior Scales - Second Edition (PKBS-2). Se realizaron análisis para los 46 ítems de la escala, posteriormente reagrupados en 16 parcelas. Se verifico el ajuste del modelo en la muestra completa (N = 2000; CFI = 0,98; RMSEA = 0,06), y se replicaron los análisis en las submuestras recogidas en casa y en la escuela (n = 1000 por contexto). Los resultados apuntan a una estructura factorial equivalente a la de la versión original, con cinco subescalas suplementarias, distribuidas en dos subescalas (Externalizante e Internalizante), que se mantiene en ambas submuestras, con valores elevados de consistencia interna (α = 0,78-0,97). La discusión acentúa la utilidad/validez de la versión portuguesa de la escala de Problemas de Conducta con preescolares.

Palabras clave:
análisis factorial; escalas trastornos de la conducta; pre escolares

The preschool age is a period of accelerated growth and development. At this developmental stage, children commonly exhibit motor instability, impulsiveness or oppositional behavior, which, even though very typical at this age, it may not be very different from behaviors exhibited by children with a diagnosis of conduct disorder (Rescorla et al., 2011), considered to be disruptive (Keenan & Wakschlag, 2002). This fact may lead to serious difficulties in distinguishing between symptomatic behaviors and problems associated with normal or transitory developmental difficulties (Fuchs, Klein, Otto, & von Klitzing, 2013; Studts & van Zyl, 2013). Moreover, the presence of a series of problems (e.g., aggressiveness or defiant behavior) exhibited in various contexts (e.g., school and home) deserves increased attention (Campbell, Shaw, & Gilliom, 2000), as some of these children may be considered "early starters" (preschoolers who manifest aggressiveness, defiance and/or cruelty) in terms of behavioral problems manifested during adolescence (Bornstein, Hahn, & Haynes, 2010; Studts & van Zyl, 2013).

The early identification of difficulties in terms of interpersonal functioning and behavioral problems is crucial in the preschool period (Basten et al., 2016; Campbell et al., 2000; Studts & van Zyl, 2013; Upshur, Wenz-Gross, & Reed, 2013), as disobedient and aggressive children represent a real challenge for parents, teachers and other caregivers. In this sense, disruptive behavior, such as aggressiveness and disobedience, are the most common reason mental health services are sought during the preschool period (Keenan & Wakschlag, 2002). Additionally, behavioral problems in the preschool period are often associated with future difficulties in terms of academic performance, social interaction with peers, and antisocial behavior (Campbell et al., 2000; Upshur et al., 2013). Further, the stability of behavioral and emotional problems over time has been emphasized (Bagner, Rodríguez, Blake, Linares, & Carter, 2012; Bornstein et al., 2010; Metcalfe, Harvey, & Laws, 2013; Upshur et al., 2013), which highlights the importance of correctly establishing and describing behaviors that may be problematic in younger children (Basten et al., 2016; Studts & van Zyl, 2013). Even though preschool-aged children may present behavioral problems that cause concern in parents and other adults, research and progress in clinical practice are more evident in the psychopathology of school-aged children when compared to younger children (Campbell et al., 2000). In fact, this reality is reflected in the low number of papers addressing the preschool period (Fuchs et al., 2013).

The last 20-30 years of research developed in this field reflects some consensus in regard to the existence of two types of relatively stable problems. On the one hand, there are externalizing problems, characterized by out-of-control and anti-social behavior, hyperactive-impulsive behavior, acting-out, and aggressive behavior with a tendency to bother and/or hurt others. On the other hand, there are internalizing problems, such as social withdrawal, anxiety, depression and somatic complaints, problems that impact mostly the child her/himself (Achenbach & Edelbrock, 1978; Cicchetti & Toth, 1991; Whitcomb & Merrell, 2013).

It is key, in the context of the early identification of problem behavior, to develop instruments to assess specific behaviors in preschoolers rather than to use a mere extension of assessment instruments devised for other age groups (Whitcomb & Merrell, 2013). Such instruments can be valuable to help practitioners distinguish normal and/or transitory behaviors from behavioral disorders that demand interventions among preschool-aged children (Keenan & Wakschlag, 2002; Studts & van Zyl, 2013).

There are various specific behavior assessment scales for preschoolers available in the literature (Bagner et al., 2012; Whitcomb & Merrell, 2013). Among these, Achenbach's inventories stand out as the most frequently studied and cited (translated into 90 languages) instruments to assess behavior, with versions for parents and teachers, applicable to children from one and a half years to 5 years old - the Child Behavior Checklist for Ages 1^1/2-5 - CBCL/1^1/2-5 (Achenbach & Rescorla, 2000) and the Caregiver-Teacher Report Form for Ages 1^1/2-5 - C-TRF (Achenbach, 1997). In turn, the Strengths and Difficulties Questionnaire - SDQ (Goodman, 2001), translated into 80 languages, is a screening questionnaire to assess signs of psychopathology and pro-social behavior in children and adolescents, with a recent update of its two versions (on June 2014), one for 2-4 year old children (early-years version) and another for 4-17 year old (standard version). The Ages & Stages Questionnaires: Social-Emotional - ASQ-SE (Squires, Bricker, & Twombley, 2002) is equally applicable to describe the behavior of children early in life. It is designed to assess socio-emotional and problem behavior, and social competency among children from 6 to 60 months of age. The Behavior Assessment System for Children, Second Edition - BASC-2 (Reynolds & Kamphaus, 2004) assesses problem behavior, problems at school and the adaptive skills of children between 2 and 5 years old. Finally, the Social Skills Improvement System Rating Scales - Preschool Level - SSIS (Gresham & Elliott, 2008) enable assessing social skills and behavioral problems in children for the ages from 3 to 5.

The Preschool and Kindergarten Behavior Scales - Second Edition - PKBS-2 (Merrell, 2002) can be added to this set of instruments. These are scales to assess Social Skills and Problem Behavior, specifically developed to assess children between 3 and 6 years of age (Merrell, 2002; Whitcomb & Merrell, 2013). The PKBS-2 are used for different purposes (e.g., screening, monitoring interventions, research) and completing them requires only 8 to 12 minutes (Merrell, 2002). The Social Skills scale is composed of 34 items divided into three subscales (Social Cooperation, Social Interaction and Social Independence). The Problem Behavior scale is composed of 42 items and is divided into Externalizing Problem Behavior (27 items refer to disruptive behavior, out-of-control, or hyperactivity) and Internalizing Problem Behavior (15 items associated with emotional problems). In its original American version, five supplemental Problem Behavior subscales stand out: Self-Centered/Explosive (11 items), Attention Problems/Over-Activity (8 items) and Anti-Social/Aggressive (8 items) for externalizing problems; and Social Withdrawal (7 items) and Anxiety/Somatic Complaints (8 items) for internalizing problems (Merrell, 2002). The PKBS-2 were translated into various languages such as Spanish (Carney & Merrell, 2002) and German (Al Awmleh & Woll, 2013), and were also translated, adapted and validated for the Portuguese preschool population (Major, 2011; Major & Seabra-Santos, 2014b).

Despite the acknowledged importance of identifying problem behaviors in preschoolers (Upshur et al., 2013), this assessment in the Portuguese context has been hindered due to a lack of screening tools specific to this age group. In this sense, in order to fill in the gap that exists due to a small number of studies addressing the adaptation and validation of instruments to assess problem behavior in Portugal, and going through the studies addressing the factor validity of the Social Skills scale of the Portuguese version of the PKBS-2 (Major & Seabra-Santos, 2014a), this study's general aim is to analyze the psychometric properties of this instrument's Problem Behavior scale. The specific objectives are to verify the factor validity (construct) and reliability of this scale's items/parcels.

Method

Participants

A sample of 1000 children, assessed according to adaptation and validation studies of PKBS-2 for the Portuguese population, was used in this study (Major, 2011; Major & Seabra-Santos, 2014b). This sample was recruited from the entire country and stratified for several variables, such as: age (250 children for each of the following ages: 3, 4, 5 and 6 years old) and sex (50% girls and 50% boys). Hence, a total of 1000 children aged 4.5 years old on average (SD = 1.12) were independently assessed by parents and teachers, totaling 2000 protocols (See Major, 2011 for an exhaustive description of the sample). Validity studies were conducted based on two contexts (family/school) following the same procedures adopted for the development of PKBS-2's normative data (Merrell, 2002) and later for the Portuguese version (Major, 2011). In this sense, 1000 protocols were completed by an informant from the family context (834 mothers, 118 fathers and 48 protocols were completed by both parents together or another caregiver such as grandparents). The same 1000 children were assessed by their respective teachers (M = 8 scales were completed by each teacher); 98.5% of the 131 teachers participating in the study were women.

Instrument

The Portuguese version of the PKBS-2 (Escalas de Comportamento para a Idade Pré-Escolar - Segunda Edição - ECIP-2; Major, 2011; Major & Seabra-Santos, 2014b) was used in this research. It has a total of 80 items distributed into two scales (Social Skills with 34 items and Problem Behavior with 46 items). Each item is rated on a four-point Likert scale (0 - Never, 1 - Rarely, 2 - Sometimes, and 3 - Often) according to the child's behavior in the last three months (Merrell, 2002). After translation/adaptation and back translation of the original version's items (Major, 2011), the exploratory factor analysis confirmed the American structure with 46 items of the Problem Behavior scale distributed into two subscales: Externalizing Problem Behavior (29 items) and Internalizing Problem Behavior (17 items). Similar to the American version, the second-order factor analyses indicate five supplemental subscales, three for externalizing problems: Anti-Social/Aggressive (12 items, e.g., "Gets into many fights"), Opposition/Explosive (9 items, e.g., "Has temper outbursts or tantrums"), and Over-Activity/Lack of Attention (8 items, e.g., "Is restless and fidgety"); and two for internalizing problems: Social Withdrawal (8 items, e.g., "Has problems making friends") and Anxiety/Somatic Complaints (9 items, e.g., "Is anxious or tense"; Major, 2011; Major & Seabra-Santos, 2014b). In comparison to the original version, item 33 "Acts younger than his or her age" was removed due to a lack of specificity, as it represents a general description of children's behavior, more than a specific behavior. It also loaded too close in the two factors considered, .37 and .34 for externalizing and internalizing problems, respectively. Of the 12 items added at the time of the adaptation studies, seven were eliminated because they overlapped original items or because of less adequate psychometric functioning (assessed through internal consistency analysis and correlation between items; Major, 2011).

The original version of the PKBS-2 was developed based on a normative sample of 3313 children and the good psychometric qualities of the Problem Behavior scale were apparent, specifically in terms of its internal consistency, assessed for the total sample (α total score = .97 and between .84 and .97 for the Externalizing and Internalizing supplemental subscales), and considering the subsamples collected in the family (α = .96 and between .78 and .95, respectively) and school contexts (α = .97 and between .84 and .97, respectively). In regard to the confirmatory factor analysis studies, a stable structure was obtained for the Problem Behavior scale: χ²(811) = 7872.90, p < .001, AGFI = .98 (Merrell, 2002).

Procedure

Data collection. First, authorizations were obtained to conduct the study (author of the PKBS-2, PRO-ED, National Data Protection Commission, Regional Education Boards, and General Board of Innovation and Curricular Development). The schools were randomly selected for the study and after selection, clarification was provided to the school's principals/managers. Afterwards, the protocols (family and school contexts) were collected, with the help of teachers. After the random selection of children from the classroom lists, the teachers received two copies of the Portuguese version of PKBS-2 for each child: one for themselves and another for the parents. The questionnaire was delivered to the parents inside an envelope together with a letter presenting the project and an informed consent form (the return rate was 87.8%). When the parents returned the informed consent form signed, the teachers completed the scales (the return rate was 100%).

Data analysis. IBM SPSS Amos version 20 was used to perform confirmatory factor analysis (CFA) for the Problem Behavior scale (PB) using the maximum likelihood estimation procedure. The CFA studies involved first- and second-order analyses, with a factor structure with two correlated factors for the Externalizing and Internalizing subscales and five factors concerning the supplemental subscales: three externalizing and two internalizing. The analyses were performed for all the protocols collected (N = 2000) and replicated for the protocols collected in the family and school contexts in order to check the stability of the model obtained.

First, the 46 items of the PB scale were submitted to CFA. Due to the high number of items in this scale and to improve the model's fit and the lower reliability of the items, when these were individually submitted to CFA (Little, Cunningham, Shahar, & Widaman, 2002), the items included in the previously established five supplemental subscales (Major, 2011) were grouped into parcels, in a similar way to how these procedures adopted in the CFA were also implemented for the Social Skills scale (Major & Seabra-Santos, 2014a). Each of the 16 parcels was composed of the sum of sets from two to four items (three to four parcels for supplemental subscale), representing an aggregated index of the constructs assessed for each subscale (items parcels; Little et al., 2002). The factor loadings that resulted from the exploratory factor analysis (Table 1) grounded the distribution of items and an attempt to reach a more equitable distribution of these parcels (Little et al., 2002). Therefore, the item-to-construct balance procedure proposed by Little et al. (2002) was used, in which the four items with the highest factor loading in the supplemental subscales of Anti-Social/Aggressive (AA) served as a basis for the four parcels to be constructed (AA1, AA2, AA3 and AA4). Afterwards, the four following items were added in inverse order to the four parcels, so that the item with the highest factor loading obtained in this second distribution, was added to the parcel that obtained the item with the lowest factor loading. This procedure was repeated until the 12 items in this subscale were exhausted, then likewise for the 9 items of the Opposition/Explosive (OE) subscale, the 8 items in the Over-Activity/Lack of Attention subscale (OAL), the 8 items of the Social Withdrawal subscale (SW), and for the 9 items concerning the Anxiety/Somatic Complaints (ASC).

The following goodness-of-fit indexes were used: Chi-square goodness-of-fit test (χ²⁾, ratio of χ² for degrees of freedom (χ²/df), goodness-of-fit index (GFI), comparative fit index (CFI) and root mean square error of approximation (RMSEA) with a 90% confidence interval (CI) (Jackson, Gillaspy, & Purc-Stephenson, 2009; Marôco, 2010). To verify the model's fit, the following cut-off points were considered: χ² with a reduced value associated with a significance level > .05, χ²/gl ˂ 2 (Marôco, 2010), as well a value close to .95 for GFI and CFI and close to .06 for RMSEA (Hu & Bentler, 1999), considering values .90-.94 for GFI and CFI as synonymous of a good model fit; ≥ .95 is considered to be a very good model fit (Marôco, 2010).

Descriptive, kurtosis and skewness analyses were performed using IBM SPSS Statistics 20. The correlations between factors were computed using Pearson's coefficient of correlation. Cronbach's alpha was also used (degree of consistency/congruency among items; Kline, 2010), as was composite reliability (CR, degree in which items reflect the latent factor; Marôco, 2010) to verify internal consistency.

Ethical Considerations

The study was approved by the Portuguese Institutional Review Board (Comissão Nacional de Proteção de Dados [National Committee for Data Protection]; Protocol n. 3222/2006). After having the authorizations required to use the PKBS-2, the parents' consent to participate in the study and their authorization for data to be collected by their children's teachers were also obtained. Both parents and teachers received clarification regarding the study's goals and were informed that their participation was voluntary and that the confidentiality of the data and their anonymity was ensured. The parents returned the completed ECIP-2 in a closed envelop to ensure the confidentiality of their responses.

Results

Five supplemental subscales distributed into two PB subscales (three subscales for Externalizing PB and two subscales for the Internalizing PB; Major, 2011) emerged from the exploratory factor analysis (principal component analysis with Direct Oblimin rotation), which served as a basis for the construction of parcels considered in CFA (Table 1).

Table 1
Exploratory Factor Analysis: Rotated Components Matrix of Supplemental Problem Behavior Subscales

	Item	Externalizing PB			Internalizing PB
		F1	F2	F3	F4	F5
29.	Bullies or intimidates ...	.91
21.	Seeks revenge ...	.81
50.	Gets into fights...	.81
34.	Destroys ...	.77
11.	Is physically aggressive ...	.67
26.	Calls names ...	.66
42.	Bothers ...	.64
35.	Is moody...	.57
3.	Teases ...	.56
14.	Takes things ...	.52
31.	Has unpredictable ...	.41
40.	Tells lies	.36
8.	Wants all ...		.85
7.	Has temper outbursts ...		.77
19.	Must have own way ...		.75
32.	Is jealous ...		.67
13.	Yells ...		.64
46.	Can not wait ...		.59
41.	Is easily provoked ...		.54
22.	Defies ...		.45
10.	Will not share		.41
20.	Is overly active ...			.75
25.	Is restless...			.73
15.	Has difficulty concentrating ...			.73
1.	Acts impulsively ...			.59
16.	Disobeys ...			.57
39.	Disrupts ...			.55
6.	Makes noises ...			.50
52.	Does not change ...			.34
17.	Has problems ...				.85
12.	Avoids playing ...				.78
28.	Withdraws ...				.78
47.	Looks apathetic ...				.73
51.	Does not have fun ...				.67
30.	Seems unhappy ...				.64
27.	Is difficult to comfort ...				.47
4.	Does not respond ...				.45
37.	Whines ...					.84
36.	Is overly sensitive ...					.81
23.	Complains ...					.73
5.	Clings to parents ...					.66
2.	Becomes sick ...					.48
24.	Resists ...					.46
9.	Is anxious ...					.46
18.	Is afraid ...					.40
38.	Gets taken advantage ...					.33

• ^Note
  PB = Problem Behavior, F1 = Anti-Social/Aggressive, F2 = Opposition/Explosive, F3 = Over- Activity/Lack of Attention, F4 = Social Withdrawal, F5 = Anxiety/Somatic Complaints. The version of items presented is abbreviated.

The first CFA performed with the 46 items of the PB scale indicated standardized factor loadings between .48 and .79, with a correlation of .60 between factors of Externalizing and Internalizing Problems. The goodness-of-fit indexes of the model, however, presented inadequate values, χ²(988) = 13386.59, p < .001; χ²/df = 13.55; GFI = .67; CFI = .77; RMSEA = .08. The introduction of changes resulting from modification indices did not improve the model to the point that reference values were achieved (∆χ²= 4192.79; ∆CFI = .08). In fact, both for the tripartite structure of externalizing problems, χ²(374) = 4314.73, p < .001; χ²/df = 11.54; GFI = .86; CFI = .89; RMSEA = .07, and for the bi-factor structure of internalizing problems, χ²(118) = 1995.34, p < .001; χ²/df = 16.91; GFI = .89; CFI = .85; RMSEA = .09, the models obtained results that were also below expectations, indicating the need of further CFA studies using parcels.

Descriptive analyses of the 16 parcels constructed for the PB scale, presented in Table 2, reveals that parcel OE1 presents the highest mean (M = 3.74; SD = 2.33), while parcel SW3 presents the lowest mean (M = 1.13; SD = 1.19). The range of the parcels' results is between a minimum of 0 (for the 16 parcels) and a maximum of 6-9. All the skewness values (Sk) are positive, as are seven of the 16 values concerning kurtosis (Ku), which meets, without exception, the assumption of normality (|Sk| < 3 and |Ku| < 10), according to criteria proposed by Marôco (2010). The fit of this set of indicators suggests that the parcels under study function appropriately.

Table 2
Descriptive Statistics of the 16 Parcels from the Problem Behavior Scale: Total Sample (N = 2000)

Parcels	M	SD	Min	Max	Skewness	Kurtosis
AA1	1.94	1.80	0	9	1.02	0.79
AA2	2.12	1.95	0	9	0.90	0.46
AA3	1.77	1.78	0	9	1.07	0.94
AA4	1.76	1.76	0	9	0.99	0.60
OE1	3.74	2.33	0	9	0.16	-0.69
OE2	3.40	2.46	0	9	0.35	-0.72
OE3	3.47	2.07	0	9	0.10	-0.66
OAL1	3.12	2.26	0	9	0.36	-0.63
OAL2	3.37	2.25	0	9	0.30	-0.60
OAL3	2.76	1.43	0	6	0.05	-0.50
SW1	1.95	1.69	0	9	0.80	0.45
SW2	2.01	1.64	0	9	0.59	0.02
SW3	1.13	1.19	0	6	0.91	0.40
ASC1	2.97	1.83	0	9	0.36	-0.25
ASC2	3.46	1.89	0	9	0.15	-0.47
ASC3	2.97	1.88	0	8	0.19	-0.58

• ^Note
  AA1/2/3/4 = Anti-Social/Aggressive 1/2/3/4, OE1/2/3 = Opposition/Explosive 1/2/3, OAL1/2/3 = Over-Activity/Lack of Attention 1/2/3, SW1/2/3 = Social Withdrawal 1/2/3, ASC1/2/3 = Anxiety/Somatic Complaints 1/2/3.

The CFA using parcels indicated a good model fit for the total sample, χ²(98) = 1274.27, p < .001; χ²/gl = 13.00; CFI = .96; GFI = .93; RMSEA = .08. The analysis of modification indices indicated the need to establish correlations between residual error of factors AA and SW (r = .54), as well as OE and ASC (r = .96), which improved the model's fit (CFI = .98; GFI = .95; RMSEA = .06). The analysis presented in Figure 1 shows that all standardized factor loadings of the 16 parcels were situated between .75 (SW2) and .92 (OAL2). The correlations among the five supplemental subscales were moderately high, and were even higher when belonging to the same subscale (Externalizing/Internalizing), ranging between .40 (Over-Activity/Lack of Attention and Social Withdrawal) and .79 (Opposition/Explosive and Over-Activity/Lack of Attention) (p < .001). The Externalizing and Internalizing subscales presented a correlation of .59.

Figure 1
Confirmatory factor analysis final model: Problem Behavior scale (Parcels)

When considering the protocols collected from the family context, χ²(96) = 359.75, p < .001; χ²/df = 3.75; GFI = .96; CFI = .97; RMSEA = .05, and school context, χ²(96) = 624.41, p < .001; χ²/df = 6.50; GFI = .93; CFI = .97; RMSEA = .07, the good model fit with 16 parcels for the PB scale is confirmed. The factor loadings of the parcels in their respective factors fall between .71 and .88 for the protocols collected in the family context and between .78 and .94 for the school context.

Table 3 presents the results concerning the internal consistency of the 46 items for the three samples under study. The values for Cronbach's alpha for the total score of the PB scale were above .90 for the three samples (.95-.97), considered by Kline (2010) to be excellent. When the two PB subscales were analyzed, as well as the five supplemental subscales, the values obtained were all between .81 and .97, except the result for the supplemental subscale Anxiety/Somatic Complaints, for the protocols that were collected in the family context (α = .78). When these coefficients were compared considering the 16 parcels, the values are very similar to those obtained when the analysis is performed for the items, both in terms of the scale's total score (α = .93 to .95), and for the Externalizing and Internalizing subscales (α = .86 to .97) and the five supplemental subscales (α = .81 to .96). Note that the coefficients, both considering the items individually and in parcels, are systematically slightly higher for the teachers' assessments. In regard to the CR, all exceeded the reference value of .70 (Marôco, 2010), ranging between .89 (for the two supplemental subscales of Internalizing PB in the family context) and .98 (for the supplemental subscale Anti-Social/Aggressive in the school context).

Table 3
Internal Consistency of Items and Parcels: Total Sample, Family and School Contexts

Problem Behavior Score	Total Sample (N = 2000)			Family Context (n = 1000)			School Context (n = 1000)
	α items	α parcels	CR parcels	α items	α parcels	CR parcels	α items	α parcels	CR parcels
AA	.93	.93	.96	.89	.90	.94	.95	.96	.98
OE	.91	.88	.95	.86	.84	.91	.92	.91	.95
OAL	.90	.90	.94	.85	.85	.91	.92	.92	.96
SW	.85	.85	.92	.81	.81	.89	.88	.88	.94
ASC	.83	.86	.92	.78	.82	.89	.85	.88	.93
EPB	.96	.95	-	.94	.93	-	.97	.97	-
IPB	.89	.88	-	.86	.86	-	.92	.91	-
TPB	.96	.94	-	.95	.93	-	.97	.95	-

• ^Note
  AA = Anti-Social/Aggressive, OE = Opposition/Explosive, OAL = Over-Activity/Lack of Attention, SW = Social Withdrawal, ASC = Anxiety/Somatic Complaints, EPB = Externalizing Problem Behaviors, IPB = Internalizing Problem Behaviors, TPB = Total Problem Behaviors, CR = Composite Reliability.

Discussion

This study's aim was to present evidence of validity (CFA studies) of the PB scale of the PKBS-2's Portuguese version. CFA is considered a reference tool in the development and refinement of assessment instruments, intended to assess construct validity (Jackson et al., 2009). In this sense, poor fit indexes were obtained in the initial CFA with 46 items when using Maximum Likelihood estimation, most common procedure in the literature (Jackson et al., 2009), and adopting procedures identical to those used for the PKBS-2 original version (whole sample fitness test and according to contexts family/school). These initial results led to the development of the 16 parcels based on the PB five supplemental subscales, previously identified and distributed into two PB subscales (Externalizing and Internalizing).

The values obtained for the descriptive statistics of the 16 parcels were within the expected range and the symmetry and kurtosis values met the assumption of normality. Note that all the parcels presented a minimum value of zero and the asymmetry values concerning the distribution of scores are all positive, which is also expected and meets the construct under study (problem behavior): since it is a normative sample, children tend to score low in terms of PB, which is in accordance with the PKBS-2 manual (Merrell, 2002). The occurrence of higher correlations among the PB three Externalizing supplemental subscales and between the two Internalizing subscales than for those obtained by the first and second is also within the expected results and confirms the choice of the model under study. The factor structure obtained for PB, in addition to being in agreement with the original version, also confirmed that the 46 items/16 parcels were distributed into two dimensions of PB commonly identified in the literature - Externalizing and Internalizing (Achenbach & Edelbrock, 1978; Cicchetti & Toth, 1991).

In regard to the model's fit, even though χ² is indicated as a parameter to be considered in CFA (Marôco, 2010), its high sensitivity to the sample's large size (N = 2000) limited its utility in determining the model's fit (Byrne, 2010). Additionally, when CFA is performed with the 16 parcels, the model fit indexes for GFI and CFI correspond to more rigorous approaches (Hu & Bentler, 1999) and the RMSEA values are acceptable (Byrne, 2010 ; Marôco, 2010). Considering the critiques associated with the use of modification indices (change the model until it achieves a perfect fit) (Marôco, 2010), we only included two correlations between two pairs of residual errors (res1-res4 and res2-res5), which presented a higher modification index, justified by the correlation between externalizing and internalizing problem behaviors. The model's stability tested in CFA meets the goodness of fit expectations of Marôco (2010), considering that the protocols collected in both the family and school contexts favored the evidence of the factor validity of the PB scale, since all the values obtained for the CFI and GFI are characterized by a very good fitness level (≥ .95), with the exception of GFI in the school context are classified as having a good adjustment index.

In regard to internal consistency, with the exception of one result (ASC subscale, family context), all the remaining values are between the excellent (.90) and very good (.80) parameters proposed by Kline (2010). The results obtained for Cronbach's alpha are very close when the Portuguese version (ECIP-2) is compared to the original one (PKBS-2). Additionally, in agreement with the pattern of PKBS-2's results (Merrell, 2002) and with CFA for the Social Skills scale (Major & Seabra-Santos, 2014a), the coefficients obtained from the family context are systematically lower than those obtained when the entire sample is taken into account, highlighting the superiority of internal consistency values for the information provided by teachers when compared to the other two samples (total and parents). When the internal consistency is compared between item-level data (items) and aggregate-level (parcels; Little et al., 2002), we verify that, despite the fact that parcels decrease the number of variables by approximately one third (46 items of the PB scale are reduced to 16 parcels), the values obtained are very similar, which favors the use of parcels in CFA when analyzing the scale's Portuguese version. The results concerning composite reliability also favor the fit of the factor structure under study.

This study presents a positive contribution, especially in regard to evidence of the good psychometric functioning of the factor structure under study for the PB scale of the PKBS-2's Portuguese version and its replication in two contexts of children's lives, using a large sample properly stratified and recruited from the entire country. There are, however, some limitations, such as the fact that this sample results from the inclusion of protocols collected in two different contexts, family and school, though this procedure was also implemented in the studies performed with the original version of the PKBS-2.

Future studies should replicate CFA while considering other variables, such as sex and age of children, or use other samples. Cross cultural studies comparing the factor structure obtained with other versions (e.g., German) or of this same version in other Portuguese speaking countries (e.g., Brazil) would represent an advantage in that it can enable analysis of the impact of cultural factors on the results.

The psychological assessment of preschoolers is not only of theoretical interest. On the contrary, it is an issue with strong practical implications, both in terms of research and clinical practice. In this sense, the availability of an assessment scale specifically developed to assess behavioral problems in preschoolers, as is the case with the PKBS-2, with evidence of validity (reinforced in this study), is a valid contribution to the early identification of problems among preschoolers for the implementation of early interventions and to attenuate potential difficulties in the future.

References

• Achenbach TM (1997). Caregiver-Teacher Report Form for Ages 11/2-5. Burlington VT: University of Vermont Research Center for Children Youth & Families.
• Achenbach TM, Edelbrock CS (1978). The classification of child psychopathology: A review and analysis of empirical efforts. Psychological Bulletin 85(6), 1275-1301. doi: 10.1037/0033-2909.85.6.1275
  » https://doi.org/10.1037/0033-2909.85.6.1275
• Achenbach TM, Rescorla LA (2000). Child Behavior Checklist for Ages 11/2-5. Burlington VT: University of Vermont Research Center for Children Youth & Families .
• Al Awmleh A, Woll A. (2013). Reliability of the German language version of the Preschool and Kindergarten Behavior Scales Second Edition. Journal of Social Sciences 9(2), 54-58. doi: 10.3844/jsssp.2013.54.58
  » https://doi.org/10.3844/jsssp.2013.54.58
• Bagner DM, Rodríguez GM, Blake CA, Linares D, Carter AS (2012). Assessment of behavioral and emotional problems in infancy: A systematic review. Clinical Child Family Psychology Review 15(2), 113-128. doi: 10.1007/s10567-012-0110-2
  » https://doi.org/10.1007/s10567-012-0110-2
• Basten M, Tiemeier H, Althoff RR, van de Schoot R, Jaddoe VWV, Hofman A, van der Ende J. (2016). The stability of problem behavior across the preschool years: An empirical approach in the general population. Journal of Abnormal Child Psychology 44(2), 393-404. doi: 10.1007/s10802-015-9993-y
  » https://doi.org/10.1007/s10802-015-9993-y
• Bornstein MH, Hahn C-S, Haynes OM (2010). Social competence externalizing and internalizing behavioral adjustment from early childhood through early adolescence: Developmental cascades. Development and Psychopathology 22(4), 717-735. doi: 10.1017/S0954579410000416
  » https://doi.org/10.1017/S0954579410000416
• Byrne BM (2010). Structural equation modeling with AMOS: Basic concepts applications and programming (2nd ed.). New York NY: Routledge.
• Campbell SB, Shaw DS, Gilliom M. (2000). Early externalizing behavior problems: Toddlers and preschoolers at risk for later maladjustment. Development and Psychopathology 12 (3), 467-488. doi: 10.1017/S0954579400003114
  » https://doi.org/10.1017/S0954579400003114
• Carney AG, Merrell KW (2002). Reliability and comparability of a Spanish-language form of the Preschool and Kindergarten Behavior Scales. Psychology in the Schools 39(4), 367-373. doi: 10.1002/pits.10033
  » https://doi.org/10.1002/pits.10033
• Cicchetti D, Toth SL (1991). A developmental perspective on internalizing and externalizing disorders. In Cicchetti D, Toth SL (Eds.), Internalizing and externalizing expressions of dysfunction: Rochester symposium on developmental psychopathology (Vol. 2, pp. 1-19). Hillsdale NJ: Lawrence Erlbaum.
• Fuchs S, Klein AM, Otto Y, von Klitzing K. (2013). Prevalence of emotional and behavioral symptoms and their impact on daily life activities in a community sample of 3 to 5-year-old children. Child Psychiatry and Human Development 44(4), 493-503. doi: 10.1007/s10578-012-0343-9
  » https://doi.org/10.1007/s10578-012-0343-9
• Goodman R. (2001). Psychometric properties of the Strenghts and Difficulties Questionnaire. Journal of the American Academy of Child and Adolescent Psychiatry 40(11), 1337-1345. doi: 10.1097/00004583-200111000-00015
  » https://doi.org/10.1097/00004583-200111000-00015
• Gresham FM, Elliott SN (2008). Social Skills Improvement System Rating Scales. Bloomington MN: Pearson Assessments.
• Hu L, Bentler PM (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling 6(1), 1-55. doi: 10.1080/10705519909540118
  » https://doi.org/10.1080/10705519909540118
• Jackson DL, Gillaspy JA Jr, Purc-Stephenson R. (2009). Reporting practices in confirmatory factor analysis: An overview and some recommendations. Psychological Methods 14(1), 6-23. doi: 10.1037/a0014694
  » https://doi.org/10.1037/a0014694
• Keenan K, Wakschlag LS (2002). Can a valid diagnosis of disruptive behavior disorder be made in preschool children? The American Journal of Psychiatry 159(3), 351-358. doi: 10.1176/appi.ajp.159.3.351
  » https://doi.org/10.1176/appi.ajp.159.3.351
• Kline RB (2010). Principles and practice of structural equation modeling (3rd ed.). New York NY: Guilford.
• Little TD, Cunningham WA, Shahar G, Widaman KF (2002). To parcel or not to parcel: Exploring the question weighing the merits. Structural Equation Modeling 9(2), 151-173. doi: 10.1207/S15328007SEM0902_1
  » https://doi.org/10.1207/S15328007SEM0902_1
• Major SO (2011). Avaliação de aptidões sociais e problemas de comportamento em idade pré-escolar: Retrato das crianças portuguesas [Assessment of social skills and problem behaviors in preschoolers: A picture of Portuguese children] (Doctoral dissertation). Retrieved from https://estudogeral.sib.uc.pt/bitstream/10316/17774/5/Tese_SofiaMajor.pdf
  » https://estudogeral.sib.uc.pt/bitstream/10316/17774/5/Tese_SofiaMajor.pdf
• Major S, Seabra-Santos MJ (2014a). Factor validation of the Portuguese version of the social skills scale from the Preschool and Kindergarten Behavior Scales. Paidéia (Ribeirão Preto), 24(58), 145-154. doi: 10.1590/1982-43272458201402
  » https://doi.org/10.1590/1982-43272458201402
• Major S, Seabra-Santos MJ (2014b). Preschool and Kindergarten Behavior Scales - Second Edition (PKBS-2): Adaptação e estudos psicométricos da versão portuguesa [Preschool and Kindergarten Behavior Scales - Second Edition (PKBS-2): Adaptation and psychometric studies of the Portuguese version]. Psicologia: Reflexão e Crítica 27(4), 689-699. doi: 10.1590/1678-7153.201427409
  » https://doi.org/10.1590/1678-7153.201427409
• Marôco J. (2010). Análise de equações estruturais: Fundamentos teóricos software & aplicações [Analysis of structural equations: Theoretical foundations software & applications]. Pêro Pinheiro Portugal: ReportNumber.
• Merrell KW (2002). Preschool and Kindergarten Behavior Scales - Second Edition. Austin TX: PRO-ED.
• Metcalfe LA, Harvey EA, Laws HB (2013). The longitudinal relation between academic/cognitive skills and externalizing behavior problems in preschool children. Journal of Educational Psychology 105(3), 881-894. doi: 10.1037/a0032624
  » https://doi.org/10.1037/a0032624
• Rescorla LA, Achenbach TM, Ivanova MY, Harder VS, Otten L, Bilenberg N, Verhulst FC (2011). International comparisons of behavioral and emotional problems in preschool children: Parents' reports from 24 societies. Journal of Clinical Child & Adolescent Psychology 40(3), 456-467. doi: 10.1080/15374416.2011.563472
  » https://doi.org/10.1080/15374416.2011.563472
• Reynolds CR, Kamphaus RW (2004). Behavior Assessment System for Children Second Edition (BASC-2). Circle Pines MN: American Guidance Service.
• Squires J, Bricker D, Twombley E. (2002). Ages & stages questionnaires: Social-emotional. Baltimore MD: Brookes.
• Studts CR, van Zyl MA (2013). Identification of developmentally appropriate screening items for disruptive behavior problems in preschoolers. Journal of Abnormal Child Psychology 41(6), 851-863. doi: 10.1007/s10802-013-9738-8
  » https://doi.org/10.1007/s10802-013-9738-8
• Upshur C, Wenz-Gross M, Reed G. (2013). A pilot study of a primary prevention curriculum to address preschool behavior problems. Journal of Primary Prevention 34(5), 309-327. doi: 10.1007/s10935-013-0316-1
  » https://doi.org/10.1007/s10935-013-0316-1
• Whitcomb SA, Merrell KW (2013). Behavioral social and emotional assessment of children and adolescents (4th ed.). New York NY: Routledge .

• 1
  Paper derived from the first author's doctoral dissertation under the second author and Dr. Roy P. Martin's supervision. Presented in 2011 at the Faculdade de Psicologia e de Ciências da Educação, Universidade de Coimbra, Portugal.
• Support: Doctoral scholarship provided by the Fundação para a Ciência e a Tecnologia, Ref.: SFRH/BD/29141/2006.

Publication Dates

• Publication in this collection
  Sep-Dec 2016

History

• Received
  07 June 2015
• Reviewed
  01 Dec 2015
• Accepted
  29 Jan 2016