Development and validation of a socio-emotional skills assessment instrument for sixth grade of Primary Education in Uruguay

PANIZZA, María Eugenia; CUEVASANTA, Diego; MELS, Cindy

doi:10.1590/1982-0275202037e190066

Acessibilidade / Reportar erro

Brasil

Estudos de Psicologia (Campinas)

Español English

Brasil

Español English

sumário « anterior atual seguinte »

Sumário

THEMATIC SECTION: SOCIO-EMOTIONAL COMPETENCES • Estud. psicol. 37 • 2020 • https://doi.org/10.1590/1982-0275202037e190066 copy

Development and validation of a socio-emotional skills assessment instrument for sixth grade of Primary Education in Uruguay

Desenvolvimento e validação de um instrumento para avaliação de habilidades socioemocionais no sexto ano do Ensino Fundamental no Uruguai

Authorship SCIMAGO INSTITUTIONS RANKINGS

Abstract

The development and implementation of instruments with high psychometric standards for the assessment and monitoring of socio-emotional skills is considered an important step in the agenda for their promotion in educational contexts. This article describes the process behind the development and validation of a large-scale assessment instrument of socio-emotional skills – including interpersonal and intrapersonal skills and aspects related to motivation and self-regulation –, for sixth grade primary school-children in Uruguay. We present results regarding factor structure and internal consistency obtained in throughout three studies. We conclude that the final instrument presents proper psychometric qualities, including evidence of convergent validity. Nonetheless, it is recognized that the development of assessment instruments of this type requires continued efforts for its improvement based on a longitudinal and iterative review.

Keywords
Educational measurement; Emotions; Validation studies

Learning Motivation and Self-Regulation: Skills focused on academic goals
Metacognitive self-regulation	Awareness and control of cognitive activities through planning, monitoring and continuous correction of cognitive activities during the execution of a task (based on Pintrich, Smith, García, & McKeachie, 1991Pintrich, P. R., Smith, D., García, T., & McKeachie, W. (1991). A manual for the use of the Motivated Strategies for Learning Questionnaire (MSLQ). Ann Arbor: National Center for Research to Improve Postsecondary Teaching and Learning.).
Growth mindset	The fundamental belief that our basic qualities, such as intelligence, can be cultivated through effort (based on Dweck, 2010Dweck, C. (2010). Mind-sets and equitable education. Principal Leadership, 10(5), 26-29.).
Intrinsic motivation	Participation in a task as an end in itself, for reasons such as defiance, curiosity, or mastery of a task or subject (based on Pintrich et al., 1991Pintrich, P. R., Smith, D., García, T., & McKeachie, W. (1991). A manual for the use of the Motivated Strategies for Learning Questionnaire (MSLQ). Ann Arbor: National Center for Research to Improve Postsecondary Teaching and Learning.).
Academic perseverance	Commitment to academic tasks, focus and persistence in the pursuit of academic goals, despite obstacles, difficulties and distractions (based on Farrington et al., 2012Farrington, C., Roderick, M., Allensworth, E., Nagaoka, E., Keyes, T., Johnson, D., & Beechum, N. O. (2012). Teaching adolescents to become learners: the role of noncognitive factors in shaping school performance: a critical literature review. Chicago: Consortium on Chicago School Research.).
Interpersonal Skills: Skills for constructive social interaction
Empathy	Ability to understand and share the emotional state of others and respond in a way compatible with it through perspective taking, recognition of emotions and their context (based on Dadds et al., 2008Dadds, M. R., Hunter, K., Hawes, D. J., Frost, A. D. J., Vassallo, S., Bunn, P., … Masry, Y. (2008). A measure of cognitive and affective empathy in children using parent ratings. Child Psychiatry and Human Development, 39(2), 111-122.).
Relationship Skills	Socially acceptable behavioral skills that allow effective interaction with others (based on Gresham & Elliot, 2007Gresham, F. M., & Elliot, S. (2007). Social skills rating system manual. Mineápolis: American Guidance Service.).
Intrapersonal Skills: Skills for managing one’s emotions and reactions
Emotional Regulation	It refers to cognitive strategies for managing internal emotional information and regulating emotional expression. Strategies refer to resources such as the ability to divert attention, take perspective or reformulate emotional reaction (based on Garnefski & Kraaij, 2007Garnefski, N., & Kraaij, V. (2007). Cognitive emotion regulation. psychometric features and prospective relationships with depression and anxiety in adults. European Journal of Psychological Assessment, 23(3), 141-149.).
Self Control	The ability to control impulsive reactions to both positive and negative situations in order to meet obligations and short-term goals. Therefore, it implies the ability to change responses and avoid undesirable behaviors in a given context (based on Bandy & Moore, 2010Bandy, T., & Moore, K. (2010). Assessing self-regulation: a guide for out-of-school time program practitioners. Child Trends, 23, 1-8.).

							Factorial load
Level 3	Level 2	Level 1	Item	n	M	SD	F1	F2	F3	F4	F5	F6	F7	F8
Habilidades socioemocionales (α = 0.76)	Motivation and Self-regulation (α = 0.60)	Metacognitive self-regulation (α = 0.74)	1	7029	2.96	0.95	0.58
			2	7029	2.95	0.96	0.51
			3	7029	3.28	0.85	0.57
			4	7029	3.48	0.77	0.68
			5	7029	3.46	0.76	0.66
			6	7029	3.43	0.76	0.58
		Growth mindset (α = 0.65)	7	7027	2.26	0.99		-0.78
			8	7027	2.30	0.96		-0.71
			9	7027	2.14	1.02		-0.79
		Intrinsic motivation (α = 0.65)	10	7034	3.50	0.75			-0.54
			11	7034	2.97	0.92			-0.71
			12	7034	2.69	0.99			-0.68
			13	7034	3.33	0.84			-0.54
			14	7034	3.58	0.69			-0.54
		Academic perseverance (α = 0.71)	15	7039	1.72	0.91				-0.73
			16	7039	1.56	0.82				-0.74
			17	7039	1.72	0.94				-0.65
			18	7039	2.01	0.96				-0.48
			19	7039	1.53	0.86				-0.48
	Interpersonal skills (α = 0.77)	Empathy (α = 0.70)	20	7033	3.38	0.78					-0.77
			21	7033	3.20	0.84					-0.72
			22	7033	3.37	0.84					-0.57
			23	7033	3.33	0.85					-0.41
			24	7033	3.33	0.77					-0.53
		Relationship Skills (α = 0.65)	25	7036	3.27	0.85						-0.66
			26	7036	3.33	0.85						-0.66
			27	7036	2.86	1.00						-0.60
			28	7036	2.89	0.94						-0.49
			29	7036	2.98	1.00						-0.33
			30	7036	2.53	1.12						-0.47
	Intrapersonal skills (α = 0.55)	Emotional Regulation (α = 0.67)	31	7031	2.73	1.01							0.63
			32	7031	2.77	0.98							0.49
			33	7031	2.94	1.06							0.67
			34	7031	2.71	1.00							0.65
			35	7031	2.99	1.02							0.75
		Self-control (α = 0.72)	36	7028	2.62	0.95								0.50
			37	7028	2.22	1.06								0.56
			38	7028	2.38	1.06								0.69
			39	7028	2.03	1.08								0.68
			40	7028	2.35	0.94								0.73
			41	7028	1.6	0.91				-0.42				0.37
		Eigenvalue					6.79	1.45	1.15	1.77	1.19	1.24	2.03	3.26
		% variance					16.55	3.53	2.80	4.33	2.90	3.01	4.96	7.94

Item nº	Item
1	I check my homework to make sure I did it right.
2	When I’m studying, I ask myself questions to know if I’m understanding correctly.
3	When I have to do some work, I get organized (for example: I look for the materials).
4	I read the questions on a test carefully before I start answering.
5	If I don’t understand something, I read it again more carefully until it becomes clear.
6	When I realize that something is going wrong in my work, I correct it.
7	If you are smart you don’t need to work hard to do well in school.
8	Some children are just born smart and do better in school.
9	A good student does not need to make an effort to do well in school.
10	I study to learn.
11	I like to study.
12	I like difficult tasks because they are challenging.
13	If I get excited about a subject, I want to continue learning beyond what the class learns.
14	I like to learn new things.
15	If a task is too difficult, I leave it undone.
16	I abandon tasks before finishing them.
17	If a task is too long, I just do the easy parts.
18	I forget to do my homework.
19	I lose things they lent me.
20	I can tell when a classmate feels sad.
21	I know when a classmate is upset, even when he says nothing.
22	It bothers me when they treat a classmate badly.
23	I am happy when others are happy.
24	I help my colleagues when they have a problem.
25	I make friends easily.
26	I feel part of a group because we do things together.
27	At recess, I invite other children to do things together.
28	When something happens that I can’t solve, I ask for help.
29	When someone gets mad at me, I talk to fix things.
30	I tell my friends my feelings.
31	When I’m angry I think about something else.
32	I can calm down when I’m nervous.
33	When I’m sad about something, I try to think of something happy.
34	When something makes me upset, I try to think it’s not so serious.
35	When I’m sad I try to distract myself.
36	I am easily distracted in class.
37	It’s hard for me to wait my turn.
38	I get upset easily.
39	I do things without thinking (like hitting, insulting) but then I regret it.
40	In some situations, I act without thinking.
41	If my work doesn’t turn out well, I get upset and stop doing it.

	1	2	3	4	5	6	7
1. Interpersonal skills	-	17^ p ≤ 0.01.	50^ p ≤ 0.01.	30^ p ≤ 0.01.	41^ p ≤ 0.01.	09^ p ≤ 0.01.	13^ p ≤ 0.01.
2. Intrapersonal skills		-	40^ p ≤ 0.01.	16^ p ≤ 0.01.	18^ p ≤ 0.01.	17^ p ≤ 0.01.	17^ p ≤ 0.01.
3. Motivation and self-regulation			-	34^ p ≤ 0.01.	27^ p ≤ 0.01.	18^ p ≤ 0.01.	18^ p ≤ 0.01.
4. Link with the teacher				-	42^ p ≤ 0.01.	08^ p ≤ 0.01.	08^ p ≤ 0.01.
5. Link between parties					-	16^ p ≤ 0.01.	14^ p ≤ 0.01.
6. Math Score						-	64^ p ≤ 0.01.
7. Reading Score							-

Programa de Pós-Graduação em Psicologia, Pontifícia Universidade Católica de Campinas Núcleo de Editoração SBI - Campus II, Av. John Boyd Dunlop, s/n. Prédio de Odontologia, 13060-900 Campinas - São Paulo Brasil, Tel./Fax: +55 19 3343-7223 - Campinas - SP - Brazil
E-mail: psychologicalstudies@puc-campinas.edu.br

Acompanhe os números deste periódico no seu leitor de RSS

[1] Correspondence to: M.E. PANIZZA. E-mail: mpanizza@ineed.edu.uy.

	1	2	3	4	5	6	7
1. Interpersonal skills	-	17^ p ≤ 0.01.	50^ p ≤ 0.01.	30^ p ≤ 0.01.	41^ p ≤ 0.01.	09^ p ≤ 0.01.	13^ p ≤ 0.01.
2. Intrapersonal skills		-	40^ p ≤ 0.01.	16^ p ≤ 0.01.	18^ p ≤ 0.01.	17^ p ≤ 0.01.	17^ p ≤ 0.01.
3. Motivation and self-regulation			-	34^ p ≤ 0.01.	27^ p ≤ 0.01.	18^ p ≤ 0.01.	18^ p ≤ 0.01.
4. Link with the teacher				-	42^ p ≤ 0.01.	08^ p ≤ 0.01.	08^ p ≤ 0.01.
5. Link between parties					-	16^ p ≤ 0.01.	14^ p ≤ 0.01.
6. Math Score						-	64^ p ≤ 0.01.
7. Reading Score							-