Two-Level Model of Attitudes and Beliefs Influencing Higher Order Thinking (HOT) Skills in Mathematics

Modelos de dos Niveles de Actitudes y Creencias que Influyen en Las Habilidades de Pensamiento de Orden Superior (HOT) en Matemáticas

Abstract

This article focuses on a two-level model analysis of attitudes and beliefs affecting students’ higher order thinking (HOT) skills in mathematics in Aceh, Indonesia. The data used are nested within the hierarchical ordering of both student (level 1) and teacher (level 2). The variables used at level 1 in the study include liking mathematics, valuing mathematics, confidence in mathematics, and individual judgement of mathematics ability, as well as beliefs concerning mathematics related to lower order thinking (LOT) and higher order thinking (HOT). The variables at level 2 involve beliefs concerning mathematics teaching related to LOT and beliefs concerning mathematics teaching related to HOT. The analysis reveals that there are four variables at level 1 contributing to student HOT skills in mathematics: liking mathematics, individual judgement of mathematics ability, beliefs concerning mathematics related to LOT, and beliefs concerning mathematics related to HOT. At level 2, the one variable affecting student HOT skills in mathematics is teacher beliefs concerning mathematics related to HOT.

Keywords:
Higher order thinking; Lower order thinking; Teacher beliefs; Hierarchical linear modelling

Resumen

Este artículo se centra en un modelo de dos niveles de análisis de actitudes y creencias que influyen en las habilidades de pensamiento de orden superior (HOT) de los estudiantes en matemáticas en Aceh, Indonesia. Los datos utilizados se anidan dentro del orden jerárquico de los estudiantes (nivel-1) y del docente (nivel-2). En el estudio, las variables utilizadas en el nivel 1incluyen las matemáticas, la valoración de las matemáticas, la confianza en las matemáticas y el juicio individual de las matemáticas, así como las creencias sobre las matemáticas de pensamiento de orden inferior (LOT) y el pensamiento de orden superior (HOT). En el nivel 2, las variables involucran creencias sobre la enseñanza de las matemáticas relacionadas con LOT y creencias relacionadas con la enseñanza de las matemáticas relacionadas con HOT. El análisis revela que hay cuatro variables en el nivel 1, HOT, habilidades en matemáticas, matemáticas, juicio individual de las habilidades matemáticas sobre matemáticas relacionadas a LOT y creencias relacionadas con las matemáticas relacionadas con HOT. En el nivel 2, la única variable afecta las habilidades HOT de los estudiantes en las creencias matemáticas sobre las matemáticas relacionadas con HOT.

Palabras-clave:
Pensamiento de orden superior; Pensamiento de orden inferior; Creencias del maestro Modelado lineal jerárquico

1 Introduction

Recent research in the field of mathematics education has highlighted the need to investigate belief and attitude constructs as well as the need to investigate how these contribute to the students’ mathematics performance. Beliefs play crucial roles in mathematics learning and the interactions between beliefs and performance have been analyzed (DI MARTINO; ZAN, 2011). Students’ beliefs may guide students’ activities in learning mathematics (LESTER JR, 2002). In addition, students’ beliefs towards mathematics have a meaningful impact on their attitudes toward mathematics (KLOOSTERMAN, 2002). Moreover, the importance of the attitudes towards mathematics learning is also widely recognized. Studies have been conducted to determine whether there is a causal relationship between students’ positive attitudes and their mathematics performance (DI MARTINO; ZAN, 2011), with a positive attitude being associated with a positive mathematics performance. It has been seen that a negative attitude toward mathematics hinders a positive self-concept of mathematics ability (HANNULA, 2002). Furthermore, there is also an argument that teacher beliefs are correlated to the students’ beliefs (ROESKEN; PEPIN; TOERNER, 2011). Therefore, it is crucial to study the students’ beliefs in relation to their attitude towards mathematics and mathematics performance while taking into account the roles of the teachers’ beliefs.

While there are some clear definitions of attitudes, beliefs have not been so clearly defined. Ernest (1989) defined attitudes to mathematics as a form of liking, enjoying and being interested in mathematics or, negatively, as an anxiety towards mathematics. These attitudes could also involve a student's confidence of their mathematical ability which in turn reflected on their self-concept and valuing of mathematics (ERNEST, 1989). Students’ attitudes toward mathematics are correlated to students’ mathematics performance. This hypothesized causal relationship between attitudes and mathematics performance has been established in a meta-analysis review conducted by Ma and Kishor (1997), with the effect size that is statistically reliable (0.23). Borg (2001) defined a belief as ‘a mental state which had as its content a proposition that is accepted as true by the individual holding it, although the individual might recognize that alternative beliefs might be held by others’. Beliefs were also seen as the result of evaluation and judgement (PAJARES, 1992). Teachers’ beliefs played a vital role in shaping the classroom practice likely to be consistent with teaching and learning as well as students’ beliefs and performance (ERNEST, 1989; NESPOR, 1987). Furthermore, beliefs dictated one's ‘thinking and action’ (BORG, 2001). Thus, attitudes toward mathematics could be seen as one's negative or positive responses to mathematics and mathematics learning. On the other hand, beliefs are one's judgement of a particular subject which guides further action.

Higher order thinking (HOT) skills have been mentioned in the earlier learning theories implicitly (BIGGE; SHERMIS, 1992; BIGGS; MOORE, 1993) The skills were described indirectly in Piaget's stage of formal operation (BIGGS; MOORE, 1993), where adolescents were able to analyze their thought and develop their ideas by employing reasoning skills. Furthermore, they were also found in Bruner's final cognitive growth, the symbolic mode. The symbolic mode referred is the situation where adolescents grasped abstract concepts through symbols, using language as the medium of thought (BIGGE; SHERMIS, 1992). HOT skills were best described using Bloom's taxonomy as involving applying, analyzing, evaluating and creating skills (ANDERSON; SOSNIAK, 1994; PEGG, 2010). These skills are clearly seen at the relational and extended abstract level in the SOLO taxonomy (PEGG, 2010). Higher order thinking skills could also be seen as the ability of students to think mathematically when solving problems (STAPLES; TRUXAW, 2010). This was in line with a review of higher order thinking skills in three countries by Fullan and Watson (2011). It was mentioned that higher order thinking skills in mathematics emerge in problem-solving skills, communicating the solution mathematically. It can be argued that higher order thinking skills in mathematics involves two important aspects: reasoning skills and problem-solving skills (utilizing analyzing, evaluating skills). Equipping students with both skills benefited them in meeting the challenge of a dynamic and innovative world (FORSTER, 2004). Therefore, it is essential to promote HOT skills and investigate variables which can give positive impacts on the development of the skills in the mathematics classroom.

While most studies have been carried out concerning the relationship between beliefs and attitudes and their relationship with the student's mathematics performance, they mainly focus on investigating the relationships of beliefs and attitudes at only one level, either student or teacher. This method has a limitation. In an educational setting, data are within the hierarchical structure. Students, for example, are situated within the hierarchical structure of classroom and school. Thus, the samples are not fully independent, as hierarchical structure data tend to be more homogenous (OSBORNE, 2000). Thus, they should not be analyzed independently at one level; preferably, a true multilevel analysis, hierarchical linear modelling (HLM), should be employed where levels 1 and 2 are specified, respectively. There has been limited study investigating the relationships of the constructs at both student and teacher levels. Moreover, there has been limited study examining the relationship between these constructs and students’ performance related to higher order thinking skill in mathematics. Thus, one intention of this article is to bridge this gap and investigate how students’ beliefs and attitudes as well as teachers’ beliefs contribute to students’ mathematics performance, specifically their performance related to higher order thinking skills. It aims at developing a broader understanding of the relationship between teachers’ beliefs and students’ beliefs and attitude and how these influence students’ mathematics performance through a two-level analysis. It can contribute specifically to the body of knowledge of students’ and teachers’ beliefs and attitudes concerning mathematics in the Asian context, more especially, in Indonesia.

2 Method

2.1 Variables Used

The variables analyzed at student level (level 1) are (a) liking mathematics (LIKE MATH); (b) valuing mathematics (VALUE MATH); (c) confidence in learning mathematics (MATH CONF); (d) individual judgement of mathematics ability (IND JUD); (e) beliefs concerning mathematics related to HOT (SBM H) and (f) beliefs concerning mathematics related to LOT (SBM L). The variables at teacher level (level 2) are: (a) beliefs concerning mathematics teaching related to HOT (TBMT H); and (b) beliefs concerning mathematics teaching related to LOT (TBMT L). The outcome variable is mathematics performance related to HOT (MATH HOT). In Figure 1, there is a conceptual diagram of a two-level model of attitudes and beliefs influencing mathematics performance related to HOT.

Figure 1
Conceptual model of two-level model of attitudes and beliefs influencing mathematics performance related to HOT

2.2 Participants, datasets, and data analysis

The datasets used in this study are based on the student and teacher data of a larger research investigating on higher order thinking skills, examining the nexus between teachers’ beliefs, classroom practices, students’ beliefs, and mathematics performance. The participants in this study are 1135 9^th Grade students and 46 mathematics teachers from 25 schools in two districts in the province of Aceh, Indonesia. The mathematics teachers selected are those who teach in 9^th Grade at the respective school. The 25 schools are selected using a stratified purposive sampling method from the total of 114 Junior High Schools located in the two districts in the province of Aceh. The schools are selected from one city to represent the urban area and one district to represent the rural area. The instruments used in the study include a student, teacher, and school questionnaire as well as mathematics test for students. All the sets of questionnaires were self-administered. The actual dataset included students’ and teachers’ demographic data, their attitudes and beliefs as well as data related to classroom practice. The mathematics test consisted of open-ended questions related to HOT and LOT. However, in this analysis, we used only measures related to attitudes and beliefs from student and teacher questionnaires together with mathematics performance related to HOT.

The student data is nested within the teacher data. When data is of a nested nature, it is likely that the relationships between variables do not occur simply at one hierarchical level but between the various hierarchical levels (HOFMANN, 1997). When nested data is mishandled, incorrect conclusions concerning the phenomena may easily occur (SNIJDERS, 1999). To obtain a better understanding of the relationship within and between the hierarchical levels, we employed a hierarchical linear modeling (HLM) analysis. HLM is a statistical technique that enables the simultaneous examination of the relationships at a single level as well as across the levels. The requirements of HLM analysis are hierarchically structured data (e.g., the first level data nested within the second level) and the variables in the model considered having a hierarchical linear structure (RAUDENBUSH, 1993). In this study, a two-level HLM analysis is conducted using the HLM program version 6.08 (RAUDENBUSH; BRYK; CONGDON, 2004).

Prior to the data analysis, the variables used in the study were validated using a Confirmatory Factor Analysis (CFA) and verified using a Rasch scaling analysis. The factor loading of all items were above the acceptable cut-off point. “Factor loadings ±.50 or greater are considered significant and factor loadings in the range of ±.30 to ±.40 are considered to meet the minimal level for interpretation of effect” (HAIR; BLACK; BABIN; ANDERSON, 2014, p.115). The reliability estimates were obtained from item separation reliability and person separation reliability of the Rasch scaling analysis. All variables also have acceptable reliability estimates. The estimates are within a satisfactory range of a required benchmark of 0.7 (JÄHNIG, 2013). There are two stages of conducting an HLM analysis. The first stage employs the fully unconditional or null model of a two-level model. This is the simplest model and consists of no explanatory variables at any level, allowing the calculation of available variance in an outcome measure across the two-level model of student and teacher. The second stage of a two-level HLM analysis specifies the conditional model, allowing the examination of which variables at each level can explain the variability obtained across different levels of the unconditional model (BRYK; RAUDENBUSH, 2002). The results of the dataset HLM analysis are presented and discussed accordingly.

3 Results

Hierarchical linear modelling (HLM) is employed to statistically analyze a data structure where students (level 1) are nested within teachers (level 2). Of specific interest is the relationship of mathematics performance related to HOT (level 1 outcome variable) and students’ attitudes and beliefs concerning mathematics (level 1 variables), and their teachers’ beliefs concerning mathematics (level 2 variables). Model testing proceeds in three phases: fully unconditional model (null model), final level 1 model and full model. The outcome variable is mathematics performance related to HOT (MATH HOT).

The null model reveals an interclass correlation (ICC) of 0.51. Thus 51% of the variance in mathematics performance related to HOT is between teachers and 49% of the variance in mathematics performance related to HOT is between students within a given teacher. The null model is summarized in Table 1.

Table 1
Null model of mathematics performance related to HOT

Final estimation of fixed effects (with robust standard errors)
Fixed effect	Coefficient	Standard error	T-ratio	Approx. d.f	P-value
For INTRCPT1, B0
INTRCPT2, G00	-0.10	0.11	-0.92	40	0.37
Final estimation variance components
Random effect	Standard deviation	Variance component	Df	Chi-square	P-value
INTRCPT1, U0	0.72	0.51	40	1235.50	0
level 1, R	0.70	0.49
Statistics for current covariance components model
Deviance					2553.80
Number of estimated parameters					2

Source: Research data

As the variance in mathematics performance related to HOT exists at both levels of the data structure, explanatory variables are individually added at each level. The final level 1 model is tested and there are four variables which are statistically significant: individual judgement of mathematics ability (IND JUD), liking mathematics (LIKE MATH), belief concerning mathematics related to HOT (SBM H) and belief concerning mathematics related to LOT (SBM L) (b = 0.04, p < 0.01; b = 0.05, p < 0.01; b = 0.08, p < 0.01 and b = – 0.14, p < 0.01, respectively). The final level one model is summarized in Table 2.

Table 2
Final level one model of mathematics performance related to HOT

Final estimation of fixed effects (with robust standard errors)
Fixed effect	Coefficient	Standard error	T-ratio	Approx. d.f	P-value
For INTRCPT1, B0
INTRCPT2, G00	-0.10	0.11	-0.96	40	0.34
For IND JUD slope, B1*
INTRCPT2, G10	0.04	0.02	2.91	40	0.01
For LIKE MATH slope, B2*
INTRCPT2, G20	0.05	0.02	3.16	40	0.00
For SBM H slope, B3*
INTRCPT2, G30	0.08	0.02	3.39	40	0.00
For SBM H slope, B4*
INTRCPT2, G40	-0.14	0.02	-7.17	40	0.00
Final estimation variance components
Random effect	Standard deviation	Variance component	Df	Chi-square	P-value
INTRCPT1, U0	0.67	0.45	40	613.61	0.00
IND JUD slope, U1*	0.04	0.00	40	39.15	>.500
LIKE MATH slope, U2*	0.06	0.00	40	51.36	0.11
SBM H slope, U3*	0.08	0.01	40	51.34	0.11
SBM L slope, U4*	0.07	0.00	40	28.73	>.500
level 1, R	0.65	0.43
Statistics for current covariance components model
Deviance			2446.13
Number of estimated parameters			16

• ^*
  LIKE MATH (liking mathematics); VALUE MATH (valuing mathematics); MATH CONF (confidence in learning mathematics); IND JUD (individual judgement of mathematics ability); SBM H (beliefs concerning mathematics related to HOT): SBM L (beliefs concerning mathematics related to LOT); TBMT H (beliefs concerning mathematics teaching related to HOT); TBMT L (beliefs concerning mathematics teaching related to LOT).
Source: Research data

The result of the final level 1 model shows that students’ mathematics performance is higher when students have judged their mathematics ability more positively. Students’ performance is also higher when they like mathematics more and when they have a more positive belief concerning mathematics related to HOT. However, students’ mathematics performance is lower when they have more positive beliefs concerning mathematics related to LOT.

The final model is tested by adding each level 2 predictor variable indicating that teacher's beliefs concerning mathematics teaching related to HOT is positive and significant (b = 0.16, p < 0.01). The final model is summarized in Table 3. This means students’ mathematics performance is higher when their teachers have more positive beliefs concerning mathematics teaching related to HOT. There is no significant cross-level interaction between level 1 and level 2 predictor variables, which means the degree of teacher beliefs has no influence on the strength of the relationship between the level 1 predictor variables and mathematics performance related to HOT. The proportions of variance explained by the final two-level model are 12% at level 1 and 12% at level 2. The illustration of the final model is presented on Figure 2.

Table 3
Final model of mathematics performance related to HOT

Final estimation of fixed effects (with robust standard errors)
Fixed effect	Coefficient	Standard error	Ratio	Approx. d.f	P-value
For INTRCPT1, B0
INTRCPT2, G00	-0.10	0.10	-1.08	39	0.29
TBMT H, G01*	0.16	0.05	2.94	39	0.01
For IND JUD slope, B1*
INTRCPT2, G10	0.04	0.01	2.96	40	0.01
For LIKE MATH slope, B2*
INTRCPT2, G20	0.05	0.02	3.04	40	0.01
For SBM H slope, B3*
INTRCPT2, G30	0.08	0.02	3.38	40	0.00
For SBM L slope, B4*
INTRCPT2, G40	-0.14	0.02	-7.259	40	0.00
Final estimation variance components
Random effect	Standard deviation	Variance component	Df	Chi-square	P-value
INTRCPT1, U0	0.61	0.37	39	541.41	0.00
IND JUD slope, U1*	0.04	0.00	40	39.17	>.500
LIKE MATH slope, U2*	0.06	0.00	40	51.35	0.11
SBM H slope, U3*	0.08	0.01	40	51.38	0.11
SBM L slope, U4*	0.07	0.00	40	28.64	>.500
level 1, R	0.65	0.43
Statistics for current covariance components model
Deviance			2440.32
Number of estimated parameters			16

• ^*
  LIKE MATH (liking mathematics); VALUE MATH (valuing mathematics); MATH CONF (confidence in learning mathematics); IND JUD (individual judgement of mathematics ability); SBM H (beliefs concerning mathematics related to HOT): SBM L (beliefs concerning mathematics related to LOT); TBMT H (beliefs concerning mathematics teaching related to HOT); TBMT L (beliefs concerning mathematics teaching related to LOT).
Source: Research data

Figure 2
Final model of two-level model of attitudes and beliefs influencing mathematics performance related to HOT

3 Discussion

The hierarchical linear analysis for examining the relationships of mathematics performance related to HOT (level 1 outcome variable) and students’ attitudes and beliefs concerning mathematics (level 1 variables), and their teachers’ beliefs concerning mathematics (level 2 variables) indicated that students’ mathematics performance related to HOT influenced by students’ attitude concerning mathematics (liking mathematics and individual judgement of mathematics ability). This result is promising for future Indonesian students’ mathematics performance in HOT as previous research has reported that Indonesian students’ attitude towards mathematics is positive (CHARLES; HARR; CECH; HENDLEY, 2014; SUPRAPTO, 2016). This is also in line with the finding in Thien, Darmawan and Ong (2015), in which attitude concerning mathematics being the predictor of the Indonesian mathematics performance. However, studies also reported that there is a decreasing trend of students’ attitude toward mathematics as they reach a higher level of education (DEIESO; FRASER, 2019; WIJSMAN; WARRENS; SAAB; VAN DRIEL; WESTENBERG, 2016). As attitude and performance is interrelated in many studies (HANNULA, 2019; WIJSMAN et al., 2016), the decreasing pattern of students’ attitude in a higher level of education is in line with the declining pattern of students’ mathematics performance (WIJSMAN et al., 2016). This may be due to the learning students experienced in the higher level of schooling is less interested (MIRZA; HUSSAIN, 2018). Thus, it is a challenge for Indonesian mathematics teachers to keep the students’ attitude remain high in each level of schooling by creating an interesting lesson.

Another finding also revealed that the students’ performance was higher when they had a more positive belief concerning mathematics related to HOT and was lower when they had more positive beliefs concerning mathematics related to LOT. These findings are consistent with those found by Schommer‐Aikins, Duell and Hutter (2005) reported that the students’ epistemological beliefs of mathematics are one of the predictors of their mathematics achievement. Even though their research did not specifically examine the higher and lower order thinking skills in mathematics rather problem-solving in mathematics, the findings are still relevant for problem-solving skills also promoting HOT. The findings of this study and the previous study shows the potential influence of beliefs for the student performance and more investigation on how to strengthen the beliefs having positive impact to the performance is required.

Teacher's beliefs concerning mathematics teaching related to HOT is positive and significant (b = 0.16, p < 0.01). The final model is summarized in Table 3. This means students’ mathematics performance is higher when their teachers have more positive beliefs concerning mathematics teaching related to HOT. This in line with Ertmer (2005) and Spruce and Bol (2015) who emphasized that teachers’ beliefs have an influence on their classroom practice, which in turn will lead to impact the students’ performance. This finding implies the concern for ensuring teachers to have favorable beliefs toward their classroom practices as its impact for the students’ achievement is unavoidable.

4 Conclusion

A two-level model of students’ mathematics performance related to HOT was analyzed. There are three explanatory variables that are positively significant (which are LIKE MATH, IND JUD and SBM H) and one predictor negatively significant (which is SBM L) at level 1 (student). Also, there is one variable that is positively significant (which is TBMT H) at level 2 (teacher). The results show that students’ attitudes and beliefs as well as teachers’ beliefs influence students’ mathematics performance related to HOT. The results can be interpreted as meaning that students who like mathematics more, have a more positive judgement of their mathematics ability and have more positive beliefs related to HOT that are more likely to have higher mathematics performance related to HOT. Conversely, students having more positive beliefs concerning mathematics related to LOT are less likely to have higher mathematics performance. Furthermore, the teachers’ beliefs also contribute to students’ performance as students whose teachers have a more positive belief concerning mathematics related to HOT are more likely to have a higher mathematics performance related to HOT.

References

• ANDERSON, L.; SOSNIAK, L. A. (Eds.). Bloom's taxonomy: A forty year retrospective. Chicago: University of Chicago Press, 1994.
• BIGGE, M. L.; SHERMIS, S. S. Learning theories for teachers New York: HarperCollins, 1992.
• BIGGS, J. B.; MOORE, P. J. The process of learning Melbourne: Prentice Hall, 1993.
• BORG, M. Teachers’ beliefs. ELT journal, Oxford, v. 55, n. 2, p. 186-188, 2001.
• BRYK, A. S.; RAUDENBUSH, S. W. Hierarchical linear models: Applications and data analysis methods. Newbury Park: SAGE, 2002. v. 1.
• CHARLES, M.; HARR, B.; CECH, E.; HENDLEY, A. Who likes math where? Gender differences in eighth-graders’ attitudes around the world. International Studies in Sociology of Education, Oxfordshire, v. 24, n. 1, p. 85-112, 2014.
• DEIESO, D.; FRASER, B. J. Learning environment, attitudes and anxiety across the transition from primary to secondary school mathematics. Learning Environments Research, New York, v. 22, n. 1, p. 133-152, 2019.
• DI MARTINO, P.; ZAN, R. Attitude towards mathematics: a bridge between beliefs and emotions. ZDM, New York, v. 43, n. 4, p. 471-482, 2011.
• ERNEST, P. The knowledge, beliefs and attitudes of the mathematics teacher: A model. Journal of Education for Teaching, Oxfordshire, v. 15, n. 1, p. 13-33, 1989.
• ERTMER, P. A. Teacher pedagogical beliefs: The final frontier in our quest for technology integration? Educational technology research and development, Boston, v. 53, n. 4, p. 25-39, 2005.
• FORSTER, M. Higher order thinking skills. Research Developments, Victoria, v. 11, n. 11, p. 1-6, 2004.
• FULLAN, M.; WATSON, N. The Slow Road to Higher Order Skills 2011. Available at: teacher.righthere.com.cn/UEditor/net/upload/file/20150409/6356419021493800007755897.pdf Access in: 20 June. 2020.
  » teacher.righthere.com.cn/UEditor/net/upload/file/20150409/6356419021493800007755897.pdf
• HAIR, J. F.; BLACK, W. C.; BABIN, B. J.; ANDERSON, R. E. Multivariate Data Analysis-Pearson New International Edition. New Jersey: Pearson, 2014.
• HANNULA, M. S. Attitude towards mathematics: Emotions, expectations and values. Educational studies in mathematics, New York, v. 49, n. 1, p. 25-46, 2002.
• HANNULA, M. S. Young learners’ mathematics-related affect: A commentary on concepts, methods, and developmental trends. Educational studies in mathematics, New York, v. 100, n. 3, p. 309-316, 2019.
• HOFMANN, D. A. An overview of the logic and rationale of hierarchical linear models. Journal of management, California, v. 23, n. 6, p. 723-744, 1997.
• JÄHNIG, C. C. Assessing Business Knowledge of Students in German Higher Education. In: JAHRBUCH DER BERUFS-UND WIRTSCHAFTSPÄDAGOGISCHEN FORSCHUNG, 2013, Opladen. Proceedings […] Opladen: Verlag Barbara Budrich, 2013. p. 47-59.
• KLOOSTERMAN, P. Beliefs about mathematics and mathematics learning in the secondary school: Measurement and implications for motivation. In: LEDER, G. C.; PEHKONEN, E.; TORNER, G. (Eds.). Beliefs: A Hidden Variable in Mathematics Education? Dordrecht: Kluwer Academic Publishers, 2002. p. 247-270.
• LESTER JR, F. K. Implications of research on students’ beliefs for classroom practice. In: LEDER, G. C.; PEHKONEN, E.; TORNER, G. (Eds.). Beliefs: A Hidden Variable in Mathematics Education? City: Springer, 2002. p. 345-353.
• MA, X.; KISHOR, N. Attitude toward self, social factors, and achievement in mathematics: A meta-analytic review. Educational Psychology Review, New York, v. 9, n. 2, p. 89-120, 1997.
• MIRZA, A.; HUSSAIN, N. Performing Below the Targeted Level: An Investigation into KS3 Pupils’ Attitudes Towards Mathematics. Journal of Education and Educational Development, Amsterdams, v. 5, n. 1, p. 8-24, 2018.
• NESPOR, J. The role of beliefs in the practice of teaching. Journal of curriculum studies, Oxfordshire, v. 19, n. 4, p. 317-328, 1987.
• OSBORNE, J. W. Advantages of hierarchical linear modeling. Practical Assessment, Research, and Evaluation, Massachusetts, v. 7, n. 1, p. 1-4, 2002.
• PAJARES, M. F. Teachers’ beliefs and educational research: Cleaning up a messy construct. Review of educational research, California, v. 62, n. 3, p. 307-332, 1992.
• PEGG, J. Promoting the acquisition of higher order skills and understandings in primary and secondary mathematics. In: TEACHING MATHEMATICS? MAKE IT COUNT: WHAT RESEARCH TELLS US ABOUT EFFECTIVE TEACHING AND LEARNING OF MATHEMATICS, 2010, Melbourne. Proceedings […] Melbourne: ACER, 2010. p. 35-38.
• RAUDENBUSH, S. W. Hierarchical linear models and experimental design. In: EDWARDS, L. K. (Ed.). Applied analysis of variance in behavioral science New York: Marrel Dekker, 1993. p. 459-496. v. 137.
• RAUDENBUSH, S. W.; BRYK, A. S.; CONGDON, R. HLM 6 for Windows Lincolnwood: Scientific Software International, p. 2004.
• ROESKEN, B.; PEPIN, B.; TOERNER, G. Beliefs and beyond: affect and the teaching and learning of mathematics. ZDM, New York, v. 43, n. 4, p. 451-455, 2011.
• SCHOMMER‐AIKINS, M.; DUELL, O. K.; HUTTER, R. Epistemological beliefs, mathematical problem‐solving beliefs, and academic performance of middle school students. The elementary school journal, Chicago, v. 105, n. 3, p. 289-304, 2005.
• SNIJDERS, T. A. Multilevel analysis Great Britain: Springer, 1999.
• SPRUCE, R.; BOL, L. Teacher beliefs, knowledge, and practice of self-regulated learning. Metacognition and Learning, New York, v. 10, n. 2, p. 245-277, 2015.
• STAPLES, M. E.; TRUXAW, M. P. The mathematics learning Discourse project: fostering higher order thinking and academic language in urban mathematics classrooms. Journal of Urban Mathematics Education, Texas, v. 3, n. 1, p. 27-56, 2010.
• SUPRAPTO, N. Students’ attitudes towards STEM education: Voices from indonesian junior high schools. Journal of Turkish Science Education, Trabzon, v. 13, n. special, p. 75-87, 2016.
• THIEN, L. M.; DARMAWAN, I. G. N.; ONG, M. Y. Affective characteristics and mathematics performance in Indonesia, Malaysia, and Thailand: what can PISA 2012 data tell us? Large-scale Assessments in Education, New York, v. 3, n. 1, p. 1-16, 2015.
• WIJSMAN, L. A.; WARRENS, M. J.; SAAB, N.; VAN DRIEL, J. H.; WESTENBERG, P. M. Declining trends in student performance in lower secondary education. European Journal of Psychology of Education, Heidelberg, v. 31, n. 4, p. 595-612, 2016.

Publication Dates

• Publication in this collection
  04 Aug 2021
• Date of issue
  Aug 2021

History

• Received
  09 July 2020
• Accepted
  30 Jan 2021