Meta-research in the field of education policy: conceptual and methodological elements

ABSTRACT

The paper discusses conceptual and methodological elements of meta-research (research about research) in the field of Education Policy. From the epistemological approach of Education Policy, a methodological proposal for meta-research on Education Policy is presented. It argues that meta-research contributes do the understanding of the current situation of research in the field, supporting both for the improvement of research and the researcher training.

Keywords:
Meta-research; Education Policy; Epistemology.

RESUMO

O artigo discute aspectos conceituais e metodológicos da metapesquisa (pesquisa sobre pesquisas) no campo da Política Educacional. A partir do enfoque das epistemologias da Política Educacional (EEPE) apresenta-se uma proposta metodológica para a realização de metapesquisa em Política Educacional. Argumenta-se que a metapesquisa contribui para a compreensão da situação da pesquisa do campo, oferecendo elementos tanto para a melhoria das pesquisas quanto para o processo de formação de pesquisadores.

Palavras-chave:
Metapesquisa; Política Educacional; Epistemologia.

RESUMEN

El artículo discute aspectos conceptuales y metodológicos de la meta-investigación en el campo de la Política Educativa. A partir del enfoque de las epistemologías de la Política Educativa (EEPE) se presenta una propuesta metodológica para la realización de la meta-investigación en la Política Educativa. Se argumenta que la meta-investigación contribuye a la comprensión de la situación de la investigación, ofreciendo elementos tanto para la mejora de las investigaciones como para el proceso de formación de investigadores.

Palabras clave:
Meta-investigación; Política Educativa; Epistemología.

Introduction

The purpose of this paper is to present conceptual and methodological aspects of meta-research in the field of Education Policy. In this text, we consider meta-research (research on research) as a strategy for the systematic analysis of research in a given field or topic. Meta-research differs from the studies of literature review, state of the art, state of knowledge, systematic review, meta-analysis, meta-synthesis, among other terms that are used in literature on research methodology.

The interest in exploring this theme arose within the scope of the Red de Estudios Teóricos y Epistemológicos en Política Educativa (ReLePe), created in 2010 by Brazilian and Argentinian researchers, with the objective of stimulating the studies on theoretical-epistemological frameworks of research on Education Policy¹⁰. After different attempts to create theoretical categories for the epistemological analysis of texts on Education Policy, Tello (2012) developed the category ‘epistemology approach of Education Policy’ (EAEP), which covers three aspects: epistemological perspective, epistemological positioning and epistemethodological approach.

It is a well-founded and suitable category for the analysis of texts on Education Policy (Tello & Mainardes, 2012, 2015b; Mainardes, 2017, 2018), which has been expanded with the definition of other concepts and categories, for instance, the discussions on pluralism as an epistemological perspective (Tello & Mainardes, 2015a, Mainardes, 2018) and the definition of the levels of approach/abstraction: description, analysis and understanding (Mainardes & Tello, 2016).

When developing the EAEP category, Tello (2012) pointed out that the aspects of the EAEP can be used by the researcher as epistemological surveillance¹¹ and can be applied in the analysis of Education Policy research (meta-research). In the initial work on the analysis of research in the field of Education Policy, in an epistemological perspective, Tello (2012) used the term meta-analysis. However, as the studies were being developed, it was concluded that the term meta-analysis was not the most appropriate, because it is a procedure that aims to aggregate results of empirical research, comparing them by means of statistical analysis. In the following section, we present the meta-research conceptualization.

Meta-research conceptualization

In the same way that the prefix meta is used for metacognition, metalinguistics and metatheory, meta-research can be conceptualized as research on research or research that seeks to explain the research process on a topic or a specific area or field (Rosenbaum & Langhinrichsen-Rohling, 2006).

In literature on procedures and strategies for the development and presentation of research synthesis, there is a diversity of terms. Zhao (1991) defines the terms meta-study, metatheory, metamethod and meta-data-analysis. Finfgeld (2003) relates the terms: meta-analysis, meta-study, meta-data-analysis, metamethod, metatheory and meta-synthesis.¹² These strategies aim to examine a set of studies, with different purposes: analysis of methods, theories, results, and so on.

In English language literature, most authors use the terms meta-research and meta-study as synonyms, but there are authors who distinguish meta-research from meta-study, arguing that meta-research has a specific focus (tapered), whereas meta-study is broader and encompasses a variety of aspects to be identified and analyzed in the texts. In the Portuguese language, metapesquisa (meta-research) and metaestudo (meta-study) are most often used with the same meaning.

Researchers in the area of Biological Sciences and Health have defined meta-research as an evolving scientific discipline, which aims to evaluate and improve the efficiency of research practices in order to generate more reliable and useful research results. It includes the analysis of methods, ways of presenting data, reproducibility, evaluation and incentives (how to do it, report it, verify it, correct it and reward science). Interdisciplinary approaches are used to study, promote and defend a more robust science committed to human progress (Finfgeld, 2003; Ioannidis, 2018; Ioannidis, Fanelli, Dunne, & Goodman, 2015).

In the case of Human and Social Sciences, meta-research can be used to carry out an evaluation of the research, identify characteristics, tendencies, weaknesses and obstacles for the development of a field or research topic. In general, it covers the analysis of theoretical-epistemological aspects, methodologies, styles of argumentation, level of internal coherence, ethical reflexivity, and so on.

Meta-research differs from literature review, systematic review, state of the art, state of knowledge. Meta-research is discipline-oriented (area or field) and is engaged with the research advances in the discipline (area or field). The review studies (literature review, systematic review, state of knowledge, state of the art) are oriented to research projects: previous research is mainly reviewed as a preparatory phase for the realization of new research projects. It is through literature review that the research acquaints itself with the knowledge already built on the research topic and identifies possible gaps that need to be filled in new research projects. Generally, literature review works are more concerned with synthesizing the results of a set of research, giving less attention to the theoretical foundations of the revised research. In turn, Meta-research seeks to analyze the theoretical foundations of the research and their meaning in the theoretical development of the field in which the research is part.

Zhao (1991) presents a very relevant synthesis on the ‘meta-study’ (called here as ‘meta-research’) (Figure 2). To the author, meta-study is a broader strategy and may encompass metatheory, meta-data-analysis and metamethod, depending on the focus set for the research.

FIGURE 2
RELATIONSHIP BETWEEN PRIMARY STUDY AND META-STUDY

In Figure 2, the primary data are those that had not been collected before, that is, it is the original data collection carried out by the researcher. When the researcher analyzes a set of already completed research, he/she performs the so-called meta-data-analysis. When he/she evaluates the methods employed, he/she performs the metamethod, and, finally, when analyzing the theories employed, he/she performs metatheory.

Zhao (1991) explains that ‘research on research’ is carried out with the purpose of re-studying the same phenomenon that has already been studied (replication, for example) or to study the results and processes (theories, methods) of studies already carried out. The latter type designates ‘meta-study’, which can be considered a ‘second-order study’. The first-order study analyzes a given phenomenon (real world) and the second-order study analyzes the studies already performed. Meta-study transcends or goes beyond previous studies and seeks not only to synthesize results but also to reflect on the processes involved in previous studies, in terms of ‘where we were and where we are going’ (Zhao, 1991, p. 378). However, Zhao also points out that there are authors who criticize meta-studies, considering them as second-order or carried out by scientists who are unable to generate real contributions to the discipline, and, to accomplish that, they dedicate themselves to analyze the research of others. Despite this, Zhao (1991) emphasizes that meta-study is an instrument for dealing with certain disciplinary problems, and in some cases becomes inevitable.

To Zhao (1991), there are at least two conditions that require the performance of meta-study. The first involves the sudden occurrence of some fundamental changes in the conception of a discipline (area), which consequently, demands a revision of the logic of scientific research prevailing in the discipline. Thus, we need such research to re-map the cognitive status of a changing discipline. The author explains that the revolution in Modern Physics caused by Einstein’s theory of relativity, for example, has triggered a huge explosion of meta-scientific studies, commonly known as the Philosophy of Science - in Natural Sciences. The second condition that requires meta-study involves the difficulties (or even failure) for a discipline to progress. There are times when a discipline seems to be going nowhere. This is because the discipline has failed to achieve the original goals set by its founders, or it has fallen behind corresponding disciplines. In this case, the researchers of this discipline become increasingly critical and begin to question their fundamentals, such as the ability to achieve their goals, the effect of their methods and the adequacy of their theories. As a result, it is likely that some researchers will venture themselves to undertake meta-study.

According to Zhao (1991), meta-study begins with examining problems encountered in the primary studies (primary data) and end with the indication of proposals to solve these problems. If the primary study is a long journey to an unknown location, meta-study often involves frequent stops for rest, route identification, review of travel plans, or even a rethinking of the final destination. And the more difficult the journey is, the more necessary the pauses will be. Thus, ‘it is the problems in primary study that lead to frequent pauses for meta-study, not the pauses for meta-study that result in the problems in primary study’ (Zhao, 1991, p. 381). Thus, meta-study is usually a product of crisis in the area/discipline (Zhao, 1991).

In the case of Sociology, meta-study emerged in the 1960s in a context of the collapse of the dominant sociological paradigm (Zhao, 1991; Ritzer, 1991). Regarding Education Policy, meta-research is very recent, since they began to be concluded after 2010. It can be argued that the very accumulation of research and expansion of the field of Education Policy has boosted interest in the achievement of meta-research. However, before that, there were already some questions about the non-explication of theoretical framework and the importance of carrying out studies on the framework that were being used in the field (Mainardes, 2009). Despite this, it should be noted that meta-research studies were preceded by important review studies carried out as of the late 1990s (Wittmann & Gracindo, 2001; Azevedo & Aguiar, 2001a, 2001b). These review studies were performed with the intention of mapping production, identifying trends, etc. Recent concerns have underlined the importance of analyzing the quality of research, the theoretical underpinnings, and the role of theory in research. However, these are recent initiatives that lack conceptual and deeper theoretical-epistemological reflection.

The most systematic critiques on research in the field of Education Policy are only just emerging and some points coincide with the criticisms of other Latin American countries (Mainardes, 2009). One common point that has been highlighted is that the field of Education Policy is comprehensive, in expansion and in permanent construction, and that there is a significant volume of research on education policies and still a limited number of works dealing with the theoretical-epistemological foundations of research (Mainardes, 2009, 2018). From this, we argue that investing in the analysis of the theoretical-epistemological foundations of research on Education Policy, however arduous that task is, is fundamental for the continuous maturation of the field.

The epistemological analysis in the area of Education and in the field of Education Policy

In the epistemological analysis of research in the area of Education, the contributions of Sánchez Gamboa (2008) are very relevant. In research conducted since the beginning of the 1980s, Sánchez Gamboa developed a paradigmatic scheme for the analysis of scientific production. The paradigmatic scheme includes the analysis of the dialectic relationship between question and answer. Regarding the construction of the answer, the author proposes the analysis of the following levels: technical, methodological, theoretical, epistemological (conception of causality, validation of scientific evidence and science - scientific criteria), gnosiological and ontological presuppositions (broad and complex categories, conceptions of man, education and society, conceptions of reality). This scheme was used in the epistemological analysis of production in some specific areas (Physical Education, Special Education, and so on) Subsequently, Silva and Sánchez Gamboa (2014) presented an ‘epistemological matrix’ as an instrument of theoretical-philosophical analysis of the scientific production of the action sciences. This matrix is formed by logical elements (technical level, methodological level, theoretical level, epistemological level, gnosiological presuppositions and ontological presuppositions) and by historical and social elements - context elements: macro-structural, mesostructural and microssocial. The epistemological matrix shows itself as a well-founded proposal for the epistemological analysis of production in the area of Education.

In the case of the epistemological analysis of research in the field of Education Policy in Brazil, studies are still recent and are closely related to the productions of researchers linked to ReLePe. Mainardes, Stremel and Soares (2018) carried out a review of the Brazilian publications on the theoretical-epistemological aspects of the research in Education Policy and listed 123 works, distributed in the following categories: a) Productions of theoretical nature on the theoretical foundations of research on Education Policy; b) Examination of the contribution of authors for the analysis of policies, as well as the contribution of specific epistemological perspectives; c) Studies of literature review and meta-research; d) Research on the constitution of the field, scientific associations, Working Groups, Research Groups and specialized journals for the theoretical development of the area of Education Policy.

In the category ‘literature review studies and meta-research in Education Policy’, Mainardes, Stremel and Soares (2018) relate 26 works (e.g. Wittmann & Gracindo, 2001; Azevedo & Aguiar, 2001a, 2001b; Arosa, 2013, 2016; Souza, 2014; Silva & Jacomini; 2016; Santos & Azevedo, 2009, 2012, 2014; Mainardes, 2017; Tonieto, 2018).

Literature review studies have mapped productions of specific periods, analyzing the privileged themes, theoretical trends, and other relevant features to understand the field of Education Policy. In this set, there are relevant works of research on Education Policy between 1991 and 1997 (Wittmann & Gracindo, 2001). In the studies classified as meta-research, the studies of Arosa (2013, 2016), Souza (2014), Silva and Jacomini (2016), Tonieto (2018) and Mainardes (2017, 2018) were included.¹³ These studies were classified as meta-research, since they do not aim to synthesize the results of the analyzed research, but to analyze the themes, theoretical framework, methodologies, among other aspects. Arosa (2013, 2016) and Souza (2014) analyzed research presented in GT 5 (State and Education Policy) of the Associação Nacional de Pós-Graduação e Pesquisa em Educação¹⁴ (ANPEd); Silva and Jacomini (2016) present an analysis of theses and dissertations on Education Policies, defended from 2000 to 2010. Tonieto (2018) analyzes a set of theses defended in the period of 2010-2012. Mainardes (2017, 2018) presents the epistemological analysis of papers published in the period of 2010-2012. These papers bring important elements for the understanding of the field research, trends, characteristics, weaknesses and epistemological obstacles.

Meta-research in the context of EAEP

In this section, we present a methodological proposal for meta-research on Education Policy that was formulated within the scope of the EAEP. This proposal constitutes a flexible theoretical exercise, which can be adapted according to the purposes and theoretical framework of the research. The proposal focuses on the analysis of the theoretical-epistemological aspects of research, assuming a double dimension: a) reflective dimension: meta-research takes the scientific production of the field of Education Policy (sample) as the object of study, reflection and analysis, which can result in important elements for the understanding of the field research (identification of trends, strengths, weaknesses, obstacles, etc.; b) the theoretical-analytical dimension: the conclusions of meta-research can be ‘reinvested in scientific work’ (Bourdieu, 2011, p.38)¹⁵, as well as this, they contribute to increase the scientificity possibilities of the scientific community as a whole.

From the EAEP formulations, we have considered that meta-research can be developed in the following steps:

1º) Definition of meta-research and sample purposes

The starting point for meta-research is the definition of its purposes and the sample, that is, the selection of a set of texts: papers, theses, dissertations or other publications. The definition of the sample varies according to the purposes of the research: the definition of a set of journals, a period of time, a specific research topic or combinations between them. Although the theoretical framework of research can be constructed and adjusted throughout the entire process, it is important that an initial theoretical framework is defined by the researcher. Sample size varies from one research project to another. For example, the study carried out by Silva and Jacomini (2016) involved the analysis of 851 theses and 432 dissertations (abstracts); Tonieto’s research (2018) included the analysis of 27 theses (complete works); Mainardes’s work (2017) involved the analysis of 140 (complete) papers.

2º) Organization and systematization of the sample

It is important that the sample texts are recorded in a spreadsheet, which can contain, among other data: complete reference, abstract, keywords and the items that will be considered in the analysis (e.g. epistemological perspective, epistemological positioning, methodological procedures, etc.).

3º) Systematic reading

The next step consists on the systematic reading of the selected texts, with the objective of identifying the essential elements of the theoretical-epistemological analysis. Among others, this scheme may include the following items:

a) Research theme.

b) Type of research (empirical research, theoretical, bibliographical, commentary or criticism).

c) Epistemological perspective:

• Does the author make an explicit epistemological perspective?

• Is there a section for presenting the theoretical framework or it is presented throughout the analyzes?

• Which authors are cited in the theoretical framework?

• What are the theoretical frameworks (concepts) that appear in the text?

• Is there evidence of a combined theorization, added or lack of theorization?¹⁶

d) Epistemological positioning:

• Is there explicitness of a specific positioning (eg, critical, descriptive analysis, etc.)?

• How can the author’s epistemological positioning be classified (critical-radical, critical, critical-reprodutivist, reformist, neoliberal, seemingly neutral, empiricist, etc.)?

e) Epistemethodological approach:

• What data collection procedures were used?

• How was the sample defined?

• Is there internal coherence (guiding thread) between objectives, theoretical reference, methodology, data analysis and conclusions?¹⁷

• Is there coherence between problems, objectives, hypotheses (mainly in theses and dissertations)?

• Is the analysis coherent with the theoretical-epistemological perspective (how does the author deal with the theory or theories he assumed as the basis for his research)?

• Is the textual configuration coherent? Is it possible to map the key points of the text?

f) Argumentation (if any):

• Are there explicit arguments (and sub-arguments)?

• Are the arguments properly based on evidence and theories?¹⁸

• Are there signs of originality, elaboration of new concepts or categories, new theorizations?

g) Scope: local, regional, national, international/global.

h) Level of abstraction: description, analysis, understanding (Mainardes & Tello, 2016).

i) Quotations (of highly relevant excerpts which synthesize the research).

An important aspect to be highlighted is that any typology or classification is arbitrary and related to specific purposes (Tello & Mainardes, 2015b). Moreover, due to their arbitrary nature, the same objects can be classified in different ways.

In the process of analysis, it should also be considered that meta-research is not intended to judge the works or their authors. The objective is to understand the main tendencies of the field, from the cut-off made by the researcher. Thus, the classifications used should be understood as a theoretical exercise on the Education Policy research. Classifications can lead to crystallization and the use of rigid schemes and closed categories. For this reason, we emphasize that the developed classifications need to be understood as systematization attempts and analysis exercises.

Some meta-research challenges

1. a) The researcher interested in conducting meta-research needs to comprehend the diversity of epistemological perspectives employed in the field as well as other classifications that are adopted in Social Sciences, Political Science and Social Theory¹⁹.

2. b) It is fundamental that the researcher understands that all classification is arbitrary and is linked to a specific purpose. Meta-research in the field of knowledge production in Education Policy has shown that many researchers do not make explicit their epistemological perspective. Thus, in meta-research, the researcher will need to identify the perspectives and epistemological positioning from the theoretical framework, the authors cited, styles of analysis, etc.

3. c) The purpose of meta-research is not to judge the works or authors individually, but to seek to understand how knowledge has been produced in the field (Education Policy), through a set (sample) of researchers. Such analyzes can bring relevant data of the research in the field.

4. d) Coherence and consistency of research and publications need to be analyzed within the epistemological perspective employed by the researcher.

5. e) It is fundamental that the researcher also expresses the theoretical reference and the ethical principles that guide his/her own analysis.

6. f) The identification of the epistemological positioning of the authors of the analyzed works is a highly complex task. Such positions need to be understood in the context (economic, historical, political and social) in which the research was carried out.

Final considerations

In this paper, we argued that meta-research in Education Policy is a relevant strategy for understanding the research situation in the field, that is, to make a diagnosis about how knowledge is produced, as well as identify characteristics, tendencies, weaknesses and obstacles for the advancement of field research. From this diagnosis, it is possible to think of collective strategies to improve research, as well as to fight for scientific policies that meet the expectations and needs of the researchers.

Due to the continuous expansion of the field of Education Policy in Brazil and Latin America, we believe that it is necessary to invest in meta-research, even if they initially have a more focused dimension (a set of theses, papers, etc.). At the same time, it is necessary to deepen the ways of doing meta-research in the field and its theoretical foundations, since it is an area of research still underdeveloped in the field.

Finally, we emphasize that the knowledge built from meta-research has significant elements for the improvement of the field research and for the continuous improvement of the education of researchers (mainly in the Graduate Programs in Education).

REFERÊNCIAS

• AROSA, A. C. C. A produção acadêmico-científica sobre política educacional no GT - 5 da ANPEd (2000 a 2009) 2013. 252 f. Tese (Doutorado em Educação) - Universidade Federal do Rio de Janeiro, Rio de Janeiro, 2013.
• AROSA, A. C. C. A pesquisa sobre Política Educacional: estudo sobre o GT5 da ANPEd. Revista de Estudios Teóricos y Epistemológicos en Política Educativa, v. 1, n. 1, p. 10-29, jan./jun. 2016.
• AZEVEDO, J. M. L.; AGUIAR, M. A. A produção do conhecimento sobre a política educacional no Brasil: um olhar a partir da ANPED. Educação & Sociedade, Campinas, v. 22, n. 77, p. 49-70, set./dez. 2001a. DOI: http://dx.doi.org/10.1590/S0101-73302001000400004.
  » http://dx.doi.org/10.1590/S0101-73302001000400004.
• AZEVEDO, J. M. L. de; AGUIAR, M. A. Políticas de educação: concepções e programas. In: WITTMANN, L. C.; GRACINDO, R. V. (Coords.). O estado da arte em política e gestão da educação no Brasil: 1991 a 1997. Brasília: ANPAE; Campinas: Autores Associados, 2001b. p. 73-87.
• BACHELARD, G. O racionalismo aplicado Rio de Janeiro: Zahar Editores, 1977.
• BOOTH, W. C.; COLOMB, G. G.; WILLIAMS, J. M. A arte da pesquisa São Paulo: Martins Fontes, 2005.
• BOURDIEU, P. Homo academicus Florianópolis: UFSC, 2011.
• BOURDIEU, P.; CHAMBOREDON, J. C.; PASSERON, J. C. Ofício de sociólogo: metodologia da pesquisa na sociologia. 6. ed. Petrópolis: Vozes, 2007.
• FINFGELD, D. L. Metasynthesis: the state of the art - so far. Qualitative Health Research, v. 13, n. 7, p. 893-904, 2003. DOI: https://10.1177/1049732303253462
  » https://10.1177/1049732303253462
• IOANNIDIS, J. P. A. Meta-research: why research on research matters. PLoS Biology, v. 16, n. 3, 2018. DOI: https://doi.org/10.1371/journal.pbio.2005468
  » https://doi.org/10.1371/journal.pbio.2005468
• IOANNIDIS, J. P. A. et al. Meta-research: evaluation and improvement of research methods and practices. PLoS Biology , v. 13, n. 10, 2015. DOI: https://doi.org/10.1371/journal.pbio.1002264
  » https://doi.org/10.1371/journal.pbio.1002264
• LOSADA, R. L.; CASAS CASAS, A. Enfoques para el análisis político: historia, epistemología y perspectivas de la ciencia política. Bogotá: Editorial Pontificia Universidad Javeriana, 2010.
• MAINARDES, J. Análise de políticas educacionais: breves considerações teórico-metodológicas. Contrapontos, Itajaí, v. 9, n. 1, p.4-16, jan./abr. 2009.
• MAINARDES, J. A pesquisa sobre política educacional no Brasil: análise de aspectos teórico-epistemológicos. Educação em Revista, Belo Horizonte, v. 33, p. 1- 25, 2017. DOI: http://dx.doi.org/10.1590/0102-4698173480
  » http://dx.doi.org/10.1590/0102-4698173480
• MAINARDES, J. A pesquisa no campo da Política Educacional: perspectivas teórico-epistemológicas e o lugar do pluralismo. Revista Brasileira de Educação, Rio de Janeiro, v. 23, p. 1-21, 2018. DOI: http://dx.doi.org/10.1590/s1413-24782018230034.
  » http://dx.doi.org/10.1590/s1413-24782018230034.
• MAINARDES, J.; TELLO, C. A Pesquisa no campo da Política Educacional: explorando diferentes níveis de abordagem e abstração. Arquivos Analíticos de Políticas Educativas, v. 24, n. 75, p. 1-17, jul. 2016. DOI: http://dx.doi.org/10.14507/epaa.24.2331.
  » http://dx.doi.org/10.14507/epaa.24.2331.
• MAINARDES, J.; STREMEL, S.; SOARES, S. T. Aspectos teórico-epistemológicos da pesquisa em Política Educacional no Brasil: mapeamento e reflexões. Movimento, Niterói, v. 5, n. 8, p. 43-74, jan./jun. 2018.
• MCLENNAN, G. Post-Marxism and the “four sins” of modernist theorizing. New Left Review, v. 218, p. 53-74, 1996.
• RITZER, G. Reflections on the rise of metatheorizing in Sociology. Sociological Perspectives, v. 34, n. 3, p. 327-348, 1991.
• ROSENBAUM, A.; LANGHINRICHSEN-ROHLING, J. Meta-research on violence and victims: the impact of data collection methods on findings and participants. Violence and Victims, v. 21, n. 4, p. 404-409, 2006. DOI: http://dx.doi.org/10.1891/vivi.21.4.404.
  » http://dx.doi.org/10.1891/vivi.21.4.404.
• SÁNCHEZ GAMBOA, S. Pesquisa em educação: métodos e epistemologias. Chapecó: Argos, 2008.
• SANTOS, A. L. F. dos; AZEVEDO, J. M. L. de. A Pós-Graduação no Brasil, a pesquisa em educação e os estudos sobre a política educacional: os contornos da constituição de um campo acadêmico. Revista Brasileira de Educação , Rio de Janeiro, v. 14, n. 42, p. 534-550, set./dez. 2009. DOI: http://dx.doi.org/10.1590/S1413-24782009000300010.
  » http://dx.doi.org/10.1590/S1413-24782009000300010.
• SANTOS, A. L. F. dos; AZEVEDO, J. M. L. de. Regulação e legitimação da pesquisa sobre a política educacional como campo acadêmico: um estudo a partir dos programas de Pós-Graduação do Nordeste brasileiro. In: GOMES, A. M.; OLIVEIRA, J. F. de (Orgs.). Reconfiguração do campo da educação superior Campinas: Mercado de Letras, 2012. p. 71-93.
• SANTOS, A. L. F. dos; AZEVEDO, J. M. L. de. O GT 05 - Estado e Política Educacional - no contexto do Encontro de Pesquisa Educacional do Norte e Nordeste: um olhar sobre a sua produção. In: GOMES, A. M.; LEAL, T. F. (Orgs.). Pesquisas em educação nas regiões Norte e Nordeste: balanço e perspectivas. Recife: UFPE, 2014. p. 67-82.
• SILVA, A. A.; JACOMINI, M. A. (Orgs.). Pesquisa em políticas educacionais: características e tendências. Feira de Santana: UEFS, 2016.
• SILVA, R. H. R.; SÁNCHEZ GAMBOA, S. Do esquema paradigmático à matriz epistemológica: sistematizando novos níveis de análise. Educação Temática Digital, Campinas, v. 16, n. 1, p. 48-66, jan./abr. 2014. DOI: https://doi.org/10.20396/etd.v16i1.1329.
  » https://doi.org/10.20396/etd.v16i1.1329.
• SOUZA, Â. R. de. A pesquisa em políticas educacionais no Brasil: de que estamos tratando? Práxis Educativa, Ponta Grossa, v. 9, n. 2, p. 355-367, jul./dez. 2014. DOI: http://dx.doi.org/10.5212/PraxEduc.v.9i2.0003
  » http://dx.doi.org/10.5212/PraxEduc.v.9i2.0003
• TELLO, C. Las epistemologías de la política educativa: vigilancia y posicionamiento epistemológico del investigador en política educativa. Práxis Educativa , Ponta Grossa, v. 7, n. 1, p. 53-68, jan./jun. 2012. DOI: http://dx.doi.org/10.5212/PraxEduc.v.7i1.0003.
  » http://dx.doi.org/10.5212/PraxEduc.v.7i1.0003.
• TELLO, C.; MAINARDES, J. La posición epistemológica de los investigadores en Política Educacional: debates teóricos en torno a las perspectivas neo-marxista, pluralista y pos-estructuralista. Arquivos Analíticos de Políticas Educativas , v. 20, n. 9, p. 1-31, 2012. DOI: http://dx.doi.org/10.14507/epaa.v20n9.2012.
  » http://dx.doi.org/10.14507/epaa.v20n9.2012.
• TELLO, C.; MAINARDES, J. Pluralismos e investigación en política educativa: una perspectiva epistemológica. Revista Mexicana de Investigación Educativa, México, v. 20, n. 66, p. 763-788, jul./set. 2015a.
• TELLO, C.; MAINARDES, J. Revisitando o enfoque das epistemologias da política educacional. Práxis Educativa , Ponta Grossa, v. 10, n. 1, p. 153-178, jan./jun. 2015b. DOI: http://dx.doi.org/10.5212/PraxEduc.v.10i1.0007.
  » http://dx.doi.org/10.5212/PraxEduc.v.10i1.0007.
• TONIETO, C. Características epistemológicas das teses de Políticas Educacional no triênio 2010-2012. 2018 2018. 214 f. Tese (Doutorado em Educação) - Universidade de Passo Fundo, Passo Fundo, 2018.
• WITTMANN, L. C.; GRACINDO, R. V. (Coords.). O estado da arte em política e gestão da educação no Brasil: 1991 a 1997 . Brasília: ANPAE ; Campinas: Autores Associados , 2001.
• ZHAO, S. Metatheory, metamethod, meta-data-analysis: what, why, and how? Sociological perspectives, v. 34, n. 3, p. 377-390, 1991.

• 10
  Network of Theoretical and Epistemological Studies on Education Policy. www.relepe.org.
• 11
  A concept explored by Bachelard (1977) and also by Bourdieu, Chamboredon and Passeron (2007), whose understanding is vital in epistemological studies. In general terms, it can be understood as ‘self-surveillance’, which consists of the ability of the scientific mind to critically analyze itself and its decision-making processes in relation to its object of study (Bachelard, 1977). Epistemological surveillance, or intellectual surveillance, or surveillance of surveillance, is the act of monitoring not only the application of the method, but the method itself. It requires that the method is put to the test, and also that one should risk, in experience, rational certainties. It also requires the analysis of obstacles that impede the development of scientific knowledge and masks the ruptures of knowledge. With this, it is a surveillance that aims to destroy the absolute method, reason and facts (Bachelard, 1977).
• 12
  ^{Annex A} presents the definition of Finfgeld’s terms (2003).
• 13
  Of these studies, only Mainardes (2017, 2018) and Tonieto (2018) refer to meta-research.
• 14
  National Association of Graduate Studies and Research in Education.
• 15
  According to Bourdieu (1988, p. 15-16): ‘When research comes to study the very realm within which it operates, the results which it obtains can be immediately reinvested in scientific work as instruments of reflexive knowledge of the conditions and the social limits of this work, which is one of the principal weapons of epistemological vigilance. Indeed, perhaps we can only make our knowledge of the scientific field progress by using whatever knowledge we may have available in order to discover and overcome the obstacles to science which are entailed by the fact of holding a determined position in the field’.
• 16
  McLennan (1996) explains that combined explanatory strategies are legitimate and perhaps promising. In this sense, combined theorization is an effort to articulate theories or concepts derived from different theories, with the objective of composing a consistent theoretical framework to support a given analysis. Such an effort demands making theoretical choices and justifying them, which implies an exercise of reflexivity and epistemological surveillance. The notion of added theorization means the more or less random the adoption of theories, concepts, ideas of different theories and epistemological perspectives are, resulting in a set of ideas and concepts without coherence, unity and theoretical articulation. The result of the simple addition and overlap of ideas of different authors results in a failed attempt to define a theoretical framework, which can be considered fragile, disjointed and epistemologically inconsistent. Regarding the concepts of combined and added theorization, see Mainardes (2017, 2018).
• 17
  Analysis of the level of consistency is challenging. Based on structuralist theories, we have employed specific strategies, such as the allocation of forces (+, ++, +++).
• 18
  Booth, Colomb and Williams (2005) explain that an argument is an assertion based on evidence, which has a foundation. In addition, arguments can (and often need to) have caveats.
• 19
  One of the available classifications is the one presented by Losada and Casas Casas (2010). According to the authors, an approach (or theoretical perspective) ‘is a powerful reflector that makes us see certain aspects of reality, but it leaves others in the dark or total darkness’ (Losada & Casas Casas, 2010, p. 15). The classification proposed by the authors includes the following approaches: a) approaches that privilege the inner self of the human being; b) approaches that privilege the social environment; c) approaches that favor institutions; d) approaches that privilege the cultural environment (constructivist, culturalist, feminist); e) approaches that privilege the total environment (structuralist, systemic, structural-functionalist, biopolitical approach); f) critical macro-model: Marxist approach or contemporary critical approaches; g) Postmodern macromodel.

Annex A

TABLE 2
METASYNTHESIS TERMINOLOGY

Term	Comment
Grounded formal theory	Inductive research approach that uses findings from substantive grounded theory studies to synthesize a new grounded formal theory (Kearney, 1998a).
Meta-analysis	Term frequently used to describe the statistical combining and analysis of results from multiple quantitative research studies (Egger & Smith, 1997). Because of the term’s common association with quantitative methods and data aggregation versus interpretation, its use in relationship to qualitative metasynthesis is discouraged.
Metastudy	Inductive research approach involving the analysis and interpretation of theory, methods, and research findings across qualitative studies, and the synthesis of this work to formulate new interpretations.
Meta-data analysis	Analysis and interpretation of findings across multiple qualitative research reports.
Metamethod	Analysis and interpretation of methodological applications across multiple qualitative research reports.
Metatheory	Analysis and interpretation of theoretical, philosophical, and cognitive perspectives; sources and assumptions; and contexts across multiple qualitative studies (Paterson, Thorne, & Dewis, 2001).
Metasynthesis	Umbrella term referring to the synthesis of findings across multiple qualitative reports to create a new interpretation. Proposed types of metasyntheses include theory building, metastudy, grounded formal theory, theory explication, and descriptive study.

• SOURCE: Adapted from Finfgeld (2003, p. 895).

Publication Dates

• Publication in this collection
  Nov-Dec 2018

History

• Received
  03 June 2018
• Accepted
  23 July 2018