Compare or Perish: A Handbook for Strengthening the Field of Policy Analysis

PETERS, B. Guy; FONTAINE, Guillaume. Handbook of Research Methods and Applications in Comparative Policy Analysis. Northampton: Edward Elgar Publishing Limited, 2020

‘Compare or Perish’, said the political scientist De Beus at a social science conference in 1992 reflecting on the poor state of comparison in political science at that time. The importance of comparison in areas such as political science and public administration has been advocated by many authors as the “very essence of the scientific method” (ALMOND and POWELL, 1966, p. 878; BRANS, 2003; DAHL, 1947; LIJPHART, 1971; PETERS and PIERRE, 2003; VERBA, 1967). Almost three decades after De Beus’ statement, we can say that the field of comparison in areas such as political science and public administration has increased considerably, with many groups concerned with improving methods and questions of comparison.

However, while we have witnessed an increase in comparative perspectives both in political science and public administration, the same cannot be said about the field of policy analysis.

Actually, during the period in question, this area was still trying to establish itself as a field, with its own community, journals, methods, and research questions. Until recently, the field of public policy analysis was subordinated to political science and public administration. The distinction of this field as an interdisciplinary one is relatively new, as it is its institutionalization – the international association of public policy – IPPA – was created in 2014. In 2013 we had the first international conference of public policy.

This does not mean that public policies were not compared in the literature, but rather the methods and theories of both political science and public administration fields were used for this. However, with the strengthening of the policy analysis area and its expansion as an interdisciplinary but independent field, the need to build its own theories and methods began to emerge. The new book organized by Guy Peters and Guillaume Fontaine, ‘Handbook of Research Methods and Applications in Comparative Policy Analysis’ is an important milestone in this effort.

In its 22 chapters, written by authors from different countries, the book intends to “advance the understanding of methodology in the study of comparative public policies, as much as to broaden the array of methods and techniques considered by CPA (comparative policy analysis) scholars in their research design” (PETERS and FONTAINE, 2020, p. 01).

Throughout the book, all the authors try to demonstrate, from different perspectives, the importance of adopting specific methods or unique approaches to analyze public policies. As proposed by the organizers in the introduction to justify this effort, “given the multi-disciplinary nature of policy studies, the methodological challenges in CPA are arguably more complex than for other research areas in social science. Therefore, there is a greater need to provide an overview of the available methods, as well as their strengths and weaknesses for addressing theoretical issues (…)” (PETERS and FONTAINE, 2020, p. 01).

One of the main justifications for developing a specific field for comparative policy analysis is its multi-disciplinary nature and the idea that this agenda requires multiple-methods. Influences from political science, public administration, sociology, anthropology, economy, and law studies are only some examples of how the policy analysis area dialogues with different theories, methods, and approaches. The organizers also rely on an idea proposed by Pollitt (2013) in one of his last books to justify the specificity of the CPA field: the argument that the context matters when analyzing public policies. Introducing the importance of the context into a comparison is one of the distinctions of the field, as the book advocates. Another distinctive feature of this field proposed by the book is that policy analysis is usually concerned with processes, elements that should also be introduced into CPA.

The book makes also an important contribution in convincing readers why it is important to use comparison in policy analysis. It answers this question by arguing with theoretical and empirical reasons. Comparison enables governments to develop more effective and efficient policies, avoiding the replication of failures and saving times with lessons from other experiences. “Comparative policy analysis and policy analytic studies contribute both to the tenet of evidence-based policy making” (GEVA-MAY, HOFFMAN and MUHLEISEN, 2020, p. 371). Comparisons also allow us understand general mechanisms and typologies, one of the main bases of knowledge construction in social science. Comparing is then a way to produce more robust insights that can contribute to the theory (PETERS and FONTAINE, 2020, p. 02).

In this way, the book is convincing in its demonstration of the importance of CPA, its characteristics, and the uses of comparison in policy analysis. It is divided into six parts. The first one presents the current methodological debate in social science, discussing issues such as comparative methods (PETERS, 2020); most-similar and most-different systems design (ANCKAR, 2020); and the uses of case studies for developing theory (DOWDING, 2020). The second part presents contemporary trends in research related to comparison and causalities. Here the authors discuss how to introduce comparisons into some of the traditional approaches to policy analysis, such as the Advocacy Coalition Framework (NOHRSTEDT, WEIBLE, INGOLD, and HENRY, 2020); how to compare agenda settings (BONAFONT, GREEN-PEDERSEN, and SEEBERG, 2020); Historical Research (JARAMILLO, 2020), and Policy Transfers (OLIVEIRA, 2020). Part III is concerned with the problem of measurements and experiments in CPA, discussing types of experiments (JOHN, 2020); how to measure changes (TOSUN and SCHNEPF, 2020), how to use indexes (ERKKILÄ, 2020), and the uses of text as data (GILARDI and WÜEST, 2020). Part IV discusses the uses of mixed and multi-methods in CPA, presenting Critical Multiplism (DUNN and PETERS, 2020); Causal Case Studies (BEACH, 2020), Qualitative Comparative Analysis, QCA (THOMAN, 2020), and Process Tracing (FONTAINE, 2020). Part V focuses on qualitative techniques, including focus groups (MARIER, DICKSON, and DUBÉ, 2020), Ethnography (PACHECO-VEGA, 2020), Q Methodology (MOLENVELD, 2020), and Narrative Policy Frameworks (SMITH-WALTER and JONES, 2020). The final section discusses further reflections for the development of CPA field, observing its trends (GEVA-MAY, HOFFMAN and MUHLEISEN, 2020) and Evolutionary Theory (KAY, 2020).

After reading the book, readers will have a good overview of the possibilities of comparative policy analysis. With a great selection of issues, authors, and approaches, Peters and Fontaine are able to answer the two questions proposed at the beginning of the book: “What are the best available methods to conduct systematic CPA across time, space, and areas? And how can scholars and practitioners select them or combine them to improve the internal consistency and the external coherence of policy design in research and practice?” (PETERS and FONTAINE, 2020, p. 02).

However, all the authors are very careful to suggest that there is neither a best method nor best way to conduct comparisons in policy analysis. Actually, the book provides readers with many possibilities and a critical evaluation of their advantages and disadvantages. The book also considers that methods should never be chosen as a technique. “A methodology is best understood as the product of a philosophical ontology (the relationship between the mind and the world) and a scientific ontology” (the relationship between the status of empirical data and our knowledge about the world) (PETER and FONTAINE, 2020, p. 09 apud JACKSON, 2016). Ultimately, the book is a useful road map for scholars from CPA to make good decisions about their research design and the methods to be used, based on their philosophical and scientific ontologies.

Many reasons make this book a very important contribution to the policy analysis field. The first one is by presenting us with the road map described above which provides a “conscious choice of the methods to be used, given the strength that triangulation of methods and theory can bring to research” (PETER and FONTAINE, 2020, p. 14).

Second, it reinforces the idea that policy analysis is a multi-disciplinary and multi-methods field that needs different approaches and methods. Issues like change, process, and context are central to this field and can be addressed by different perspectives and analytical tools.

The third is that it proposes the construction of coherent models and methods that consider the specificities of policy analysis but also enables scholars to recognize themselves as a field.

Actually, it is important also to recognize Guy Peters’ efforts in this effort to constitute the field of policy analysis, offering us this new important book.

Specifically for the Brazilian agenda, the book arrives at a very important moment, as the policy analysis area is still very new (HOCHMAN, ARRETCHE, and MARQUES, 2007) and still very connected to political science and public administration.

However, considering the Brazilian agenda and that of other developing countries, there is one important issue that is neglected in the book and requires greater coverage in future CPA discussions. Policy analysis, like political science and public administration, are areas dominated by authors from the global north. The most important scholars, journals, and analytical models were developed in liberal democracies with trust in the state and institutions. This poses a set of challenges for comparative research in developing countries, considering that the constitutive bases of the state and society are different. If context matters, as advocated by Pollitt (2013), the construction of theoretical models that incorporate different contexts should also be a priority in the field of CPA. Methods of comparison of most different systems are not enough to account for comparisons when there is a difference in contexts such as the nature of state authority. Just to give an example, in the current Brazilian context, evangelical churches and criminal enterprises (ARIAS, 2017) are very influential in terms of policy agenda, policy change, and policy implementation. Issues as corruption, clientelism, and patronage are not ‘mistakes’ in many of these countries, they are actually constitutive of those societies.

If we just use CPA to compare countries like these with countries from the global north, we tend to always reinforce the idea that in these countries we have less democracy, less accountability, and worse policy outcomes – which may be true, but this may also not enable us to understand the real processes that occur in these contexts.

I must say that it is quite relevant that one of the book’s organizers, Guillaume Fontaine, works in Ecuador. And that the book has a chapter from a Brazilian scholar, Osmany Porto, and from a Mexican scholar, Raul Pacheco-Vega. This shows how scholars from developing countries are gaining space in the policy analysis field. We hope that they, together with other scholars, may be able in the future to address these questions and propose new approaches and methods that incorporate in its grounds these differences of contexts.

For this reason as well, the Handbook of Research Methods and Applications in Comparative Policy Analysis is a must-read book for those interested in advancing in the field of public policy.

References

• ALMOND, Gabriel A. and POWELL, G. Bingham (1966), Comparative politics: a developmental approach. New York: Little, Brown & Co. 348 pp..
• ANCKAR, Carsten (2020), The most-similar and most-different systems design in comparative policy analysis. In: Handbook of research methods and applications in comparative policy analysis Edited by PETERS, B. Guy and FONTAINE, Guillaume. Northampton: Edward Elgar Publishing Limited. pp. 33-48.
• ARIAS, Enrique Desmond (2017), Criminal enterprises and governance in Latin America and the Caribbean Cambridge: Cambridge University Press. 303 pp..
• BEACH, Derek (2020), Causal case studies for comparative policy analysis. In: Handbook of research methods and applications in comparative policy analysis Edited by PETERS, B. Guy and FONTAINE, Guillaume. Northampton: Edward Elgar Publishing Limited. pp. 238-253.
• BONAFONT Laura Chaquês; GREEN-PEDERSEN, Christopher, and SEEBERG, Henrik Bech (2020), Comparing agenda-settings: the Comparative Agendas Project. In: Handbook of research methods and applications in comparative policy analysis Edited by PETERS, B. Guy and FONTAINE, Guillaume. Northampton: Edward Elgar Publishing Limited. pp. 90-112.
• BRANS, Marleen (2003), Comparative public administration: from general theory to general frameworks. In: Handbook of public administration Edited by PETERS, B. Guy and PIERRE, Jon. London: SAGE Publications. pp. 424-439.
• DAHL, Robert A. (1947), The science of public administration: three problems. Public Administration Review Vol. 07, Nº 01, pp. 01-11.
• DOWDING, Keith (2020), Can a case study test a theory? Types and tokens in comparative policy analysis. In: Handbook of research methods and applications in comparative policy analysis Edited by PETERS, B. Guy and FONTAINE, Guillaume. Northampton: Edward Elgar Publishing Limited. pp. 49-66.
• DUNN, William N. and PETERS, B. Guy (2020), Critical multiplism for comparative policy analysis. In: Handbook of research methods and applications in comparative policy analysis Edited by PETERS, B. Guy and FONTAINE, Guillaume. Northampton: Edward Elgar Publishing Limited. pp. 219-237.
• ERKKILÄ, Tero (2020), Using indexes in comparative policy analysis: global comparisons. In: Handbook of research methods and applications in comparative policy analysis Edited by PETERS, B. Guy and FONTAINE, Guillaume. Northampton: Edward Elgar Publishing Limited. pp. 186-198.
• FONTAINE, Guillaume (2020), Process tracing for comparative policy analysis: a realist approach. In: Handbook of research methods and applications in comparative policy analysis Edited by PETERS, B. Guy and FONTAINE, Guillaume. Northampton: Edward Elgar Publishing Limited. pp. 277-296.
• GEVA-MAY, Iris; HOFFMAN, David C., and MUHLEISEN, Joselyn (2020), Trends in the development of comparative policy analysis. In: Handbook of research methods and applications in comparative policy analysis Edited by PETERS, B. Guy and FONTAINE, Guillaume. Northampton: Edward Elgar Publishing Limited. pp. 367-384.
• GILARDI, Fabrizio and WÜEST, Bruno (2020), Using text-as-data methods in comparative policy analysis. In: Handbook of research methods and applications in comparative policy analysis Edited by PETERS, B. Guy and FONTAINE, Guillaume. Northampton: Edward Elgar Publishing Limited. pp. 203-218.
• HOCHMAN, Gilberto; ARRETCHE, Marta, and MARQUES, Eduardo (eds)(2007), Políticas públicas no Brasil Rio de Janeiro: Editora Fiocruz. 398 pp..
• JACKSON, Patrick Thaddeus (2016), The conduct of inquiry in international relations: philosophy of science and its implications for the study of world politics. London: Routledge. 272 pp..
• JARAMILLO, Grace (2020), Comparing historical cases: advances in comparative historical research. In: Handbook of research methods and applications in comparative policy analysis Edited by PETERS, B. Guy and FONTAINE, Guillaume. Northampton: Edward Elgar Publishing Limited. pp. 113-133.
• JOHN, Peter (2020), Using experiments im comparative policy analysis: from policy evaluation. In: Handbook of research methods and applications in comparative policy analysis Edited by PETERS, B. Guy and FONTAINE, Guillaume. Northampton: Edward Elgar Publishing Limited. pp. 153-166.
• KAY, Adrian (2020), Evolutionary theory in comparative policy analysis. In: Handbook of research methods and applications in comparative policy analysis Edited by PETERS, B. Guy and FONTAINE, Guillaume. Northampton: Edward Elgar Publishing Limited. pp. 385-400.
• LIJPHART, Arend (1971), Comparative politics and the comparative method. American Political Science Review Vol. 65, Nº 03, pp. 682-693.
• MARIER, Patrik; DICKSON, Daniel, and DUBÉ, Anne-Sophie (2020), Using focus groups in comparative policy analysis. In: Handbook of research methods and applications in comparative policy analysis Edited by PETERS, B. Guy and FONTAINE, Guillaume. Northampton: Edward Elgar Publishing Limited. pp. 297-311.
• MOLENVELD, Astrid (2020), Using Q methodology in comparative policy analysis. In: Handbook of research methods and applications in comparative policy analysis Edited by PETERS, B. Guy and FONTAINE, Guillaume. Northampton: Edward Elgar Publishing Limited. pp. 333-347.
• NOHRSTEDT, Daniel; WEIBLE, Christopher M.; INGOLD, Karin, and HENRY, Adam D. (2020), Comparing policy processes: insights and lessons from the Advocacy. In: Handbook of research methods and applications in comparative policy analysis Edited by PETERS, B. Guy and FONTAINE, Guillaume. Northampton: Edward Elgar Publishing Limited. pp. 67-89.
• OLIVEIRA, Osmany Porto de (2020), Comparing international policy transfers. In: Handbook of research methods and applications in comparative policy analysis Edited by PETERS, B. Guy and FONTAINE, Guillaume. Northampton: Edward Elgar Publishing Limited. pp. 134-152.
• PACHECO-VEGA, Raul (2020), Using etnography in comparative policy analysis: premises, promises. In: Handbook of research methods and applications in comparative policy analysis Edited by PETERS, B. Guy and FONTAINE, Guillaume. Northampton: Edward Elgar Publishing Limited. pp. 312-332.
• PETERS, B. Guy (2020), The comparative method and comparative policy analysis. In: Handbook of research methods and applications in comparative policy analysis Edited by PETERS, B. Guy and FONTAINE, Guillaume. Northampton: Edward Elgar Publishing Limited. pp. 20-32.
• PETERS, B. Guy and PIERRE, J. (2003), Handbook of public administration London: SAGE Publications Ltd. 816 pp..
• POLLITT, Christopher (2013), Context in public policy and management: the missing link? Cheltenham: Edward Elgar Publishing. 464 pp..
• SMITH-WALTER, Aaron and JONES, Michael D. (2020), Using the narrative policy framework in comparative policy analysis. In: Handbook of research methods and applications in comparative policy analysis Edited by PETERS, B. Guy and FONTAINE, Guillaume. Northampton: Edward Elgar Publishing Limited. pp. 348-366.
• THOMAN, Eva (2020), Qualitative comparative analysis for comparative policy analysis. In: Handbook of research methods and applications in comparative policy analysis Edited by PETERS, B. Guy and FONTAINE, Guillaume. Northampton: Edward Elgar Publishing Limited. pp. 254-276.
• TOSUN, Jale and SCHNEPF, Julia (2020), Measuring change in comparative policy analysis: concepts and empirical. In: Handbook of research methods and applications in comparative policy analysis Edited by PETERS, B. Guy and FONTAINE, Guillaume. Northampton: Edward Elgar Publishing Limited. pp. 167-185.
• VERBA, Sidney (1967), Some dilemmas in comparative research. World Politics Vol. 20, Nº 01, pp. 111-127.

Publication Dates

• Publication in this collection
  12 Oct 2020
• Date of issue
  2020