Acessibilidade / Reportar erro

Manifestos and ideology: methodological issues and applications to Latin America

Manifestos e ideologia: questões metodológicas e aplicações na América Latina

Manifiestos e ideología: cuestiones metodológicas y aplicaciones en América Latina

Abstract:

In this paper we compare the recent dataset of Latin American country party manifestos as coded by the Manifest Project database to other estimations of position on the left-right scale and to another coding of the same documents, discussing their limits and potentialities. The differences found between results offer an interesting opportunity to discuss the method, its reliability, and the validity of the coding scheme and the scales. Our findings suggest that the fragile reliability of the hand-coded content analysis could be circumvented by employing intercoder reliability tests and that users must be cautious when basing conclusions on this project’s results.

Keywords:
party manifestos; ideology; content analysis; political parties

Resumo:

Este artigo compara a recente base de dados do Manifesto Project de documentos partidários de países latino-americanos a outras medidas de posicionamento na escala esquerda-direita e a outra codificação dos mesmos documentos, discutindo seus limites e potencialidades. As diferenças encontradas entre os resultados oferecem uma ótima oportunidade para discutir o método, sua confiabilidade e a validade do esquema de codificação e das escalas. Nossos achados sugerem que a frágil confiabilidade da análise de conteúdo manual pode ser contornada com testes de confiabilidade entre codificadores e que os usuários dos dados devem ser cautelosos ao basear conclusões sobre seus resultados.

Palavras-chave:
manifestos; ideologia; análise de conteúdo; partidos políticos

Resumen:

La reciente publicación por parte del Manifesto Project de una base de datos de manifiestos partidarios de América del Sur, además de propiciar una vasta gama de investigaciones, es una valiosa oportunidad de evaluar la confiabilidad del método de análisis manual de contenido. Este artículo compara las codificaciones y medidas producidas por el Manifesto Project con otras codificaciones manuales de los mismos documentos y con otras medidas de posición en la escala izquierda-derecha. Esta comparación nos permite discutir los límites y potencialidades del método y las implicaciones de la utilización de sus resultados como medida de ideología. Los resultados encontrados sugieren que la debilidad del análisis manual de contenido respecto de los sesgos de los codificadores podría eludirse empleando pruebas de confiabilidad entre codificadores y que los usuarios deben tener cuidado al basar las conclusiones en sus resultados.

Palabras clave:
manifiestos partidarios; ideología; análisis del contenido; partidos

Introduction5 5 Former versions of this paper were presented at the 3rd LAPolMeth Meeting in 2019, at the 43º ANPOCS Annual Meeting in 2019, and in a research seminar at the DCP/UFPE also in 2019. We thank the participants of these events and the anonymous RBCP reviewers for their helpful comments. Any remaining mistakes are our full responsibility.

How reliable are our conclusions on the political saliency and ideology of Latin American political parties based on hand-coded content analyses of manifestos? Is the left-right scale built from such analyses adequate for capturing the ideological dimension of political competition outside European countries?

Ideology is one of those concepts in social sciences that, despite not being directly observable, has implications for several observable variables. In this paper, we bracket the ontological and epistemological dimensions of ideology, and focus on its methodological dimension. That is, we neither doubt the actual existence of our object, nor that it can be objectively known by the political science. However, we question one of the methods that have been used to do that.

Party ideology seems to have lost importance - to public policy, voting behaviour, electoral coalitions, for example - in some advanced democracies as distinct governments worldwide converged into a globalized economic policy and various societies embraced post-materialist values. In contemporary Latin America, however, where struggles about redistribution are pervasive, party ideology still plays a role and has received corresponding attention from political science. Examining party ideology is imperative for understanding both the recent pink tide and the right turn in the region, for example. Thus, we need good measures of ideology.

There are many ways to measure ideology. Interviews (or the application of questionnaires) with political elites provide us with data from those who are knowledgeable about politics but have a direct interest in the political game (which can result in answers that are not always sincere). National opinion polls on the ideological positioning of parties tend to collect more sincere opinions, though with a low degree of familiarity with the political game. Expert surveys, in turn, are considered to be a strategy that would make well-informed and politically motivated responses compatible, although academics often have clearly established preferences and even organic ties to political parties, governments etc.

Another field of observation widely used for the analysis of ideological positioning is the analysis of roll call votes. Contrary to the previous strategies, here the concrete performance of the different parties is being taken as an empirical ballast. However, numerous elements demand caution in the organization of this type of research, in the selection of votes and in the reading of the data. First, many of the votes that take place daily in a legislative house are not related to topics that require ideological positions. Second, other divisions (such as government versus opposition, North versus South, large cities versus inland cities, gender issues etc.) may override voters’ initial ideological preferences.

Party manifestos also have their pros and cons as sources of ideology measures. While being the formal party statement presented to the public, such documents are at the same time targeted at the electoral context. Considering this, this article explores the content of party manifestos as a proxy for party ideology.

The Manifesto Project (MP) collects, codes, and publishes content data on electoral party manifestos from many countries. The method is hand-coded content analysis, and the standard coding scheme contains 56 categories grouped into seven domains. Although this method has been a target of various criticisms, the MP has provided data for significant body of research in political science (VOLKENS et al., 2015VOLKENS et al. Scope, range, and extent of manifesto project data usage: a survey of publications in eight high-impact journals. Berlin: WZB Berlin Social Science Center, 2015. Handbook for Data Users and Coders. version I. Disponível em: Disponível em: https://manifesto-project.wzb.eu/datasets/mpduds# . Acesso em: 27 out. 2019.
https://manifesto-project.wzb.eu/dataset...
).

The MP recently released a dataset of 98 coded manifestos of 61 political parties from five countries of Latin America, launched for 24 elections from 1989 to 2018 (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
; KRAUSE et al., 2018KRAUSE, W. et al. Manifesto Corpus. Version: 2018-2. WZB Berlin Social Science Center, 2018. Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2018b . Acesso em: 16 out. 2019.
https://doi.org/10.25522/manifesto.mpdss...
; VOLKENS et al., 2019VOLKENS, A. et al. The manifesto data collection (MRG/CMP/MARPOR), M. P. Berlin 2019.). Data includes the frequency of each category in each document, as well as some scales, including the left-right scale. These data provide a broad perspective for new analyses, allowing for comparison of parties either in a specific country or between countries, and either in a specific election or over time. The availability of such data provided by the MP also makes it possible to test the reliability of content analysis and the validity of the coding scheme. These are the goals of this paper.

Ultimately, none of the available alternatives is able to fully reflect party ideology. Although each approach has its limitations, the analysis of party manifestos has gained visibility and the MP data has been widely used. In view of such dissemination, we chose to inquire into the hand-coded content analysis of party manifestos, by comparing it to other ideology measures.

We pursue that aim in two ways: first, by comparing the MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) results to other ideology measures in Latin American countries, and second, by comparing MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) coding of two Brazilian parties’ manifestos to our own coding (MADEIRA; VIEIRA; TAROUCO, 2017MADEIRA, Rafael, VIEIRA, Soraia; TAROUCO, Gabriela. Agendas, preferências, competição: PT e PSDB em disputas presidenciais. Caderno CRH, v. 30, n. 80, p. 257-273, 2017. ), hereafter MVT, which applied the same protocol. We, then, discuss the implications of distinct codings, namely, the distinct positions in the Right-Left (RILE) scale.

Our findings suggest that there is room for improvement in the reliability of hand-coded analyses of manifestos. Despite many advantages of the MP method, we found that the comparability of the coding results is limited. Nevertheless, we argue for the internal validity of both the coding scheme and the left-right scale. The ideological positions measured through MP coding are quite similar to some other assessments of Latin American parties’ positions.

In the next sections we compare the MP left-right scale position to other ideology measures for several Latin American parties. Next, we discuss the classifications of salient policy issues that resulted from the analysis of six Brazilian manifestos, comparing their ideological positions, and discussing the implications of discrepancies for further analyses. Finally, we make some considerations about the weaknesses and advantages of the method and about the research agenda.

Latin American parties’ ideological positions

This section compares the RILE scale as computed using MP data (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) to other assessments of party ideology in Latin America. The MP RILE measures are available for 96 Latin American manifestos, from 1989 to 2018, and vary from -51.1 (PSOL manifesto, issued for the 2006 election) to 42.5 (PSL manifesto, issued for the 2018 election). The complete set of measures is available in the appendix Appendix. 1a. Means in RILE categories, for four Brazilian documents of 2006 and 2010 elections, MP and MVT coders Category Coder Mean Std. Dev. Right: 104 - Military (positive) MP318 0.894 0.460 MVT 0.700 0.707 Right: 201 - Freedom and human rights (positive)20 MP318 1.901 1.442 MVT 0.650 0.480 Right: 203 - Constitutionalism (positive) MP318 0.252 0.291 MVT 0.000 0.000 Right: 305 - Political authority (positive)21 MP318 3.288 3.781 MVT 0.000 0.000 Right: 401 - Free enterprise (positive) MP318 0.223 0.281 MVT 0.050 0.050 Right: 402 - Economic Incentives (positive) MP318 2.212 0.885 MVT 3.275 2.205 Right: 407 - Protectionism (negative) MP318 0.427 0.070 MVT 0.000 0.000 Right: 414 - Economic orthodoxy (positive) MP318 1.918 0.618 MVT 0.325 0.650 Right: 505 - Welfare state limitation (positive) MP318 0.103 0.206 MVT 0.000 0.000 Right: 601 - National way of life (positive)22 MP318 0.974 0.876 MVT 0.000 0.000 Right: 603 - Traditional morality (positive) MP318 0.165 0.227 MVT 0.000 0.000 Right: 605 - Law and order (positive)23 MP318 4.076 0.273 MVT 4.650 1.303 Right: 606 - Social harmony (positive)24 MP318 0.937 0.713 MVT 0.050 0.050 Left: 103 - Anti-imperialism (positive)25 MP318 1.187 1.276 MVT 0.000 0.000 Left: 105 - Military (negative) MP318 0.031 0.062 MVT 0.000 0.000 Left: 106 - Peace (positive) MP318 0.527 0.700 MVT 0.400 0.616 Left: 107 - Internationalism (positive) MP318 1.873 1.688 MVT 2.025 1.323 Left: 202 - Democracy (positive)26 MP318 5.289 3.253 MVT 1.800 1.804 Left: 403 - Market regulation (positive) MP318 1.809 1.000 MVT 0.025 0.050 Left: 404 - Economic planning (positive) MP318 1.127 0.287 MVT 0.475 0.591 Left: 406 - Protectionism (positive) MP318 0.166 0.247 MVT 0.425 0.506 Left: 412 - Controlled economy (positive) MP318 0.407 0.303 MVT 0.000 0.000 Left: 413 - Nationalization (positive) MP318 0.245 0.093 MVT 0.000 0.000 Left: 504 - Welfare state expansion (positive) MP318 9.518 1.316 MVT 11.450 5.500 Left: 506 - Education expansion (positive) MP318 5.737 0.813 MVT 6.300 1.503 Left: 701 - Labour groups (positive) MP318 2.639 0.962 MVT 0.950 0.929 Source: Krause et al. (2019) and MVT (2017) 1b. Means in RILE categories, for two Brazilian documents of 2014 election, MP and MVT coders Category Coder Mean Std. Dev. Right: 104 - Military (positive) MP328 0.565 0.799 MVT 0.050 0.050 Right: 201 - Freedom and human rights (positive) 27 MP328 1.369 0.498 MVT 1.700 2.121 Right: 203 - Constitutionalism (positive) MP328 0.000 0.000 MVT 0.000 0.000 Right: 305 - Political authority (positive)28 MP328 3.349 3.297 MVT 0.000 0.000 Right: 401 - Free enterprise (positive) MP328 1.499 1.876 MVT 0.000 0.000 Right: 402 - Economic Incentives (positive) MP328 4.557 0.371 MVT 5.750 1.061 Right: 407 - Protectionism (negative) MP328 0.283 0.400 MVT 0.000 0.000 Right: 414 - Economic orthodoxy (positive) MP328 1.988 0.864 MVT 0.150 0.150 Right: 505 - Welfare state limitation (positive) MP328 0.452 0.639 MVT 0.000 0.000 Right: 601 - National way of life (positive) 29 MP328 0.398 0.076 MVT 0.000 0.000 Right: 603 - Traditional morality (positive) MP328 0.000 0.000 MVT 0.000 0.000 Right: 605 - Law and order (positive) 30 MP328 4.035 1.325 MVT 4.250 1.768 Right: 606 - Social harmony (positive) 31 MP328 1.983 1.831 MVT 0.050 0.050 Left: 103 - Anti-imperialism (positive) 32 MP328 0.086 0.122 MVT 0.000 0.000 Left: 105 - Military (negative) MP328 0.000 0.000 MVT 0.000 0.000 Left: 106 - Peace (positive) MP328 0.172 0.243 MVT 0.000 0.000 Left: 107 - Internationalism (positive) MP328 1.598 0.661 MVT 1.650 0.071 Left: 202 - Democracy (positive) 33 MP328 4.274 0.771 MVT 0.900 0.990 Left: 403 - Market regulation (positive) MP328 0.909 0.313 MVT 1.550 2.192 Left: 404 - Economic planning (positive) MP328 0.568 0.316 MVT 0.000 0.000 Left: 406 - Protectionism (positive) MP328 0.229 0.163 MVT 0.350 0.350 Left: 412 - Controlled economy (positive) MP328 0.775 1.095 MVT 0.050 0.050 Left: 413 - Nationalization (positive) MP328 0.086 0.122 MVT 0.000 0.000 Left: 504 - Welfare state expansion (positive) MP328 9.154 2.235 MVT 12.450 5.162 Left: 506 - Education expansion (positive) MP328 7.887 0.287 MVT 6.750 2.192 Left: 701 - Labour groups (positive) MP328 3.306 0.680 MVT 0.000 0.000 Source: Krause et al. (2019) and MVT (2017) 2.1. Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Argentina edate party partyname partyabbrev rile MP Alcantara Saez 2012 (1-10) Oliveira 2015 VIdeo Huber et al. 2008 num PREPPS (1-20) Lopez et al 2013 D’Alessandro 2013 V-party 2020 (-5 - +5) 14/05/1989 150040 Center Alliance 13.287 4 -1.30 14/05/1989 150029 Justicialist Party of Popular Unity FREJUPO -3.82 4 -5.34 -0.688 14/05/1989 150301 Civic Radical Union UCR -19.416 3 -10.66 -0.148 14/05/1995 150202 Front for a Country in Solidarity FREPASO -15.217 2 -20.59 -1.198 14/05/1995 150201 Justicialist Party PJ 1.997 7.40 4 -6.27 1.174 14/05/1995 150301 Civic Radical Union UCR -11.136 5.63 3 -16.06 0.038 24/10/1999 150601 Action for the Republic AR 0.176 4 -2.10 2.251 24/10/1999 150028 Justicialist Coalition for Change -6.644 7.78 4 -4.84 0.825 24/10/1999 150032 Alliance for Work, Justice, and Education ALIANZA -13.04 5.57 0.23 3 -13.09 27/04/2003 150205 Front of the popular movement MP 4.167 -4.76 27/04/2003 150210 Alternative for a Republic of Equals ARI -26.135 -0.597 27/04/2003 150401 Recreate for Growth RECREAR -0.381 9.26 27/04/2003 150025 Front for Victory FpV -22.667 -0.17 -1.25 27/04/2003 150022 Front for Loyalty and Union of the Democratic Centre FPL + UCeDé -7.958 6.56 5.02 -0.405 28/10/2007 150027 Justice, Union and Liberty Front FREJULI -26.923 28/10/2007 150031 An Advanced Union UNA -13.213 -5.31 28/10/2007 150023 Civic Coalition CC -11.006 4.71 -0.269 28/10/2007 150025 Front for Victory FpV -20.755 4.79 -0.56 -10.71 -1.669 28/06/2009 150024 Social and Civic Agreement ACyS -20 28/06/2009 150220 New Encounter -16.346 28/06/2009 150221 Neuquén People’s Movement MPN -12.027 28/06/2009 150222 It is possible -18.957 28/06/2009 150025 Front for Victory FpV -40 -1.663 28/06/2009 150061 Alliance Union-PRO -18.023 1.537 23/10/2011 150027 Federal Commitment 1.245 43.5 23/10/2011 150230 Popular Front -50 29.8 23/10/2011 150026 Broad Progressive Front FAP -17.351 3.71 2.5 -19.91 -1.274 23/10/2011 150301 Civic Radical Union UCR -13.333 6.35 10.360 5.0 -21.56 0.197 23/10/2011 150025 Front for Victory FpV -3.425 6.37 -1.00 7.923 -1.4 -23.38 -1.677 27/10/2013 150026 Broad Progressive Front FAP -6.838 27/10/2013 150301 Civic Radical Union UCR -15.094 11.375 27/10/2013 150025 Front for Victory FpV 0 7.083 -1.654 Sources cited in Table 1. 2.2. Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Bolivia edate party partyname partyabbrev rile MP Alcantara Saez 2012 (1-10) Oliveira 2015 VIdeo PREPPS (1-20) V-party 2020 (-5 - +5) 06/12/2009 151255 Social Alliance AS 8.832 06/12/2009 151617 Autonomy for Bolivia APB -3.091 06/12/2009 151062 Plan Progress for Bolivia - National Convergence PPB-CN -38.356 2.253 06/12/2009 151610 Nationalist Revolutionary Movement MNR 11.64 8.28 06/12/2009 151640 National Unity Front UN -2.381 8.28 06/12/2009 151250 Movement towards Socialism - Political Instrument for the Sovereignty of the Peoples MAS-IPSP -13.401 4.17 -0.26 -2.192 12/10/2014 151640 Democratic Union UD -3.287 12/10/2014 151540 Christian Democratic Party PDC -14.107 17.667 1.34 12/10/2014 151250 Movement towards Socialism - Political Instrument for the Sovereignty of the Peoples MAS-IPSP -17.41 -0.73 4.900 -2.192 Sources cited in Table 1. 2.3. Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Brazil edate party partyname partyabbrev rile MP Zucco and Power 2019 (1-10) Alcantara Saez 2012 (1-10) Oliveira 2015 VIdeo Huber et al. 2008 num PREPPS (1-20) Gonzales y Queirolo 2013 num Lopez et al 2013 V-party 2020 (-5 - +5) 15/11/1989 180250 Democratic Labour Party PDT 2 15/11/1989 180610 Democratic Social Party PDS 5 15/11/1989 180410 New Brazil Movement 10.733 7.4 15/11/1989 180230 Popular Front Brazil -46.855 1.5 1 15/11/1989 180309 Party of Brazilian Social Democracy PSDB 32.836 3.9 2 03/10/1994 180710 Party of the Reconstruction of the National Order PRONA -27.333 03/10/1994 180230 A democratic revolution in Brazil -27.142 2.0 1 -2.609 03/10/1994 180310 Let’s work Brazil 4.843 4.4 0.28 0.37 04/10/1998 180232 Socialist People’s Party PPS -15.974 3.0 1 04/10/1998 180230 Union of the People - Change Brazil -19.905 1.9 1 -2.227 04/10/1998 180310 Go ahead Brazil -18.318 5.9 0.33 2 0.936 06/10/2002 180320 Labour Front Coalition -20.711 3.8 06/10/2002 180240 Hope for Brasil Coalition -12.53 2.9 -1.639 06/10/2002 180310 Coalition Big Alliance -10.523 6.1 4 0.881 06/10/2002 180230 Coalition Lula for President -21.955 2.2 -0.10 1 -1.437 01/10/2006 180231 Left Front -51.128 2.955 01/10/2006 180310 Coalition for a decent Brazil -9.202 5.9 6.89 13.462 1.071 01/10/2006 180230 Coalition the People’s Power -19.005 3.6 4.44 -0.29 6.370 -1.437 03/10/2010 180110 Green Party PV -28.302 4.6 8.158 03/10/2010 180310 Brazil can do more -11.302 5.8 7.13 12.000 40.4 1.071 03/10/2010 180230 For Brazil to keep on changing -13.636 3.6 4.73 0.00 5.429 13.1 -1.292 05/10/2014 180240 United for Brazil -5.394 4.1 -1.131 05/10/2014 180310 Change, Brazil -5.085 6.0 13.833 1.516 05/10/2014 180230 With the strength of the people -12.048 3.7 -0.22 6.222 -1.45 07/10/2018 180620 Brazil above everything, God above everyone 42.466 18.355 3.242 07/10/2018 180230 The People happy again -26.123 2.7 5.044 -1.897 07/10/2018 180251 Sovereign Brazil -20.298 3.8 6.661 -1.187 07/10/2018 180310 To unite Brazil 4.474 7.1 14.772 1.416 Sources cited in Table 1. 2.4. Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Chile edate party partyname partyabbrev rile MP Alcantara Saez 2012 (1-10) Oliveira 2015 VIdeo Huber et al 2008 num PREPPS (1-20) Gonzales y Queirolo 2013 num Lopez et al 2013 V-party 2020 (-5 - +5) 14/12/1989 155620 Liberal-Socialists of Chile -3.968 14/12/1989 155061 Democracy and Progress 21.405 5 2.656 14/12/1989 155021 Concert of Parties for Democracy -15.152 -0.33 3 0.348 11/12/1993 155021 Concert of Parties for Democracy -10.493 4.85 3 11/12/1993 155061 Union for Chile’s Progress 25.347 9.13 5 2.656 12/12/1999 155061 Alliance for Chile -5.017 9.58 5 12/12/1999 155021 Concert of Parties for Democracy -13.596 3.71 0.13 2 0.146 11/12/2005 155025 Together we can do more for Chile PODEMOS -36.323 11/12/2005 155021 Concert of Parties for Democracy -19.715 2.51 0.00 2 11/12/2005 155601 Independent Democratic Union UDI 16.892 9.53 18.100 4 2.656 11/12/2005 155602 National Renewal RN -0.809 7.70 16.333 5 1.67 13/12/2009 155023 New Majority for Chile -28.325 -0.3 13/12/2009 155025 Together we can do more for Chile PODEMOS -41.486 -21.6 13/12/2009 155021 Concert of Parties for Democracy -18.857 4.62 -7.7 13/12/2009 155061 Coalition for Change -3.029 7.78 0.42 8.7 17/11/2013 155024 If you want it, Chile changes -37.011 17/11/2013 155061 Alliance -28.011 17.333 17/11/2013 155021 New Majority for Chile/Socialist Party of Chile -18.974 -0.30 6.333 -1.403 19/11/2017 155501 Christian Democratic Party PDC -29.713 10.000 0.297 20/11/2017 155021 New Majority for Chile -24.938 21/11/2017 155026 Broad Front FA -39.175 22/11/2017 155061 Chile let’s go 3.958 23/11/2017 155201 Progressive Party PRO -31.919 24/11/2017 155980 Independent -2.332 Sources cited in Table 1. 2.5. Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Uruguay edate party partyname Party abbrev rile MP Alcantara Saez 2012 (1-10) Oliveira 2015 VIdeo PREPPS (1-20) V-party 2020 (-5 - +5) 26/10/2014 162210 Popular Assembly UP -43.399 1.750 26/10/2014 162510 Independent Party PI -0.077 9.250 26/10/2014 162410 Colorado Party PC -8.831 8.05 15.625 1.259 26/10/2014 162610 National Party PN -3.24 7.66 13.875 1.268 26/10/2014 162310 Broad Front FA -21.2 3.71 0.43 4.750 -1.354 Sources cited in Table 1. .

The main classification of Latin American parties is that developed by Coppedge (1997)COPPEDGE, Michael. A classification of Latin American political parties. Kellogg Institute Working Papers, n. 244, 1997. Disponível em: Disponível em: https://kellogg.nd.edu/sites/default/files/old_files/documents/244_0.pdf Acesso em: 20 fev. 2006.
https://kellogg.nd.edu/sites/default/fil...
based on information provided by country specialists. Here we use two versions of that classification, expanded and updated by Huber et al. (2008)HUBER, Evelyne et al. Politics and social spending in latin america. The Journal of Politics, v. 70, n. 2, p. 420-436, 2008. Disponível em: Disponível em: https://www.jstor.org/stable/10.1017/s0022381608080407 Acesso em: 28 set. 2019.
https://www.jstor.org/stable/10.1017/s00...
and by González Ferrer and Quierolo Velasco (2013)GONZÁLEZ FERRER, Luis Eduardo; QUIEROLO VELASCO, Rosario. Izquierda y derecha: formas de definirlas, el caso latinoamericano y sus implicaciones. América Latina Hoy, v. 65, p. 79-105, 2013.. The classification scheme comprises five categories in the left-right dimension: left, center-left, center, center-right, and right.6 6 The original classification in Coppedge (1997) had also a religious dimension with two categories (Christian and secular) that we do not use in our comparison. In order to compare it to the RILE scale, we converted the classification in an ordinal measure, where left corresponds to 1.0 and right takes the value of 5.0.7 7 Lorenzoni and Pérez (2013) have applied the MP method to measure ideological position of Frente Amplio, in Uruguay from 1971 to 2009. Unfortunately, MP dataset on Uruguay includes only manifestos from the 2014 election.

We also compared the RILE positions to other estimations of ideology: two surveys conducted with parliamentarian elites, two expert surveys, an alternative content analysis, and three content analysis using MP method. The first two estimations are provided by the PELA project (ALCÁNTARA SAEZ, 2012ALCÁNTARA SAEZ, Manuel. Elections in Latin America 2009-2011: a comparative analysis. Kellogg Institute Working Papers, n. 386, 2012. Disponível em: Disponível em: https://kellogg.nd.edu/sites/default/files/old_files/documents/386_0.pdf Acesso em: 17 fev. 2020.
https://kellogg.nd.edu/sites/default/fil...
))8 8 PELA - Proyecto Elites Lationoamericanas is a collection of surveys conducted with representatives in legislatures. Available at Universidad de Salamanca (https://oir.org.es/pela/) and by Zucco and Power (2019)ZUCCO, Cesar; POWER, Timothy. Brazilian legislative surveys (Waves 1-8, 1990-2017). Harvard Dataverse, V2, UNF:6:OZ4b3ybYKksC255DkruPxg== [fileUNF] 2019. Disponível em: Disponível em: https://dataverse.harvard.edu/dataverse/bls Acesso em:04 abr. 2020.
https://dataverse.harvard.edu/dataverse/...
9 9 Brazilian Legislative Survey uses ideological position in a scale from 1 (Left) to 10 (Right) in several waves: 1990, 1993, 1997, 2001, 2005, 2009, 2013, 2017. Here we use the average position in the nearest year. . The third one comes from the PREPPS10 10 PREPPS - The Political Representation, Executives, and Political Parties Survey is a collection of expert surveys, carried out in four waves. , published by Wiesehomeier (WIESEHOMEIER, 2011WIESEHOMEIER, Nina. Policy positions of 47 political parties and 4 presidents in 4 Latin American countries. In: PREPPS: The political representation, executives, and political parties survey: banco de dados [Internet]. 2011. Disponível em: Disponível em: http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA Acesso em: 10 abr. 2020.
http://ninaw.webfactional.com/prepps e e...
; WIESEHOMEIER, 2015WIESEHOMEIER, Nina. Policy positions of 165 political parties and 18 presidents in 18 Latin American countries. In: PREPPS: The political representation, executives, and political parties survey: banco de dados [Internet]. 2015. Disponível em: Disponível em: http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA Acesso em: 10 abr. 2020.
http://ninaw.webfactional.com/prepps e e...
; WIESEHOMEIER; BENOIT, 2006WIESEHOMEIER, Nina; BENOIT, K. Policy positions of 146 political parties and 18 presidents in 18 Latin American countries. In: PREPPS: The political representation, executives, and political parties survey: banco de dados [Internet]. 2006. Disponível em: Disponível em: http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA Acesso em: 10 abr. 2020.
http://ninaw.webfactional.com/prepps e e...
; WIESEHOMEIER; SINGER, 2019WIESEHOMEIER, Nina; SINGER, Mathew.; RUTH-LOVELL, Saskia. Political representation, executives, and political parties survey: Data from Expert Surveys in 18 Latin American Countries, 2018-2019. IE University, University of Connecticut and GIGA Institute Hamburg. 2019. Disponível em: Disponível em: https://dataverse.harvard.edu/dataverse/prepps . Acesso em: 23 abr. 2020.
https://dataverse.harvard.edu/dataverse/...
). The fourth one, also an expert survey, is the V-PARTY (LÜHRMANN et al. 2020LÜHRMANN, A et al. Varieties of Party Identity and Organization (V-Party) Dataset V1. PROJECT., V. o. D. V.-D. 2020. Disponível em: Disponível em: https://www.v-dem.net/en/data/data/v-party-dataset/ Acesso em: 08 jan. 2021.
https://www.v-dem.net/en/data/data/v-par...
).11 11 V-PARTY - The Varieties of Party Identity and Organization (LÜHRMANN et al., 2020) is a dataset produced by the V-Dem Institute. Based on its several indicators, we used the variable Economic left-right scale (v2pariglef), which aggregates experts’ ordinal answers in a standardized interval scale varying from -5 to +5 (PEMSTEIN et al., 2020). The fifth one creates an indicator of the economic ideology of presidents, built from dummy variables located in their platforms (OLIVEIRA, 2015OLIVEIRA, Augusto Neftali. A ideologia econômica dos presidentes na América Latina: um indicador a partir dos programas de governo eleitorais. Indicadores Econômicos FEE, v. 43, n. 1, p. 83-98, 2015.). The last two measures (D’ALESSANDRO, 2013D’ALESSANDRO, Martín. Las plataformas electorales en la Argentina moderna. América Latina Hoy, v. 65, p. 107-139, 2013. Disponível em: Disponível em: https://revistas.usal.es//index.php/1130-2887/article/view/alh201365107139 Acesso em: 10 fev. 2014
https://revistas.usal.es//index.php/1130...
; LÓPEZ; MIRANDA; VALENZUELA-GUTIÉRREZ, 2013LÓPEZ, Miguel Ángel; MIRANDA, Nicolás; VALENZUELA-GUTIÉRREZ, Pablo. Estimando el espacio politico del cono sur y Brasil: las elecciones presidenciales en el eje izquierda-derecha. PostData, v. 18, n. 2, p. 403-442, 2013. Disponível em: Disponível em: https://www.researchgate.net/publication/260837426_Estimando_el_espacio_politico_del_cono_sur_y_brasil_las_elecciones_presidenciales_en_el_eje_izquierda-derecha Acesso em: 16 jan. 2014.
https://www.researchgate.net/publication...
) are codings of Latin American political party platforms performed according to the MP methodology.

These sources analyzed a total of 96 documents, of which 18 are only present in the MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) dataset,12 12 The South America dataset has 98 lines, but two of them (documents from Brazilian PDT and PDS in 1989 election) are not coded. meaning that these documents cannot be compared with other sources.13 13 The matching between measurements is somewhat arbitrary in some cases in which the position in the alternative scale refers to the party in a legislature elected in a different year than the presidential manifesto. In these cases, we matched the manifesto with the nearest year possible. Graph 1 shows the number of parties whose ideology is measured by more than one source. The list of the 78 documents for which we have at least one alternative measure to compare is available in the appendix Appendix. 1a. Means in RILE categories, for four Brazilian documents of 2006 and 2010 elections, MP and MVT coders Category Coder Mean Std. Dev. Right: 104 - Military (positive) MP318 0.894 0.460 MVT 0.700 0.707 Right: 201 - Freedom and human rights (positive)20 MP318 1.901 1.442 MVT 0.650 0.480 Right: 203 - Constitutionalism (positive) MP318 0.252 0.291 MVT 0.000 0.000 Right: 305 - Political authority (positive)21 MP318 3.288 3.781 MVT 0.000 0.000 Right: 401 - Free enterprise (positive) MP318 0.223 0.281 MVT 0.050 0.050 Right: 402 - Economic Incentives (positive) MP318 2.212 0.885 MVT 3.275 2.205 Right: 407 - Protectionism (negative) MP318 0.427 0.070 MVT 0.000 0.000 Right: 414 - Economic orthodoxy (positive) MP318 1.918 0.618 MVT 0.325 0.650 Right: 505 - Welfare state limitation (positive) MP318 0.103 0.206 MVT 0.000 0.000 Right: 601 - National way of life (positive)22 MP318 0.974 0.876 MVT 0.000 0.000 Right: 603 - Traditional morality (positive) MP318 0.165 0.227 MVT 0.000 0.000 Right: 605 - Law and order (positive)23 MP318 4.076 0.273 MVT 4.650 1.303 Right: 606 - Social harmony (positive)24 MP318 0.937 0.713 MVT 0.050 0.050 Left: 103 - Anti-imperialism (positive)25 MP318 1.187 1.276 MVT 0.000 0.000 Left: 105 - Military (negative) MP318 0.031 0.062 MVT 0.000 0.000 Left: 106 - Peace (positive) MP318 0.527 0.700 MVT 0.400 0.616 Left: 107 - Internationalism (positive) MP318 1.873 1.688 MVT 2.025 1.323 Left: 202 - Democracy (positive)26 MP318 5.289 3.253 MVT 1.800 1.804 Left: 403 - Market regulation (positive) MP318 1.809 1.000 MVT 0.025 0.050 Left: 404 - Economic planning (positive) MP318 1.127 0.287 MVT 0.475 0.591 Left: 406 - Protectionism (positive) MP318 0.166 0.247 MVT 0.425 0.506 Left: 412 - Controlled economy (positive) MP318 0.407 0.303 MVT 0.000 0.000 Left: 413 - Nationalization (positive) MP318 0.245 0.093 MVT 0.000 0.000 Left: 504 - Welfare state expansion (positive) MP318 9.518 1.316 MVT 11.450 5.500 Left: 506 - Education expansion (positive) MP318 5.737 0.813 MVT 6.300 1.503 Left: 701 - Labour groups (positive) MP318 2.639 0.962 MVT 0.950 0.929 Source: Krause et al. (2019) and MVT (2017) 1b. Means in RILE categories, for two Brazilian documents of 2014 election, MP and MVT coders Category Coder Mean Std. Dev. Right: 104 - Military (positive) MP328 0.565 0.799 MVT 0.050 0.050 Right: 201 - Freedom and human rights (positive) 27 MP328 1.369 0.498 MVT 1.700 2.121 Right: 203 - Constitutionalism (positive) MP328 0.000 0.000 MVT 0.000 0.000 Right: 305 - Political authority (positive)28 MP328 3.349 3.297 MVT 0.000 0.000 Right: 401 - Free enterprise (positive) MP328 1.499 1.876 MVT 0.000 0.000 Right: 402 - Economic Incentives (positive) MP328 4.557 0.371 MVT 5.750 1.061 Right: 407 - Protectionism (negative) MP328 0.283 0.400 MVT 0.000 0.000 Right: 414 - Economic orthodoxy (positive) MP328 1.988 0.864 MVT 0.150 0.150 Right: 505 - Welfare state limitation (positive) MP328 0.452 0.639 MVT 0.000 0.000 Right: 601 - National way of life (positive) 29 MP328 0.398 0.076 MVT 0.000 0.000 Right: 603 - Traditional morality (positive) MP328 0.000 0.000 MVT 0.000 0.000 Right: 605 - Law and order (positive) 30 MP328 4.035 1.325 MVT 4.250 1.768 Right: 606 - Social harmony (positive) 31 MP328 1.983 1.831 MVT 0.050 0.050 Left: 103 - Anti-imperialism (positive) 32 MP328 0.086 0.122 MVT 0.000 0.000 Left: 105 - Military (negative) MP328 0.000 0.000 MVT 0.000 0.000 Left: 106 - Peace (positive) MP328 0.172 0.243 MVT 0.000 0.000 Left: 107 - Internationalism (positive) MP328 1.598 0.661 MVT 1.650 0.071 Left: 202 - Democracy (positive) 33 MP328 4.274 0.771 MVT 0.900 0.990 Left: 403 - Market regulation (positive) MP328 0.909 0.313 MVT 1.550 2.192 Left: 404 - Economic planning (positive) MP328 0.568 0.316 MVT 0.000 0.000 Left: 406 - Protectionism (positive) MP328 0.229 0.163 MVT 0.350 0.350 Left: 412 - Controlled economy (positive) MP328 0.775 1.095 MVT 0.050 0.050 Left: 413 - Nationalization (positive) MP328 0.086 0.122 MVT 0.000 0.000 Left: 504 - Welfare state expansion (positive) MP328 9.154 2.235 MVT 12.450 5.162 Left: 506 - Education expansion (positive) MP328 7.887 0.287 MVT 6.750 2.192 Left: 701 - Labour groups (positive) MP328 3.306 0.680 MVT 0.000 0.000 Source: Krause et al. (2019) and MVT (2017) 2.1. Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Argentina edate party partyname partyabbrev rile MP Alcantara Saez 2012 (1-10) Oliveira 2015 VIdeo Huber et al. 2008 num PREPPS (1-20) Lopez et al 2013 D’Alessandro 2013 V-party 2020 (-5 - +5) 14/05/1989 150040 Center Alliance 13.287 4 -1.30 14/05/1989 150029 Justicialist Party of Popular Unity FREJUPO -3.82 4 -5.34 -0.688 14/05/1989 150301 Civic Radical Union UCR -19.416 3 -10.66 -0.148 14/05/1995 150202 Front for a Country in Solidarity FREPASO -15.217 2 -20.59 -1.198 14/05/1995 150201 Justicialist Party PJ 1.997 7.40 4 -6.27 1.174 14/05/1995 150301 Civic Radical Union UCR -11.136 5.63 3 -16.06 0.038 24/10/1999 150601 Action for the Republic AR 0.176 4 -2.10 2.251 24/10/1999 150028 Justicialist Coalition for Change -6.644 7.78 4 -4.84 0.825 24/10/1999 150032 Alliance for Work, Justice, and Education ALIANZA -13.04 5.57 0.23 3 -13.09 27/04/2003 150205 Front of the popular movement MP 4.167 -4.76 27/04/2003 150210 Alternative for a Republic of Equals ARI -26.135 -0.597 27/04/2003 150401 Recreate for Growth RECREAR -0.381 9.26 27/04/2003 150025 Front for Victory FpV -22.667 -0.17 -1.25 27/04/2003 150022 Front for Loyalty and Union of the Democratic Centre FPL + UCeDé -7.958 6.56 5.02 -0.405 28/10/2007 150027 Justice, Union and Liberty Front FREJULI -26.923 28/10/2007 150031 An Advanced Union UNA -13.213 -5.31 28/10/2007 150023 Civic Coalition CC -11.006 4.71 -0.269 28/10/2007 150025 Front for Victory FpV -20.755 4.79 -0.56 -10.71 -1.669 28/06/2009 150024 Social and Civic Agreement ACyS -20 28/06/2009 150220 New Encounter -16.346 28/06/2009 150221 Neuquén People’s Movement MPN -12.027 28/06/2009 150222 It is possible -18.957 28/06/2009 150025 Front for Victory FpV -40 -1.663 28/06/2009 150061 Alliance Union-PRO -18.023 1.537 23/10/2011 150027 Federal Commitment 1.245 43.5 23/10/2011 150230 Popular Front -50 29.8 23/10/2011 150026 Broad Progressive Front FAP -17.351 3.71 2.5 -19.91 -1.274 23/10/2011 150301 Civic Radical Union UCR -13.333 6.35 10.360 5.0 -21.56 0.197 23/10/2011 150025 Front for Victory FpV -3.425 6.37 -1.00 7.923 -1.4 -23.38 -1.677 27/10/2013 150026 Broad Progressive Front FAP -6.838 27/10/2013 150301 Civic Radical Union UCR -15.094 11.375 27/10/2013 150025 Front for Victory FpV 0 7.083 -1.654 Sources cited in Table 1. 2.2. Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Bolivia edate party partyname partyabbrev rile MP Alcantara Saez 2012 (1-10) Oliveira 2015 VIdeo PREPPS (1-20) V-party 2020 (-5 - +5) 06/12/2009 151255 Social Alliance AS 8.832 06/12/2009 151617 Autonomy for Bolivia APB -3.091 06/12/2009 151062 Plan Progress for Bolivia - National Convergence PPB-CN -38.356 2.253 06/12/2009 151610 Nationalist Revolutionary Movement MNR 11.64 8.28 06/12/2009 151640 National Unity Front UN -2.381 8.28 06/12/2009 151250 Movement towards Socialism - Political Instrument for the Sovereignty of the Peoples MAS-IPSP -13.401 4.17 -0.26 -2.192 12/10/2014 151640 Democratic Union UD -3.287 12/10/2014 151540 Christian Democratic Party PDC -14.107 17.667 1.34 12/10/2014 151250 Movement towards Socialism - Political Instrument for the Sovereignty of the Peoples MAS-IPSP -17.41 -0.73 4.900 -2.192 Sources cited in Table 1. 2.3. Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Brazil edate party partyname partyabbrev rile MP Zucco and Power 2019 (1-10) Alcantara Saez 2012 (1-10) Oliveira 2015 VIdeo Huber et al. 2008 num PREPPS (1-20) Gonzales y Queirolo 2013 num Lopez et al 2013 V-party 2020 (-5 - +5) 15/11/1989 180250 Democratic Labour Party PDT 2 15/11/1989 180610 Democratic Social Party PDS 5 15/11/1989 180410 New Brazil Movement 10.733 7.4 15/11/1989 180230 Popular Front Brazil -46.855 1.5 1 15/11/1989 180309 Party of Brazilian Social Democracy PSDB 32.836 3.9 2 03/10/1994 180710 Party of the Reconstruction of the National Order PRONA -27.333 03/10/1994 180230 A democratic revolution in Brazil -27.142 2.0 1 -2.609 03/10/1994 180310 Let’s work Brazil 4.843 4.4 0.28 0.37 04/10/1998 180232 Socialist People’s Party PPS -15.974 3.0 1 04/10/1998 180230 Union of the People - Change Brazil -19.905 1.9 1 -2.227 04/10/1998 180310 Go ahead Brazil -18.318 5.9 0.33 2 0.936 06/10/2002 180320 Labour Front Coalition -20.711 3.8 06/10/2002 180240 Hope for Brasil Coalition -12.53 2.9 -1.639 06/10/2002 180310 Coalition Big Alliance -10.523 6.1 4 0.881 06/10/2002 180230 Coalition Lula for President -21.955 2.2 -0.10 1 -1.437 01/10/2006 180231 Left Front -51.128 2.955 01/10/2006 180310 Coalition for a decent Brazil -9.202 5.9 6.89 13.462 1.071 01/10/2006 180230 Coalition the People’s Power -19.005 3.6 4.44 -0.29 6.370 -1.437 03/10/2010 180110 Green Party PV -28.302 4.6 8.158 03/10/2010 180310 Brazil can do more -11.302 5.8 7.13 12.000 40.4 1.071 03/10/2010 180230 For Brazil to keep on changing -13.636 3.6 4.73 0.00 5.429 13.1 -1.292 05/10/2014 180240 United for Brazil -5.394 4.1 -1.131 05/10/2014 180310 Change, Brazil -5.085 6.0 13.833 1.516 05/10/2014 180230 With the strength of the people -12.048 3.7 -0.22 6.222 -1.45 07/10/2018 180620 Brazil above everything, God above everyone 42.466 18.355 3.242 07/10/2018 180230 The People happy again -26.123 2.7 5.044 -1.897 07/10/2018 180251 Sovereign Brazil -20.298 3.8 6.661 -1.187 07/10/2018 180310 To unite Brazil 4.474 7.1 14.772 1.416 Sources cited in Table 1. 2.4. Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Chile edate party partyname partyabbrev rile MP Alcantara Saez 2012 (1-10) Oliveira 2015 VIdeo Huber et al 2008 num PREPPS (1-20) Gonzales y Queirolo 2013 num Lopez et al 2013 V-party 2020 (-5 - +5) 14/12/1989 155620 Liberal-Socialists of Chile -3.968 14/12/1989 155061 Democracy and Progress 21.405 5 2.656 14/12/1989 155021 Concert of Parties for Democracy -15.152 -0.33 3 0.348 11/12/1993 155021 Concert of Parties for Democracy -10.493 4.85 3 11/12/1993 155061 Union for Chile’s Progress 25.347 9.13 5 2.656 12/12/1999 155061 Alliance for Chile -5.017 9.58 5 12/12/1999 155021 Concert of Parties for Democracy -13.596 3.71 0.13 2 0.146 11/12/2005 155025 Together we can do more for Chile PODEMOS -36.323 11/12/2005 155021 Concert of Parties for Democracy -19.715 2.51 0.00 2 11/12/2005 155601 Independent Democratic Union UDI 16.892 9.53 18.100 4 2.656 11/12/2005 155602 National Renewal RN -0.809 7.70 16.333 5 1.67 13/12/2009 155023 New Majority for Chile -28.325 -0.3 13/12/2009 155025 Together we can do more for Chile PODEMOS -41.486 -21.6 13/12/2009 155021 Concert of Parties for Democracy -18.857 4.62 -7.7 13/12/2009 155061 Coalition for Change -3.029 7.78 0.42 8.7 17/11/2013 155024 If you want it, Chile changes -37.011 17/11/2013 155061 Alliance -28.011 17.333 17/11/2013 155021 New Majority for Chile/Socialist Party of Chile -18.974 -0.30 6.333 -1.403 19/11/2017 155501 Christian Democratic Party PDC -29.713 10.000 0.297 20/11/2017 155021 New Majority for Chile -24.938 21/11/2017 155026 Broad Front FA -39.175 22/11/2017 155061 Chile let’s go 3.958 23/11/2017 155201 Progressive Party PRO -31.919 24/11/2017 155980 Independent -2.332 Sources cited in Table 1. 2.5. Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Uruguay edate party partyname Party abbrev rile MP Alcantara Saez 2012 (1-10) Oliveira 2015 VIdeo PREPPS (1-20) V-party 2020 (-5 - +5) 26/10/2014 162210 Popular Assembly UP -43.399 1.750 26/10/2014 162510 Independent Party PI -0.077 9.250 26/10/2014 162410 Colorado Party PC -8.831 8.05 15.625 1.259 26/10/2014 162610 National Party PN -3.24 7.66 13.875 1.268 26/10/2014 162310 Broad Front FA -21.2 3.71 0.43 4.750 -1.354 Sources cited in Table 1. . The correlation between the RILE scale and each one of the other five measures is shown in table 1.

Graph 1.
Frequency of measures available for each party.

Table 1.
Correlation among measures of ideology

We can see that the RILE scale has a strong and significant correlation with two other measures: Alcántara Saez (2012)ALCÁNTARA SAEZ, Manuel. Elections in Latin America 2009-2011: a comparative analysis. Kellogg Institute Working Papers, n. 386, 2012. Disponível em: Disponível em: https://kellogg.nd.edu/sites/default/files/old_files/documents/386_0.pdf Acesso em: 17 fev. 2020.
https://kellogg.nd.edu/sites/default/fil...
and Huber et al. (2008)HUBER, Evelyne et al. Politics and social spending in latin america. The Journal of Politics, v. 70, n. 2, p. 420-436, 2008. Disponível em: Disponível em: https://www.jstor.org/stable/10.1017/s0022381608080407 Acesso em: 28 set. 2019.
https://www.jstor.org/stable/10.1017/s00...
. This result indicates that the ideological position of Latin American parties does not vary much across assessments. Legislators and experts provide measures very similar to those computed from manifestos.

On the other hand, it is noteworthy that both measures of the RILE scale provided by Latin American independent coders D’Alessandro (2013)D’ALESSANDRO, Martín. Las plataformas electorales en la Argentina moderna. América Latina Hoy, v. 65, p. 107-139, 2013. Disponível em: Disponível em: https://revistas.usal.es//index.php/1130-2887/article/view/alh201365107139 Acesso em: 10 fev. 2014
https://revistas.usal.es//index.php/1130...
and López; Miranda; Valenzuela-Gutiérrez (2013)LÓPEZ, Miguel Ángel; MIRANDA, Nicolás; VALENZUELA-GUTIÉRREZ, Pablo. Estimando el espacio politico del cono sur y Brasil: las elecciones presidenciales en el eje izquierda-derecha. PostData, v. 18, n. 2, p. 403-442, 2013. Disponível em: Disponível em: https://www.researchgate.net/publication/260837426_Estimando_el_espacio_politico_del_cono_sur_y_brasil_las_elecciones_presidenciales_en_el_eje_izquierda-derecha Acesso em: 16 jan. 2014.
https://www.researchgate.net/publication...
, which applied the same MP categories and protocol, do not correlate to the MP Rile scale at all, as lines 8 and 9 in Table 1 and Graph 2 show. Both, however, are highly correlated to other measures, although their small n hinders statistical significance. This disparity suggests a coder bias not resolvable by the MP protocol.

Graph 2.
Independent coders and MP RILE measures

Another possible cause of the weak correlations is the presumable ambiguity of center parties. As Graph 3 shows, most of MP RILE measures concentrate in the center of the scale, between -30,0 and +20,0, in a scale from -100 to +100. Several alternative assessments spread their measures more widely from left to right.

Graph 3.
Histogram of positions at RILE scale

Unfortunately, as Table 2 shows, most measures are available only for a small number of parties, preventing us from performing a factor analysis (which would be a great contribution to our knowledge of ideology in Latin America).

Table 2.
Number of measures in each pair of sources (n for the correlation tests)

The comparison between distinct measures of party ideology in Latin America allows us to draw three important conclusions. First, some of these measures (Alcántara Saez 2012ALCÁNTARA SAEZ, Manuel. Elections in Latin America 2009-2011: a comparative analysis. Kellogg Institute Working Papers, n. 386, 2012. Disponível em: Disponível em: https://kellogg.nd.edu/sites/default/files/old_files/documents/386_0.pdf Acesso em: 17 fev. 2020.
https://kellogg.nd.edu/sites/default/fil...
; Huber et al., 2008HUBER, Evelyne et al. Politics and social spending in latin america. The Journal of Politics, v. 70, n. 2, p. 420-436, 2008. Disponível em: Disponível em: https://www.jstor.org/stable/10.1017/s0022381608080407 Acesso em: 28 set. 2019.
https://www.jstor.org/stable/10.1017/s00...
) seem to capture the same latent variable as MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
), whatever label we assign it. Second, the three applications of the MP protocol return very distinct measures of party positions at the RILE scale. Third, three measures are highly and statistically correlated to one another (BLS, PREPPS, and V-Party), suggesting that we do have good measures of ideology validated across distinct methods (legislators and expert surveys), but they do not include manifestos content analyses.

These findings have two implications. The first is methodological: survey methods produce quite similar ideological positions, regardless of who applies them, but manifesto content analysis does not. The second implication is theoretical: across the distinct faces of parties (KATZ; MAIR, 1994KATZ, Richard; MAIR, Peter. How parties organize: change and adaptation in party organizations in western democracies. London: Sage, 1994.), legislators (who belong to the party in public office) and experts (who can also consider the ideology of the party on the ground and in the central office) may identify them in the same way.

Case study: Brazilian PT and PSDB manifestos

Before the MP launched its South America dataset, its method was already being applied to Brazilian political party manifestos (TAROUCO, 2011TAROUCO, Gabriela. Brazilian parties according to their manifestos: political identity and programmatic emphases. Brazilian Political Science Review v. 5, n. 1, p. 54-76, 2011. Disponível em: Disponível em: https://www.redalyc.org/pdf/3943/394341997003.pdf Acesso em: 25 fev. 2012.
https://www.redalyc.org/pdf/3943/3943419...
) and programs for Brazilian presidential elections (MADEIRA; VIEIRA; TAROUCO, 2017MADEIRA, Rafael, VIEIRA, Soraia; TAROUCO, Gabriela. Agendas, preferências, competição: PT e PSDB em disputas presidenciais. Caderno CRH, v. 30, n. 80, p. 257-273, 2017. ). Following MP instructions (VOLKENS et al., 2013VOLKENS, A. et al. The manifesto data collection. manifesto project (MRG/CMP/MARPOR). Version 2013b. Berlin, 2013. Disponível em: Disponível em: https://manifesto-project.wzb.eu/ . Acesso em: 23 ago. 2015.
https://manifesto-project.wzb.eu/...
), we coded (MADEIRA; VIEIRA; TAROUCO, 2017MADEIRA, Rafael, VIEIRA, Soraia; TAROUCO, Gabriela. Agendas, preferências, competição: PT e PSDB em disputas presidenciais. Caderno CRH, v. 30, n. 80, p. 257-273, 2017. ) six documents issued for the 2006, 2010 and 2014 elections by presidential candidates of the Workers’ Party (Partido dos Trabalhadores - PT) and the Brazilian Social Democracy Party (Partido da Social Democracia Brasileira - PSDB). These parties polarized political competition at the presidential level for many years, and their candidates occupied the federal executive office from 1994 to 2016. None of us received MP training, nor were we submitted to its tests or hired by the project. This was an independent initiative.

The first step of hand-coded content analysis of party manifestos is the identification of the coding units. According to MP coding instructions (VOLKENS et al., 2013VOLKENS, A. et al. The manifesto data collection. manifesto project (MRG/CMP/MARPOR). Version 2013b. Berlin, 2013. Disponível em: Disponível em: https://manifesto-project.wzb.eu/ . Acesso em: 23 ago. 2015.
https://manifesto-project.wzb.eu/...
), the coding unit is each sentence segment that contains a single argument - the quasi-sentence. The quantity of identified quasi-sentences is the “n” of each document (the number of observations of units of analysis). Despite the detailed instructions on how to identify quasi-sentences, this procedure appeared to be very susceptible to coder bias. The fact that one coder can identify fewer, bigger quasi-sentences while another splits the same document into more, shorter units is revealing.

This initial procedure, when performed by distinct coders, results in diverging numbers of coding units, thus affecting the quantification of the policy content of each document and preventing further tests for inter-coder reliability. As table 3 shows, MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) coders systematically identified more and shorter units than we did.14 14 During the segmentation step, we teamed up to consolidate our individual segmentations in a final, unified one.

Table 3.
Number of units (quasi-sentences) identified by the two teams

The next step in the analysis is the classification of each unit according to the 56 categories in the MP coding frame. These categories constitute seven domains: external relations, freedom and democracy, political system, economy, welfare and quality of life, fabric of society, and social groups.

Despite detailed instructions and decision criteria provided in the MP coding manual, it is possible that distinct coders classify a same unit into different categories. This is the primary source of reliability problems in hand-coded content analysis. We conducted double-blind coding and checked our classifications against each other through the Kappa test of inter-coder reliability (COHEN, 1960COHEN, Jacob. A coefficient of agreement for nominal scales. Educational and Psychological Measurement, XX, n. 1, p. 37-46, 1960.), reaching high scores of agreement (MADEIRA; VIEIRA; TAROUCO, 2017MADEIRA, Rafael, VIEIRA, Soraia; TAROUCO, Gabriela. Agendas, preferências, competição: PT e PSDB em disputas presidenciais. Caderno CRH, v. 30, n. 80, p. 257-273, 2017. ).15 15 The values of the kappa index for the three pairs of MVT coders are 0.88 (TxV), 0.68 (VxM), and 0.66 (MxT). According to Landis and Koch (1977) and Bonnardel (2001), kappa values equal or greater than 0.81 indicate excellent agreement, and values between 0.61 and 0.80 are good. The MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) coder, instead, worked by herself, after taking the MP training and having passed a pre-hiring test of reliability. Unfortunately, the distinct set of quasi-sentences resulting from the two distinct unitizing processes prevents a reliability test between MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) and MVT coding results.

An additional source of divergence is that we adopted the original standard scheme of 56 categories (BUDGE et al., 2001BUDGE, Ian et al. Mapping policy preferences: estimates for parties, electors, and governments 1945-1998. New York: Oxford University Press, 2001. 274 p. ), adding a set of specific sub-categories (ALONSO; VOLKENS; GÓMEZ, 2012ALONSO, Sonia; VOLKENS, Andrea; GÓMEZ, Braulio. Análisis de contenido de textos políticos. Un enfoque cuantitativo. Madrid: Centro de Investigaciones Sociologicas, 2012.).16 16 These subcategories were created in order to accommodate the specificities of new democracies, like Eastern and Central European countries and Mexico. They were then used to analyze Latin American party manifestos. The MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) coders, instead, used a new set of sub-categories, added in the last version of the coding manual (WERNER; LACEWELL; VOLKENS, 2014WERNER, A.; LACEWELL, O.; VOLKENS, A. Manifesto coding instructions. (5th revised edition), 2014. Disponível em: Disponível em: https://manifesto-project.wzb.eu/information/documents/handbooks . Acesso em 23 ago. 2015.
https://manifesto-project.wzb.eu/informa...
).

Fortunately, the distinct sets of subcategories do not affect their clustering within domains, whose proportions in the documents can, then, be compared. Graphs 4 and 5 show that results agree on locating both PT and PSDB documents within a shared public agenda: economy and welfare issues concentrate most of the space in their programs for presidential elections in Brazil, from 2006 to 2014, according to the two coding results. Moreover, in both cases, the political system is the third priority category. It is also worth mentioning that the other domains are far less frequently used by both parties as observed in the two classifications (about 10%). The only exception is the uncoded units of PT’s 2010 manifesto by MVT.

Graph 4.
Proportion of domains according to Madeira et al. (2017)MADEIRA, Rafael, VIEIRA, Soraia; TAROUCO, Gabriela. Agendas, preferências, competição: PT e PSDB em disputas presidenciais. Caderno CRH, v. 30, n. 80, p. 257-273, 2017. .

Graph 5.
Proportion of domains according to Manifesto Project

The differences are mostly related to categories in the “Freedom and Democracy” domain. It is also worth noting the disagreement between the two coder teams regarding the proportion of text that does not fit into any category. Uncoded sentences are much more frequent in the MVT coding than in the MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) coding. This is important because it illustrates how individual interpretations and personal bias can compromise the reliability in this kind of analysis.17 17 This difference may be related to a supposed emphasis that MP instructors put, while training the coders, on avoiding the ‘uncoded’ category, as we were recently warned about. Table 4 shows the differences of means for each domain in the full set of documents according to the analysis of each coder team.

Table 4.
Means of domain proportions in two coding results

The main difference occurs in the proportion of domain 2. The MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) team identified many more statements on freedom and democracy than the MVT coders in the same set of texts.

The classifications for each sentence are not comparable between the two coding results because the sentences are not equal, as we said above. However, we can compare the proportions attributed by each team to each category in each document. Here we adapt two indicators of proportionality often used in institutional studies, in order to measure the congruence between the two coding results at the level of categories.

The first measure is the index of coalescence (AMORIM NETO, 2000AMORIM NETO, Octavio. Gabinetes presidenciais, ciclos eleitorais e disciplina legislativa no Brasil. Dados, v. 43, n. 3, 2000. Disponível em: Disponível em: https://www.scielo.br/j/dados/a/3xyyXsZD8NdvxFcSLJjbKrN/?lang=pt Acesso em: 24 nov. 2021
https://www.scielo.br/j/dados/a/3xyyXsZD...
) that is used to measure how much the distribution of portfolios in coalition governments is proportional to party seat share in the legislature. The index varies from 0 (no correspondence) to 1 (perfect correspondence). For our purposes, values near one in the coalescence index indicate a high similarity between the two coding results.

The second measure is an adaptation of the index of disproportionality (GALLAGHER, 1991GALLAGHER, Michael. Proportionality, disproportionality and electoral systems. Electoral Studies, v. 10, n. 1, p. 33-51, 1991.) used to estimate how much the assignment of legislative seats is proportional to the vote share of each party. That original index varies from 0 (perfectly proportional) to 100 (totally disproportional). For our purposes, values near to zero in the index of disproportionality indicate good similarity between the two coding results. Thus, we computed our index of proportionality by subtracting the original one from 100. To fit better in the graphs, we divided this index by 100, so it varies from 0 to 1, from the least to the most proportional.

We used both measures of congruence as a sort of alternative reliability test. A document’s two coding results are considered to be more congruent when both indexes (coalescence and proportionality) are high. We argue that congruent coding results could support a claim of reliability of the method even when the coding units are not the same. We calculated both indices for three distinct proportions: the proportion of each category in the entire set of sentences; the proportion of each category in the set of sentences coded with a valid category (excluding uncoded ones); and the proportion of each domain. Table 5 shows both measures of congruence between the two coding results for the six Brazilian documents.

Table 5.
Measures of congruence between the two coding results

Not surprisingly, for every document coding that we compared, the congruence is greater for domains than for categories. The exclusion of uncoded sentences also improves both measures of congruence. It is interesting to note that the highest and lowest congruences are found in programs for the same election. In 2010, the PSDB’s program was coded by the two teams with the most similar proportion of categories, while the PT’s program showed the most distinct proportions. Graph 6 shows how the congruence between the two codings varies among documents and over time.

Graph 6.
Measures of congruence between two coding results

The congruence between proportions of categories (excluded the uncoded) in each of the two measures is rather appreciable. The index of proportionality is always higher than 0.85, thus much closer to one than to zero. The index of coalescence is always higher than 0.5, reaching 0.66 in 2010. If the proportions of categories are not so different, we should expect to find similar results for the scales built on those categories. The next section describes the disparities in the left-right positions.

Implications: comparing ideological positions

Up to this point, we have discussed the reliability of the method of text unit coding. Seemingly, the hand-coded content analysis of party manifestos produces quite different results when applied by distinct coders, even when they follow the same rules. However, the greater or lesser relevance of these disparities depends on their implications for substantive analysis. In this section, we try to validate MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) coding results by comparing the positions in the RILE scale to MVT coding of Brazilian manifestos.

The MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) database encompasses the RILE scale, a left-right scale based on a specific set of categories.18 18 Laver and Budge (1992) built the RILE scale using a series of exploratory factors to identify possible combinations of variables that would reflect either the left or the right position. A manifesto position in the scale is computed by deducting the percentage of left categories from the percentage of the right categories. The RILE scale varies from -100 (if all units were coded with left categories) to +100 (if the whole text was coded with right categories).

Table 6.
Categories in the RILE scale

The availability of two codings (MP and MVT) of the same six Brazilian documents enables us to compare each manifesto’s values on that scale. Graph 7 shows both measures for the six documents.

Graph 7.
RILE scale, according to the two coding results

Graph 7 shows how distinct are the RILE scale positions resulting from the two codings. Although the two measures locate all documents in the left field (negative values in the RILE scale), there is an important difference in PT programs’ positions for the 2006 and 2010 elections. While the RILE scale applied to the MP coding shows the party moving seven points from left to right over time, the same scale applied to the Brazilian coding shows PT’s return in 2014 to practically the same position of 2006, after a deepening of its left position in 2010. As for the PSDB documents, the main difference is also in the change from 2006 to 2010: despite the similarity between the MP and MVT measures for 2006, the MVT scale shows a much deeper displacement toward left than the MP scale.

The scores in the RILE scale measured for the two codings are more distant in some documents (PSDB 2014, PSDB 2010, and PT 2006) than in the others. However, both measures result in changes over time in the same direction. The PSDB and PT movements toward the center from 2010 to 2014 are detected in both assessments. The same happens to the PSDB, toward left from 2006 to 2010. The only divergence in direction is the PT’s movement from 2006 to 2010.

The distinct measures of RILE positions have implications on how we analyze the Brazilian party system dynamics. Although both measures point to similar positions for the 2014 PT program, the contrasting results for its positions in 2006 lead to opposite interpretations about ideological changes and competitive strategies in the presidential elections in Brazil. MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) assessments would support interpretations that PT has consistently softened its left platform during its presidential mandates. Alternatively, the MVT measures would support interpretations about the ideological convergence of the two main parties in Brazilian presidential elections within a bipolar mode of competition.

The divergences do not seem to be related to the coders. While the MVT team kept the same three coders for all elections and documents, the MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) coder was replaced for the 2014 documents’ analysis. However, there are considerable divergences in the coding of documents of all three elections.

So, where the differences come from? We searched for different proportions of RILE categories between the MP and MVT coders. While the same MVT coders coded the six documents, in MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) , two different coders worked on different documents. They are labeled in the MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) dataset as coders 318 and 328, and worked on the four documents of 2006 and 2010 elections and on the two documents of 2014 election, respectively. Since the MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) coders worked on different documents, we cannot compare them to each other. Tables showing the differences of means between coders for each category of the RILE scale are available in the appendix Appendix. 1a. Means in RILE categories, for four Brazilian documents of 2006 and 2010 elections, MP and MVT coders Category Coder Mean Std. Dev. Right: 104 - Military (positive) MP318 0.894 0.460 MVT 0.700 0.707 Right: 201 - Freedom and human rights (positive)20 MP318 1.901 1.442 MVT 0.650 0.480 Right: 203 - Constitutionalism (positive) MP318 0.252 0.291 MVT 0.000 0.000 Right: 305 - Political authority (positive)21 MP318 3.288 3.781 MVT 0.000 0.000 Right: 401 - Free enterprise (positive) MP318 0.223 0.281 MVT 0.050 0.050 Right: 402 - Economic Incentives (positive) MP318 2.212 0.885 MVT 3.275 2.205 Right: 407 - Protectionism (negative) MP318 0.427 0.070 MVT 0.000 0.000 Right: 414 - Economic orthodoxy (positive) MP318 1.918 0.618 MVT 0.325 0.650 Right: 505 - Welfare state limitation (positive) MP318 0.103 0.206 MVT 0.000 0.000 Right: 601 - National way of life (positive)22 MP318 0.974 0.876 MVT 0.000 0.000 Right: 603 - Traditional morality (positive) MP318 0.165 0.227 MVT 0.000 0.000 Right: 605 - Law and order (positive)23 MP318 4.076 0.273 MVT 4.650 1.303 Right: 606 - Social harmony (positive)24 MP318 0.937 0.713 MVT 0.050 0.050 Left: 103 - Anti-imperialism (positive)25 MP318 1.187 1.276 MVT 0.000 0.000 Left: 105 - Military (negative) MP318 0.031 0.062 MVT 0.000 0.000 Left: 106 - Peace (positive) MP318 0.527 0.700 MVT 0.400 0.616 Left: 107 - Internationalism (positive) MP318 1.873 1.688 MVT 2.025 1.323 Left: 202 - Democracy (positive)26 MP318 5.289 3.253 MVT 1.800 1.804 Left: 403 - Market regulation (positive) MP318 1.809 1.000 MVT 0.025 0.050 Left: 404 - Economic planning (positive) MP318 1.127 0.287 MVT 0.475 0.591 Left: 406 - Protectionism (positive) MP318 0.166 0.247 MVT 0.425 0.506 Left: 412 - Controlled economy (positive) MP318 0.407 0.303 MVT 0.000 0.000 Left: 413 - Nationalization (positive) MP318 0.245 0.093 MVT 0.000 0.000 Left: 504 - Welfare state expansion (positive) MP318 9.518 1.316 MVT 11.450 5.500 Left: 506 - Education expansion (positive) MP318 5.737 0.813 MVT 6.300 1.503 Left: 701 - Labour groups (positive) MP318 2.639 0.962 MVT 0.950 0.929 Source: Krause et al. (2019) and MVT (2017) 1b. Means in RILE categories, for two Brazilian documents of 2014 election, MP and MVT coders Category Coder Mean Std. Dev. Right: 104 - Military (positive) MP328 0.565 0.799 MVT 0.050 0.050 Right: 201 - Freedom and human rights (positive) 27 MP328 1.369 0.498 MVT 1.700 2.121 Right: 203 - Constitutionalism (positive) MP328 0.000 0.000 MVT 0.000 0.000 Right: 305 - Political authority (positive)28 MP328 3.349 3.297 MVT 0.000 0.000 Right: 401 - Free enterprise (positive) MP328 1.499 1.876 MVT 0.000 0.000 Right: 402 - Economic Incentives (positive) MP328 4.557 0.371 MVT 5.750 1.061 Right: 407 - Protectionism (negative) MP328 0.283 0.400 MVT 0.000 0.000 Right: 414 - Economic orthodoxy (positive) MP328 1.988 0.864 MVT 0.150 0.150 Right: 505 - Welfare state limitation (positive) MP328 0.452 0.639 MVT 0.000 0.000 Right: 601 - National way of life (positive) 29 MP328 0.398 0.076 MVT 0.000 0.000 Right: 603 - Traditional morality (positive) MP328 0.000 0.000 MVT 0.000 0.000 Right: 605 - Law and order (positive) 30 MP328 4.035 1.325 MVT 4.250 1.768 Right: 606 - Social harmony (positive) 31 MP328 1.983 1.831 MVT 0.050 0.050 Left: 103 - Anti-imperialism (positive) 32 MP328 0.086 0.122 MVT 0.000 0.000 Left: 105 - Military (negative) MP328 0.000 0.000 MVT 0.000 0.000 Left: 106 - Peace (positive) MP328 0.172 0.243 MVT 0.000 0.000 Left: 107 - Internationalism (positive) MP328 1.598 0.661 MVT 1.650 0.071 Left: 202 - Democracy (positive) 33 MP328 4.274 0.771 MVT 0.900 0.990 Left: 403 - Market regulation (positive) MP328 0.909 0.313 MVT 1.550 2.192 Left: 404 - Economic planning (positive) MP328 0.568 0.316 MVT 0.000 0.000 Left: 406 - Protectionism (positive) MP328 0.229 0.163 MVT 0.350 0.350 Left: 412 - Controlled economy (positive) MP328 0.775 1.095 MVT 0.050 0.050 Left: 413 - Nationalization (positive) MP328 0.086 0.122 MVT 0.000 0.000 Left: 504 - Welfare state expansion (positive) MP328 9.154 2.235 MVT 12.450 5.162 Left: 506 - Education expansion (positive) MP328 7.887 0.287 MVT 6.750 2.192 Left: 701 - Labour groups (positive) MP328 3.306 0.680 MVT 0.000 0.000 Source: Krause et al. (2019) and MVT (2017) 2.1. Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Argentina edate party partyname partyabbrev rile MP Alcantara Saez 2012 (1-10) Oliveira 2015 VIdeo Huber et al. 2008 num PREPPS (1-20) Lopez et al 2013 D’Alessandro 2013 V-party 2020 (-5 - +5) 14/05/1989 150040 Center Alliance 13.287 4 -1.30 14/05/1989 150029 Justicialist Party of Popular Unity FREJUPO -3.82 4 -5.34 -0.688 14/05/1989 150301 Civic Radical Union UCR -19.416 3 -10.66 -0.148 14/05/1995 150202 Front for a Country in Solidarity FREPASO -15.217 2 -20.59 -1.198 14/05/1995 150201 Justicialist Party PJ 1.997 7.40 4 -6.27 1.174 14/05/1995 150301 Civic Radical Union UCR -11.136 5.63 3 -16.06 0.038 24/10/1999 150601 Action for the Republic AR 0.176 4 -2.10 2.251 24/10/1999 150028 Justicialist Coalition for Change -6.644 7.78 4 -4.84 0.825 24/10/1999 150032 Alliance for Work, Justice, and Education ALIANZA -13.04 5.57 0.23 3 -13.09 27/04/2003 150205 Front of the popular movement MP 4.167 -4.76 27/04/2003 150210 Alternative for a Republic of Equals ARI -26.135 -0.597 27/04/2003 150401 Recreate for Growth RECREAR -0.381 9.26 27/04/2003 150025 Front for Victory FpV -22.667 -0.17 -1.25 27/04/2003 150022 Front for Loyalty and Union of the Democratic Centre FPL + UCeDé -7.958 6.56 5.02 -0.405 28/10/2007 150027 Justice, Union and Liberty Front FREJULI -26.923 28/10/2007 150031 An Advanced Union UNA -13.213 -5.31 28/10/2007 150023 Civic Coalition CC -11.006 4.71 -0.269 28/10/2007 150025 Front for Victory FpV -20.755 4.79 -0.56 -10.71 -1.669 28/06/2009 150024 Social and Civic Agreement ACyS -20 28/06/2009 150220 New Encounter -16.346 28/06/2009 150221 Neuquén People’s Movement MPN -12.027 28/06/2009 150222 It is possible -18.957 28/06/2009 150025 Front for Victory FpV -40 -1.663 28/06/2009 150061 Alliance Union-PRO -18.023 1.537 23/10/2011 150027 Federal Commitment 1.245 43.5 23/10/2011 150230 Popular Front -50 29.8 23/10/2011 150026 Broad Progressive Front FAP -17.351 3.71 2.5 -19.91 -1.274 23/10/2011 150301 Civic Radical Union UCR -13.333 6.35 10.360 5.0 -21.56 0.197 23/10/2011 150025 Front for Victory FpV -3.425 6.37 -1.00 7.923 -1.4 -23.38 -1.677 27/10/2013 150026 Broad Progressive Front FAP -6.838 27/10/2013 150301 Civic Radical Union UCR -15.094 11.375 27/10/2013 150025 Front for Victory FpV 0 7.083 -1.654 Sources cited in Table 1. 2.2. Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Bolivia edate party partyname partyabbrev rile MP Alcantara Saez 2012 (1-10) Oliveira 2015 VIdeo PREPPS (1-20) V-party 2020 (-5 - +5) 06/12/2009 151255 Social Alliance AS 8.832 06/12/2009 151617 Autonomy for Bolivia APB -3.091 06/12/2009 151062 Plan Progress for Bolivia - National Convergence PPB-CN -38.356 2.253 06/12/2009 151610 Nationalist Revolutionary Movement MNR 11.64 8.28 06/12/2009 151640 National Unity Front UN -2.381 8.28 06/12/2009 151250 Movement towards Socialism - Political Instrument for the Sovereignty of the Peoples MAS-IPSP -13.401 4.17 -0.26 -2.192 12/10/2014 151640 Democratic Union UD -3.287 12/10/2014 151540 Christian Democratic Party PDC -14.107 17.667 1.34 12/10/2014 151250 Movement towards Socialism - Political Instrument for the Sovereignty of the Peoples MAS-IPSP -17.41 -0.73 4.900 -2.192 Sources cited in Table 1. 2.3. Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Brazil edate party partyname partyabbrev rile MP Zucco and Power 2019 (1-10) Alcantara Saez 2012 (1-10) Oliveira 2015 VIdeo Huber et al. 2008 num PREPPS (1-20) Gonzales y Queirolo 2013 num Lopez et al 2013 V-party 2020 (-5 - +5) 15/11/1989 180250 Democratic Labour Party PDT 2 15/11/1989 180610 Democratic Social Party PDS 5 15/11/1989 180410 New Brazil Movement 10.733 7.4 15/11/1989 180230 Popular Front Brazil -46.855 1.5 1 15/11/1989 180309 Party of Brazilian Social Democracy PSDB 32.836 3.9 2 03/10/1994 180710 Party of the Reconstruction of the National Order PRONA -27.333 03/10/1994 180230 A democratic revolution in Brazil -27.142 2.0 1 -2.609 03/10/1994 180310 Let’s work Brazil 4.843 4.4 0.28 0.37 04/10/1998 180232 Socialist People’s Party PPS -15.974 3.0 1 04/10/1998 180230 Union of the People - Change Brazil -19.905 1.9 1 -2.227 04/10/1998 180310 Go ahead Brazil -18.318 5.9 0.33 2 0.936 06/10/2002 180320 Labour Front Coalition -20.711 3.8 06/10/2002 180240 Hope for Brasil Coalition -12.53 2.9 -1.639 06/10/2002 180310 Coalition Big Alliance -10.523 6.1 4 0.881 06/10/2002 180230 Coalition Lula for President -21.955 2.2 -0.10 1 -1.437 01/10/2006 180231 Left Front -51.128 2.955 01/10/2006 180310 Coalition for a decent Brazil -9.202 5.9 6.89 13.462 1.071 01/10/2006 180230 Coalition the People’s Power -19.005 3.6 4.44 -0.29 6.370 -1.437 03/10/2010 180110 Green Party PV -28.302 4.6 8.158 03/10/2010 180310 Brazil can do more -11.302 5.8 7.13 12.000 40.4 1.071 03/10/2010 180230 For Brazil to keep on changing -13.636 3.6 4.73 0.00 5.429 13.1 -1.292 05/10/2014 180240 United for Brazil -5.394 4.1 -1.131 05/10/2014 180310 Change, Brazil -5.085 6.0 13.833 1.516 05/10/2014 180230 With the strength of the people -12.048 3.7 -0.22 6.222 -1.45 07/10/2018 180620 Brazil above everything, God above everyone 42.466 18.355 3.242 07/10/2018 180230 The People happy again -26.123 2.7 5.044 -1.897 07/10/2018 180251 Sovereign Brazil -20.298 3.8 6.661 -1.187 07/10/2018 180310 To unite Brazil 4.474 7.1 14.772 1.416 Sources cited in Table 1. 2.4. Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Chile edate party partyname partyabbrev rile MP Alcantara Saez 2012 (1-10) Oliveira 2015 VIdeo Huber et al 2008 num PREPPS (1-20) Gonzales y Queirolo 2013 num Lopez et al 2013 V-party 2020 (-5 - +5) 14/12/1989 155620 Liberal-Socialists of Chile -3.968 14/12/1989 155061 Democracy and Progress 21.405 5 2.656 14/12/1989 155021 Concert of Parties for Democracy -15.152 -0.33 3 0.348 11/12/1993 155021 Concert of Parties for Democracy -10.493 4.85 3 11/12/1993 155061 Union for Chile’s Progress 25.347 9.13 5 2.656 12/12/1999 155061 Alliance for Chile -5.017 9.58 5 12/12/1999 155021 Concert of Parties for Democracy -13.596 3.71 0.13 2 0.146 11/12/2005 155025 Together we can do more for Chile PODEMOS -36.323 11/12/2005 155021 Concert of Parties for Democracy -19.715 2.51 0.00 2 11/12/2005 155601 Independent Democratic Union UDI 16.892 9.53 18.100 4 2.656 11/12/2005 155602 National Renewal RN -0.809 7.70 16.333 5 1.67 13/12/2009 155023 New Majority for Chile -28.325 -0.3 13/12/2009 155025 Together we can do more for Chile PODEMOS -41.486 -21.6 13/12/2009 155021 Concert of Parties for Democracy -18.857 4.62 -7.7 13/12/2009 155061 Coalition for Change -3.029 7.78 0.42 8.7 17/11/2013 155024 If you want it, Chile changes -37.011 17/11/2013 155061 Alliance -28.011 17.333 17/11/2013 155021 New Majority for Chile/Socialist Party of Chile -18.974 -0.30 6.333 -1.403 19/11/2017 155501 Christian Democratic Party PDC -29.713 10.000 0.297 20/11/2017 155021 New Majority for Chile -24.938 21/11/2017 155026 Broad Front FA -39.175 22/11/2017 155061 Chile let’s go 3.958 23/11/2017 155201 Progressive Party PRO -31.919 24/11/2017 155980 Independent -2.332 Sources cited in Table 1. 2.5. Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Uruguay edate party partyname Party abbrev rile MP Alcantara Saez 2012 (1-10) Oliveira 2015 VIdeo PREPPS (1-20) V-party 2020 (-5 - +5) 26/10/2014 162210 Popular Assembly UP -43.399 1.750 26/10/2014 162510 Independent Party PI -0.077 9.250 26/10/2014 162410 Colorado Party PC -8.831 8.05 15.625 1.259 26/10/2014 162610 National Party PN -3.24 7.66 13.875 1.268 26/10/2014 162310 Broad Front FA -21.2 3.71 0.43 4.750 -1.354 Sources cited in Table 1. .

Another aspect of this difference is that, although there is room for positions ranging from -100 to +100, in fact, the distance between the PT and the PSDB is small (their positions vary between -5 and -20) according to the two assessments, showing that the manifestos effectively occupy a narrow range of the left-right dimension.

From the 26 categories in the RILE scale, MP and MVT teams found very different proportions of four right categories and four left categories. MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) coders identified left-right content in the documents systematically more often than the MVT team, whose means in these categories are always smaller, or even zero in several cases.

Differences reach more categories when we compare MVT codings to those conducted by MP318 than when compared to codings by MP328. This is possibly related to the differences showed in graph 7, since MP318 coded 2006 and 2010 documents, exactly those for which the values in the RILE scale showed major differences between MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) and MVT calculations.

The category that generates the main differences between MVT and both MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) coders is “701 - Labour groups”, defined as “Favorable references to all labor groups, the working class, and unemployed workers in general. Support for trade unions and calls for the good treatment of all employees, including: More jobs; Good working conditions; Fair wages; Pension provisions etc.” (WERNER; LACEWELL; VOLKENS, 2014WERNER, A.; LACEWELL, O.; VOLKENS, A. Manifesto coding instructions. (5th revised edition), 2014. Disponível em: Disponível em: https://manifesto-project.wzb.eu/information/documents/handbooks . Acesso em 23 ago. 2015.
https://manifesto-project.wzb.eu/informa...
). This category is used much more often by the two MP coders than by the MVT team. Unfortunately, we cannot compare measured parties’ position between MP coders, since they did not code the same documents.

The comparison between the scales resulting from the two independent codings of Brazilian manifestos leads us to question their adoption as proxies for parties’ ideology. The Brazilian case is an example of the challenges posed by hand-coded content analysis. Coders express disparate interpretations, and, therefore, the scales computed from the distinct codings show distinct ideological positions for the same documents. Discrepant measures of ideology cause discrepant interpretations of the party competition. Having only one coding of each document (as is the case for the MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) measures) hinders us from knowing the magnitude of that discrepancy.

What makes the analysis of the manifestos comparable (JORGE; FARIA; SILVA, 2020JORGE, Vladymir Lombardo; FARIA, Alessandra Maia Terra de; SILVA, Mayra Goulart da. Posicionamento dos partidos políticos brasileiros na escala esquerda-direita: dilemas metodológicos e revisão da literatura. Revista Brasileira de Ciência Política, n. 33, 2020. Disponível em: Disponível em: https://www.scielo.br/j/rbcpol/a/XNBnwhWwbSsMPFrj4zmHQsG/?lang=pt Acesso em: 20 jan. 2021.
https://www.scielo.br/j/rbcpol/a/XNBnwhW...
) is also what allows us to grasp its fragility: the coding protocol published and openly available to any researcher. Following the same techniques and categories used by MP, we found in Brazil, years before the launching of their dataset for South America, different proportions of contents in the same texts.19 29 Comprises subcategories 601_1 and 601_2 used by MP coders. Other authors also reached distinct results in the analysis of Latin American manifestos (D’ALESSANDRO, 2013D’ALESSANDRO, Martín. Las plataformas electorales en la Argentina moderna. América Latina Hoy, v. 65, p. 107-139, 2013. Disponível em: Disponível em: https://revistas.usal.es//index.php/1130-2887/article/view/alh201365107139 Acesso em: 10 fev. 2014
https://revistas.usal.es//index.php/1130...
; LÓPEZ; MIRANDA; VALENZUELA-GUTIÉRREZ, 2013LÓPEZ, Miguel Ángel; MIRANDA, Nicolás; VALENZUELA-GUTIÉRREZ, Pablo. Estimando el espacio politico del cono sur y Brasil: las elecciones presidenciales en el eje izquierda-derecha. PostData, v. 18, n. 2, p. 403-442, 2013. Disponível em: Disponível em: https://www.researchgate.net/publication/260837426_Estimando_el_espacio_politico_del_cono_sur_y_brasil_las_elecciones_presidenciales_en_el_eje_izquierda-derecha Acesso em: 16 jan. 2014.
https://www.researchgate.net/publication...
). We ascribe these differences to the known weaknesses of the method: the unavoidable bias of human coders. Bias does not belittle the hard work of so many MP coders, nor is it exclusive to this method. However, we argue that the bias must be identified- and reduced - through intercoder reliability tests.

Beyond these methodological concerns, we hope to shed some light on the debate about concepts of left and right. The weak correlations among some distinct measures in Latin American data make us wonder about the different contents being measured. For instance, in Latin America, measures of ideological positions regarding economic issues (OLIVEIRA, 2015OLIVEIRA, Augusto Neftali. A ideologia econômica dos presidentes na América Latina: um indicador a partir dos programas de governo eleitorais. Indicadores Econômicos FEE, v. 43, n. 1, p. 83-98, 2015.) do not match either experts’ surveys measures (WIESEHOMEIER, 2011WIESEHOMEIER, Nina. Policy positions of 47 political parties and 4 presidents in 4 Latin American countries. In: PREPPS: The political representation, executives, and political parties survey: banco de dados [Internet]. 2011. Disponível em: Disponível em: http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA Acesso em: 10 abr. 2020.
http://ninaw.webfactional.com/prepps e e...
; WIESEHOMEIER, 2015WIESEHOMEIER, Nina. Policy positions of 165 political parties and 18 presidents in 18 Latin American countries. In: PREPPS: The political representation, executives, and political parties survey: banco de dados [Internet]. 2015. Disponível em: Disponível em: http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA Acesso em: 10 abr. 2020.
http://ninaw.webfactional.com/prepps e e...
; WIESEHOMEIER; BENOIT, 2006WIESEHOMEIER, Nina; BENOIT, K. Policy positions of 146 political parties and 18 presidents in 18 Latin American countries. In: PREPPS: The political representation, executives, and political parties survey: banco de dados [Internet]. 2006. Disponível em: Disponível em: http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA Acesso em: 10 abr. 2020.
http://ninaw.webfactional.com/prepps e e...
; WIESEHOMEIER; SINGER, 2019WIESEHOMEIER, Nina; SINGER, Mathew.; RUTH-LOVELL, Saskia. Political representation, executives, and political parties survey: Data from Expert Surveys in 18 Latin American Countries, 2018-2019. IE University, University of Connecticut and GIGA Institute Hamburg. 2019. Disponível em: Disponível em: https://dataverse.harvard.edu/dataverse/prepps . Acesso em: 23 abr. 2020.
https://dataverse.harvard.edu/dataverse/...
) or that of the RILE scale (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
; D’ALESSANDRO, 2013D’ALESSANDRO, Martín. Las plataformas electorales en la Argentina moderna. América Latina Hoy, v. 65, p. 107-139, 2013. Disponível em: Disponível em: https://revistas.usal.es//index.php/1130-2887/article/view/alh201365107139 Acesso em: 10 fev. 2014
https://revistas.usal.es//index.php/1130...
; LÓPEZ; MIRANDA; VALENZUELA-GUTIÉRREZ, 2013LÓPEZ, Miguel Ángel; MIRANDA, Nicolás; VALENZUELA-GUTIÉRREZ, Pablo. Estimando el espacio politico del cono sur y Brasil: las elecciones presidenciales en el eje izquierda-derecha. PostData, v. 18, n. 2, p. 403-442, 2013. Disponível em: Disponível em: https://www.researchgate.net/publication/260837426_Estimando_el_espacio_politico_del_cono_sur_y_brasil_las_elecciones_presidenciales_en_el_eje_izquierda-derecha Acesso em: 16 jan. 2014.
https://www.researchgate.net/publication...
). So, if we need to mobilize political party ideology as a variable, the first challenge before choosing a measure is how to define it.

Unlike the MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) measures, other measures validate each other. The surprisingly high correlation between two of our compared measures is worth mentioning. Measures based on an expert survey (WIESEHOMEIER, 2011WIESEHOMEIER, Nina. Policy positions of 47 political parties and 4 presidents in 4 Latin American countries. In: PREPPS: The political representation, executives, and political parties survey: banco de dados [Internet]. 2011. Disponível em: Disponível em: http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA Acesso em: 10 abr. 2020.
http://ninaw.webfactional.com/prepps e e...
; WIESEHOMEIER, 2015WIESEHOMEIER, Nina. Policy positions of 165 political parties and 18 presidents in 18 Latin American countries. In: PREPPS: The political representation, executives, and political parties survey: banco de dados [Internet]. 2015. Disponível em: Disponível em: http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA Acesso em: 10 abr. 2020.
http://ninaw.webfactional.com/prepps e e...
; WIESEHOMEIER; BENOIT, 2006WIESEHOMEIER, Nina; BENOIT, K. Policy positions of 146 political parties and 18 presidents in 18 Latin American countries. In: PREPPS: The political representation, executives, and political parties survey: banco de dados [Internet]. 2006. Disponível em: Disponível em: http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA Acesso em: 10 abr. 2020.
http://ninaw.webfactional.com/prepps e e...
; WIESEHOMEIER; SINGER, 2019WIESEHOMEIER, Nina; SINGER, Mathew.; RUTH-LOVELL, Saskia. Political representation, executives, and political parties survey: Data from Expert Surveys in 18 Latin American Countries, 2018-2019. IE University, University of Connecticut and GIGA Institute Hamburg. 2019. Disponível em: Disponível em: https://dataverse.harvard.edu/dataverse/prepps . Acesso em: 23 abr. 2020.
https://dataverse.harvard.edu/dataverse/...
) and a parliamentary survey (ZUCCO; POWER, 2019ZUCCO, Cesar; POWER, Timothy. Brazilian legislative surveys (Waves 1-8, 1990-2017). Harvard Dataverse, V2, UNF:6:OZ4b3ybYKksC255DkruPxg== [fileUNF] 2019. Disponível em: Disponível em: https://dataverse.harvard.edu/dataverse/bls Acesso em:04 abr. 2020.
https://dataverse.harvard.edu/dataverse/...
) are almost the same for 10 Brazilian political parties. In both cases, Brazilian respondents (both political scientists and legislators respectively) were simply asked to locate parties on the left-right scale, with no explanation about its meaning.

Concluding remarks and research agenda

When we think of ways to analyze ideologies and classify ideological positions, we must always bear in mind the fact that ideology cannot be directly observed. There is no concrete metric or measure directly observable and capable of accounting for (and exhausting) the ideological classification of any political agent (leaders, parties, governments etc.). In addition to these difficulties, the ideological framework is a complex phenomenon that involves different dimensions of reality, such as: classifications based on agents’ actions (what they do effectively), classifications based on agents’ speech (what they say/affirm) and classifications based on their reputation (how others see a particular agent and locate her on the ideological spectrum). We assume that all these dimensions are important, and that none of them exhaust the topic.

That said, the most fruitful strategy for scholars, in our view, is not to discuss the best methodology or approach, or which one better explains this phenomenon. Rather, it is to superimpose and to compare the different strategies for measuring ideological positioning, so as to produce knowledge from the contrasts and approaches identified in the different analyses. The literature provides many strategies that attempt to measure ideological positioning. Each has advantages and disadvantages.

Examining and analyzing the rich (and complex) mosaic formed by the overlap of these studies is as important as analyzing and discussing the scope and limitations of each strategy by itself. Starting from very different points (and taking distinct paths), each approach presents us with the delimitation of one of the (numerous) manifestations of ideology as a political phenomenon.

Hand-coded content analysis of manifestos is very promising, but reliability is not its leading feature. The disparities that we found suggest that: 1) there is room for improvement of manual coding of manifestos content; 2) caution is needed when drawing conclusions about ideology based on the scales constructed with the codifications.

Although we acknowledge the relevance of the content analysis of manifestos, the question is how to seize the advantages of the hand-coded procedures and minimize their problems. Human coding allows for quantifying qualitative data and deductively describing political positions expressed in texts in terms of theoretically built categories. Human coders can identify meanings and interpret complex writings. Hand-coded analysis may feed dictionaries for automated analysis of other materials, that would be too numerous or too long to be coded by hand.

However, hand-coded analysis is vulnerable to human limitations and bias. In addition to different physical and mental capacities, distinct coders often disagree about what they read. Because of that, the reliability of the method depends on several challenges. The first is the quality of the coding scheme, coding instructions, and coding criteria. Another challenge is the evaluation of the coding results and, eventually, the revision of the coding work.

MP does an excellent job on the first challenge but can improve on the second one. Without inter-coder reliability tests for each document, the application of the data is vulnerable to a level of error that we cannot estimate. This is what we found by comparing MP measures to other ones.

Adopting inter-coder reliability tests for each document would, of course, make the process more expensive and slower, but would assure that we can seize the advantages of the method with more confidence. Keeping only one coder for each document certainly improves the viability of such an ambitious project, but we are not sure that the results are worthwhile. Mistaken measures of political positions could compromise the study of their effects, for example. Speed and volume are advantages that we should seek in automated analysis, not in hand-coded analysis.

We are aware of the great value of the MP project contribution and of the vast comparative possibilities it opens to political science (JORGE; FARIA; SILVA, 2020JORGE, Vladymir Lombardo; FARIA, Alessandra Maia Terra de; SILVA, Mayra Goulart da. Posicionamento dos partidos políticos brasileiros na escala esquerda-direita: dilemas metodológicos e revisão da literatura. Revista Brasileira de Ciência Política, n. 33, 2020. Disponível em: Disponível em: https://www.scielo.br/j/rbcpol/a/XNBnwhWwbSsMPFrj4zmHQsG/?lang=pt Acesso em: 20 jan. 2021.
https://www.scielo.br/j/rbcpol/a/XNBnwhW...
). Even so, we believe that, as for any other proxy, hand-coded analysis of manifestos must be carefully used, taking its limitations into account. With this paper we expect to contribute to advance knowledge, by pointing out those limitations, at least with regard to Latin American parties. Regarding the research agenda, the main task for future research is to compare the MP coding results to a computer-based analysis, to measure its strength vis-a-vis a non-human classification.

References

  • ALCÁNTARA SAEZ, Manuel. Elections in Latin America 2009-2011: a comparative analysis. Kellogg Institute Working Papers, n. 386, 2012. Disponível em: Disponível em: https://kellogg.nd.edu/sites/default/files/old_files/documents/386_0.pdf Acesso em: 17 fev. 2020.
    » https://kellogg.nd.edu/sites/default/files/old_files/documents/386_0.pdf
  • ALONSO, Sonia; VOLKENS, Andrea; GÓMEZ, Braulio. Análisis de contenido de textos políticos. Un enfoque cuantitativo. Madrid: Centro de Investigaciones Sociologicas, 2012.
  • AMORIM NETO, Octavio. Gabinetes presidenciais, ciclos eleitorais e disciplina legislativa no Brasil. Dados, v. 43, n. 3, 2000. Disponível em: Disponível em: https://www.scielo.br/j/dados/a/3xyyXsZD8NdvxFcSLJjbKrN/?lang=pt Acesso em: 24 nov. 2021
    » https://www.scielo.br/j/dados/a/3xyyXsZD8NdvxFcSLJjbKrN/?lang=pt
  • BONNARDEL, Philippe. The Kappa coefficient: The measurement of interrater agreement when the ratings are on categorical scales. The case of two raters. Le Coefficient Kappa. 2001. Disponível em: Disponível em: http://kappa.chez-alice.fr/kappa.txt Acesso em: 24 nov. 2011.
    » http://kappa.chez-alice.fr/kappa.txt
  • BUDGE, Ian et al. Mapping policy preferences: estimates for parties, electors, and governments 1945-1998. New York: Oxford University Press, 2001. 274 p.
  • BUDGE, Ian; MEYER, Thomas. Understanding and validating the left-right scale (RILE). In: VOLKENS, Andrea; BARA, Judith; BUDGE, Ian et al. (ed.). Mapping policy preferences from texts: statistical solutions for manifesto analysts. Oxford: Oxford University Press, 2013.
  • COHEN, Jacob. A coefficient of agreement for nominal scales. Educational and Psychological Measurement, XX, n. 1, p. 37-46, 1960.
  • COPPEDGE, Michael. A classification of Latin American political parties. Kellogg Institute Working Papers, n. 244, 1997. Disponível em: Disponível em: https://kellogg.nd.edu/sites/default/files/old_files/documents/244_0.pdf Acesso em: 20 fev. 2006.
    » https://kellogg.nd.edu/sites/default/files/old_files/documents/244_0.pdf
  • D’ALESSANDRO, Martín. Las plataformas electorales en la Argentina moderna. América Latina Hoy, v. 65, p. 107-139, 2013. Disponível em: Disponível em: https://revistas.usal.es//index.php/1130-2887/article/view/alh201365107139 Acesso em: 10 fev. 2014
    » https://revistas.usal.es//index.php/1130-2887/article/view/alh201365107139
  • GALLAGHER, Michael. Proportionality, disproportionality and electoral systems. Electoral Studies, v. 10, n. 1, p. 33-51, 1991.
  • GONZÁLEZ FERRER, Luis Eduardo; QUIEROLO VELASCO, Rosario. Izquierda y derecha: formas de definirlas, el caso latinoamericano y sus implicaciones. América Latina Hoy, v. 65, p. 79-105, 2013.
  • HUBER, Evelyne et al. Politics and social spending in latin america. The Journal of Politics, v. 70, n. 2, p. 420-436, 2008. Disponível em: Disponível em: https://www.jstor.org/stable/10.1017/s0022381608080407 Acesso em: 28 set. 2019.
    » https://www.jstor.org/stable/10.1017/s0022381608080407
  • JORGE, Vladymir Lombardo; FARIA, Alessandra Maia Terra de; SILVA, Mayra Goulart da. Posicionamento dos partidos políticos brasileiros na escala esquerda-direita: dilemas metodológicos e revisão da literatura. Revista Brasileira de Ciência Política, n. 33, 2020. Disponível em: Disponível em: https://www.scielo.br/j/rbcpol/a/XNBnwhWwbSsMPFrj4zmHQsG/?lang=pt Acesso em: 20 jan. 2021.
    » https://www.scielo.br/j/rbcpol/a/XNBnwhWwbSsMPFrj4zmHQsG/?lang=pt
  • KATZ, Richard; MAIR, Peter. How parties organize: change and adaptation in party organizations in western democracies. London: Sage, 1994.
  • KRAUSE, W. et al. Manifesto Corpus. Version: 2018-2. WZB Berlin Social Science Center, 2018. Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2018b Acesso em: 16 out. 2019.
    » https://doi.org/10.25522/manifesto.mpdssa.2018b
  • KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b Acesso em: 26 dez. 2020.
    » https://doi.org/10.25522/manifesto.mpdssa.2019b
  • LANDIS, J. R.; KOCH, G. G. The measurement of observer agreement for categorical data. Biometrics, v. 33, n. 1, p. 159-174, 1977.
  • LAVER, M.; BUDGE, I. Party policy and government coalitions. Nova Iorque: St. Martin’s Press, 1992.
  • LÓPEZ, Miguel Ángel; MIRANDA, Nicolás; VALENZUELA-GUTIÉRREZ, Pablo. Estimando el espacio politico del cono sur y Brasil: las elecciones presidenciales en el eje izquierda-derecha. PostData, v. 18, n. 2, p. 403-442, 2013. Disponível em: Disponível em: https://www.researchgate.net/publication/260837426_Estimando_el_espacio_politico_del_cono_sur_y_brasil_las_elecciones_presidenciales_en_el_eje_izquierda-derecha Acesso em: 16 jan. 2014.
    » https://www.researchgate.net/publication/260837426_Estimando_el_espacio_politico_del_cono_sur_y_brasil_las_elecciones_presidenciales_en_el_eje_izquierda-derecha
  • LORENZONI, Miguel; PÉREZ, Verónica. Cambios y continuidades de la izquierda en Uruguay: un análisis a partir de las propuestas programáticas del Frente Amplio 1971-2009. Revista Uruguaya de Ciencia Política, v. 22, n. 1, p. 61-81, 2013.
  • LÜHRMANN, A et al. Varieties of Party Identity and Organization (V-Party) Dataset V1. PROJECT., V. o. D. V.-D. 2020. Disponível em: Disponível em: https://www.v-dem.net/en/data/data/v-party-dataset/ Acesso em: 08 jan. 2021.
    » https://www.v-dem.net/en/data/data/v-party-dataset/
  • MADEIRA, Rafael, VIEIRA, Soraia; TAROUCO, Gabriela. Agendas, preferências, competição: PT e PSDB em disputas presidenciais. Caderno CRH, v. 30, n. 80, p. 257-273, 2017.
  • MP - The Manifesto Project/MARPOR (Manifesto Research on Political Representation): bancos de dados [Internet]. Berlin. German Science Foundation (DFG). 1979 --. Disponível em: Disponível em: https://manifesto-project.wzb.eu/ Acesso em: 17 out. 2021.
    » https://manifesto-project.wzb.eu/
  • OLIVEIRA, Augusto Neftali. A ideologia econômica dos presidentes na América Latina: um indicador a partir dos programas de governo eleitorais. Indicadores Econômicos FEE, v. 43, n. 1, p. 83-98, 2015.
  • PEMSTEIN, Dan et al. The V-Dem Measurement Model: latent variable analysis for cross-national and cross-temporal expert-coded data. V-Dem working paper series, n. 21, 2020.
  • TAROUCO, Gabriela. Brazilian parties according to their manifestos: political identity and programmatic emphases. Brazilian Political Science Review v. 5, n. 1, p. 54-76, 2011. Disponível em: Disponível em: https://www.redalyc.org/pdf/3943/394341997003.pdf Acesso em: 25 fev. 2012.
    » https://www.redalyc.org/pdf/3943/394341997003.pdf
  • TAROUCO, Gabriela; MADEIRA, Rafael. Esquerda e direita no sistema partidário brasileiro: análise de conteúdo de documentos programáticos. Revista Debates, v. 7, n. 2, Porto Alegre, p. 93-114, 2013.
  • VOLKENS et al. Scope, range, and extent of manifesto project data usage: a survey of publications in eight high-impact journals. Berlin: WZB Berlin Social Science Center, 2015. Handbook for Data Users and Coders. version I. Disponível em: Disponível em: https://manifesto-project.wzb.eu/datasets/mpduds# Acesso em: 27 out. 2019.
    » https://manifesto-project.wzb.eu/datasets/mpduds#
  • VOLKENS, A. et al. The manifesto data collection (MRG/CMP/MARPOR), M. P. Berlin 2019.
  • VOLKENS, A. et al. The manifesto data collection. manifesto project (MRG/CMP/MARPOR). Version 2013b. Berlin, 2013. Disponível em: Disponível em: https://manifesto-project.wzb.eu/ Acesso em: 23 ago. 2015.
    » https://manifesto-project.wzb.eu/
  • WERNER, A.; LACEWELL, O.; VOLKENS, A. Manifesto coding instructions. (5th revised edition), 2014. Disponível em: Disponível em: https://manifesto-project.wzb.eu/information/documents/handbooks Acesso em 23 ago. 2015.
    » https://manifesto-project.wzb.eu/information/documents/handbooks
  • WIESEHOMEIER, Nina. Policy positions of 47 political parties and 4 presidents in 4 Latin American countries. In: PREPPS: The political representation, executives, and political parties survey: banco de dados [Internet]. 2011. Disponível em: Disponível em: http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA Acesso em: 10 abr. 2020.
    » http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA
  • WIESEHOMEIER, Nina. Policy positions of 165 political parties and 18 presidents in 18 Latin American countries. In: PREPPS: The political representation, executives, and political parties survey: banco de dados [Internet]. 2015. Disponível em: Disponível em: http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA Acesso em: 10 abr. 2020.
    » http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA
  • WIESEHOMEIER, Nina; BENOIT, K. Policy positions of 146 political parties and 18 presidents in 18 Latin American countries. In: PREPPS: The political representation, executives, and political parties survey: banco de dados [Internet]. 2006. Disponível em: Disponível em: http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA Acesso em: 10 abr. 2020.
    » http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA
  • WIESEHOMEIER, Nina; SINGER, Mathew.; RUTH-LOVELL, Saskia. Political representation, executives, and political parties survey: Data from Expert Surveys in 18 Latin American Countries, 2018-2019. IE University, University of Connecticut and GIGA Institute Hamburg. 2019. Disponível em: Disponível em: https://dataverse.harvard.edu/dataverse/prepps Acesso em: 23 abr. 2020.
    » https://dataverse.harvard.edu/dataverse/prepps
  • ZUCCO, Cesar; POWER, Timothy. Brazilian legislative surveys (Waves 1-8, 1990-2017). Harvard Dataverse, V2, UNF:6:OZ4b3ybYKksC255DkruPxg== [fileUNF] 2019. Disponível em: Disponível em: https://dataverse.harvard.edu/dataverse/bls Acesso em:04 abr. 2020.
    » https://dataverse.harvard.edu/dataverse/bls
  • 5
    Former versions of this paper were presented at the 3rd LAPolMeth Meeting in 2019, at the 43º ANPOCS Annual Meeting in 2019, and in a research seminar at the DCP/UFPE also in 2019. We thank the participants of these events and the anonymous RBCP reviewers for their helpful comments. Any remaining mistakes are our full responsibility.
  • 6
    The original classification in Coppedge (1997)COPPEDGE, Michael. A classification of Latin American political parties. Kellogg Institute Working Papers, n. 244, 1997. Disponível em: Disponível em: https://kellogg.nd.edu/sites/default/files/old_files/documents/244_0.pdf Acesso em: 20 fev. 2006.
    https://kellogg.nd.edu/sites/default/fil...
    had also a religious dimension with two categories (Christian and secular) that we do not use in our comparison.
  • 7
    Lorenzoni and Pérez (2013)LORENZONI, Miguel; PÉREZ, Verónica. Cambios y continuidades de la izquierda en Uruguay: un análisis a partir de las propuestas programáticas del Frente Amplio 1971-2009. Revista Uruguaya de Ciencia Política, v. 22, n. 1, p. 61-81, 2013. have applied the MP method to measure ideological position of Frente Amplio, in Uruguay from 1971 to 2009. Unfortunately, MP dataset on Uruguay includes only manifestos from the 2014 election.
  • 8
    PELA - Proyecto Elites Lationoamericanas is a collection of surveys conducted with representatives in legislatures. Available at Universidad de Salamanca (https://oir.org.es/pela/)
  • 9
    Brazilian Legislative Survey uses ideological position in a scale from 1 (Left) to 10 (Right) in several waves: 1990, 1993, 1997, 2001, 2005, 2009, 2013, 2017. Here we use the average position in the nearest year.
  • 10
    PREPPS - The Political Representation, Executives, and Political Parties Survey is a collection of expert surveys, carried out in four waves.
  • 11
    V-PARTY - The Varieties of Party Identity and Organization (LÜHRMANN et al., 2020LÜHRMANN, A et al. Varieties of Party Identity and Organization (V-Party) Dataset V1. PROJECT., V. o. D. V.-D. 2020. Disponível em: Disponível em: https://www.v-dem.net/en/data/data/v-party-dataset/ Acesso em: 08 jan. 2021.
    https://www.v-dem.net/en/data/data/v-par...
    ) is a dataset produced by the V-Dem Institute. Based on its several indicators, we used the variable Economic left-right scale (v2pariglef), which aggregates experts’ ordinal answers in a standardized interval scale varying from -5 to +5 (PEMSTEIN et al., 2020PEMSTEIN, Dan et al. The V-Dem Measurement Model: latent variable analysis for cross-national and cross-temporal expert-coded data. V-Dem working paper series, n. 21, 2020.).
  • 12
    The South America dataset has 98 lines, but two of them (documents from Brazilian PDT and PDS in 1989 election) are not coded.
  • 13
    The matching between measurements is somewhat arbitrary in some cases in which the position in the alternative scale refers to the party in a legislature elected in a different year than the presidential manifesto. In these cases, we matched the manifesto with the nearest year possible.
  • 14
    During the segmentation step, we teamed up to consolidate our individual segmentations in a final, unified one.
  • 15
    The values of the kappa index for the three pairs of MVT coders are 0.88 (TxV), 0.68 (VxM), and 0.66 (MxT). According to Landis and Koch (1977)LANDIS, J. R.; KOCH, G. G. The measurement of observer agreement for categorical data. Biometrics, v. 33, n. 1, p. 159-174, 1977. and Bonnardel (2001)BONNARDEL, Philippe. The Kappa coefficient: The measurement of interrater agreement when the ratings are on categorical scales. The case of two raters. Le Coefficient Kappa. 2001. Disponível em: Disponível em: http://kappa.chez-alice.fr/kappa.txt . Acesso em: 24 nov. 2011.
    http://kappa.chez-alice.fr/kappa.txt...
    , kappa values equal or greater than 0.81 indicate excellent agreement, and values between 0.61 and 0.80 are good.
  • 16
    These subcategories were created in order to accommodate the specificities of new democracies, like Eastern and Central European countries and Mexico. They were then used to analyze Latin American party manifestos.
  • 17
    This difference may be related to a supposed emphasis that MP instructors put, while training the coders, on avoiding the ‘uncoded’ category, as we were recently warned about.
  • 18
    Laver and Budge (1992)LAVER, M.; BUDGE, I. Party policy and government coalitions. Nova Iorque: St. Martin’s Press, 1992. built the RILE scale using a series of exploratory factors to identify possible combinations of variables that would reflect either the left or the right position.
  • 29
    Comprises subcategories 601_1 and 601_2 used by MP coders.
  • 1
    Para replicação dos dados: https://doi.org/10.7910/DVN/CM3LXP
  • 19
    In the present paper, we compute the MP original RILE scale (BUDGE; MEYER, 2013BUDGE, Ian; MEYER, Thomas. Understanding and validating the left-right scale (RILE). In: VOLKENS, Andrea; BARA, Judith; BUDGE, Ian et al. (ed.). Mapping policy preferences from texts: statistical solutions for manifesto analysts. Oxford: Oxford University Press, 2013.) using the original set of categories, although elsewhere (TAROUCO; MADEIRA, 2013TAROUCO, Gabriela; MADEIRA, Rafael. Esquerda e direita no sistema partidário brasileiro: análise de conteúdo de documentos programáticos. Revista Debates, v. 7, n. 2, Porto Alegre, p. 93-114, 2013.; MADEIRA; VIEIRA; TAROUCO, 2017MADEIRA, Rafael, VIEIRA, Soraia; TAROUCO, Gabriela. Agendas, preferências, competição: PT e PSDB em disputas presidenciais. Caderno CRH, v. 30, n. 80, p. 257-273, 2017. ) we had proposed an adaptation of the left-right scale.
  • 20
    Comprises subcategories 201_1 and 201_2 used by MP coders.
  • 21
    Comprises subcategories 305_1, 305_2, 305_3, 305_4, 305_5 and 305_6 used by MP coders.
  • 22
    Comprises subcategories 601_1 and 601_2 used by MP coders.
  • 23
    Comprises subcategories 605_1 and 605_2 used by MP coders.
  • 24
    Comprises subcategories 606_1 and 606_2 used by MP coders.
  • 25
    Comprises subcategories 103_1 and 103_2 used by MP coders.
  • 26
    Comprises subcategories 202_1, 202_2, 202_3 and 202_4 used by MP coders.
  • 27
    Comprises subcategories 201_1 and 201_2 used by MP coders.
  • 28
    Comprises subcategories 305_1, 305_2, 305_3, 305_4, 305_5 and 305_6 used by MP coders.
  • 30
    Comprises subcategories 605_1 and 605_2 used by MP coders.
  • 31
    Comprises subcategories 606_1 and 606_2 used by MP coders.
  • 32
    Comprises subcategories 103_1 and 103_2 used by MP coders.
  • 33
    Comprises subcategories 202_1, 202_2, 202_3 and 202_4 used by MP coders.

Appendix.

1a.
Means in RILE categories, for four Brazilian documents of 2006 and 2010 elections, MP and MVT coders
1b.
Means in RILE categories, for two Brazilian documents of 2014 election, MP and MVT coders
2.1.
Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Argentina
2.2.
Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Bolivia
2.3.
Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Brazil
2.4.
Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Chile
2.5.
Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Uruguay

Publication Dates

  • Publication in this collection
    08 Dec 2021
  • Date of issue
    2022

History

  • Received
    10 Feb 2021
  • Accepted
    28 June 2021
Universidade de Brasília. Instituto de Ciência Política Instituto de Ciência Política, Universidade de Brasília, Campus Universitário Darcy Ribeiro - Gleba A Asa Norte, 70904-970 Brasília - DF Brasil, Tel.: (55 61) 3107-0777 , Cel.: (55 61) 3107 0780 - Brasília - DF - Brazil
E-mail: rbcp@unb.br