Abstract:
In this paper we compare the recent dataset of Latin American country party manifestos as coded by the Manifest Project database to other estimations of position on the left-right scale and to another coding of the same documents, discussing their limits and potentialities. The differences found between results offer an interesting opportunity to discuss the method, its reliability, and the validity of the coding scheme and the scales. Our findings suggest that the fragile reliability of the hand-coded content analysis could be circumvented by employing intercoder reliability tests and that users must be cautious when basing conclusions on this project’s results.
Keywords:
party manifestos; ideology; content analysis; political parties
Resumo:
Este artigo compara a recente base de dados do Manifesto Project de documentos partidários de países latino-americanos a outras medidas de posicionamento na escala esquerda-direita e a outra codificação dos mesmos documentos, discutindo seus limites e potencialidades. As diferenças encontradas entre os resultados oferecem uma ótima oportunidade para discutir o método, sua confiabilidade e a validade do esquema de codificação e das escalas. Nossos achados sugerem que a frágil confiabilidade da análise de conteúdo manual pode ser contornada com testes de confiabilidade entre codificadores e que os usuários dos dados devem ser cautelosos ao basear conclusões sobre seus resultados.
Palavras-chave:
manifestos; ideologia; análise de conteúdo; partidos políticos
Resumen:
La reciente publicación por parte del Manifesto Project de una base de datos de manifiestos partidarios de América del Sur, además de propiciar una vasta gama de investigaciones, es una valiosa oportunidad de evaluar la confiabilidad del método de análisis manual de contenido. Este artículo compara las codificaciones y medidas producidas por el Manifesto Project con otras codificaciones manuales de los mismos documentos y con otras medidas de posición en la escala izquierda-derecha. Esta comparación nos permite discutir los límites y potencialidades del método y las implicaciones de la utilización de sus resultados como medida de ideología. Los resultados encontrados sugieren que la debilidad del análisis manual de contenido respecto de los sesgos de los codificadores podría eludirse empleando pruebas de confiabilidad entre codificadores y que los usuarios deben tener cuidado al basar las conclusiones en sus resultados.
Palabras clave:
manifiestos partidarios; ideología; análisis del contenido; partidos
Introduction5 5 Former versions of this paper were presented at the 3rd LAPolMeth Meeting in 2019, at the 43º ANPOCS Annual Meeting in 2019, and in a research seminar at the DCP/UFPE also in 2019. We thank the participants of these events and the anonymous RBCP reviewers for their helpful comments. Any remaining mistakes are our full responsibility.
How reliable are our conclusions on the political saliency and ideology of Latin American political parties based on hand-coded content analyses of manifestos? Is the left-right scale built from such analyses adequate for capturing the ideological dimension of political competition outside European countries?
Ideology is one of those concepts in social sciences that, despite not being directly observable, has implications for several observable variables. In this paper, we bracket the ontological and epistemological dimensions of ideology, and focus on its methodological dimension. That is, we neither doubt the actual existence of our object, nor that it can be objectively known by the political science. However, we question one of the methods that have been used to do that.
Party ideology seems to have lost importance - to public policy, voting behaviour, electoral coalitions, for example - in some advanced democracies as distinct governments worldwide converged into a globalized economic policy and various societies embraced post-materialist values. In contemporary Latin America, however, where struggles about redistribution are pervasive, party ideology still plays a role and has received corresponding attention from political science. Examining party ideology is imperative for understanding both the recent pink tide and the right turn in the region, for example. Thus, we need good measures of ideology.
There are many ways to measure ideology. Interviews (or the application of questionnaires) with political elites provide us with data from those who are knowledgeable about politics but have a direct interest in the political game (which can result in answers that are not always sincere). National opinion polls on the ideological positioning of parties tend to collect more sincere opinions, though with a low degree of familiarity with the political game. Expert surveys, in turn, are considered to be a strategy that would make well-informed and politically motivated responses compatible, although academics often have clearly established preferences and even organic ties to political parties, governments etc.
Another field of observation widely used for the analysis of ideological positioning is the analysis of roll call votes. Contrary to the previous strategies, here the concrete performance of the different parties is being taken as an empirical ballast. However, numerous elements demand caution in the organization of this type of research, in the selection of votes and in the reading of the data. First, many of the votes that take place daily in a legislative house are not related to topics that require ideological positions. Second, other divisions (such as government versus opposition, North versus South, large cities versus inland cities, gender issues etc.) may override voters’ initial ideological preferences.
Party manifestos also have their pros and cons as sources of ideology measures. While being the formal party statement presented to the public, such documents are at the same time targeted at the electoral context. Considering this, this article explores the content of party manifestos as a proxy for party ideology.
The Manifesto Project (MP) collects, codes, and publishes content data on electoral party manifestos from many countries. The method is hand-coded content analysis, and the standard coding scheme contains 56 categories grouped into seven domains. Although this method has been a target of various criticisms, the MP has provided data for significant body of research in political science (VOLKENS et al., 2015VOLKENS et al. Scope, range, and extent of manifesto project data usage: a survey of publications in eight high-impact journals. Berlin: WZB Berlin Social Science Center, 2015. Handbook for Data Users and Coders. version I. Disponível em: Disponível em: https://manifesto-project.wzb.eu/datasets/mpduds# . Acesso em: 27 out. 2019.
https://manifesto-project.wzb.eu/dataset...
).
The MP recently released a dataset of 98 coded manifestos of 61 political parties from five countries of Latin America, launched for 24 elections from 1989 to 2018 (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
; KRAUSE et al., 2018KRAUSE, W. et al. Manifesto Corpus. Version: 2018-2. WZB Berlin Social Science Center, 2018. Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2018b . Acesso em: 16 out. 2019.
https://doi.org/10.25522/manifesto.mpdss...
; VOLKENS et al., 2019VOLKENS, A. et al. The manifesto data collection (MRG/CMP/MARPOR), M. P. Berlin 2019.). Data includes the frequency of each category in each document, as well as some scales, including the left-right scale. These data provide a broad perspective for new analyses, allowing for comparison of parties either in a specific country or between countries, and either in a specific election or over time. The availability of such data provided by the MP also makes it possible to test the reliability of content analysis and the validity of the coding scheme. These are the goals of this paper.
Ultimately, none of the available alternatives is able to fully reflect party ideology. Although each approach has its limitations, the analysis of party manifestos has gained visibility and the MP data has been widely used. In view of such dissemination, we chose to inquire into the hand-coded content analysis of party manifestos, by comparing it to other ideology measures.
We pursue that aim in two ways: first, by comparing the MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) results to other ideology measures in Latin American countries, and second, by comparing MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) coding of two Brazilian parties’ manifestos to our own coding (MADEIRA; VIEIRA; TAROUCO, 2017MADEIRA, Rafael, VIEIRA, Soraia; TAROUCO, Gabriela. Agendas, preferências, competição: PT e PSDB em disputas presidenciais. Caderno CRH, v. 30, n. 80, p. 257-273, 2017. ), hereafter MVT, which applied the same protocol. We, then, discuss the implications of distinct codings, namely, the distinct positions in the Right-Left (RILE) scale.
Our findings suggest that there is room for improvement in the reliability of hand-coded analyses of manifestos. Despite many advantages of the MP method, we found that the comparability of the coding results is limited. Nevertheless, we argue for the internal validity of both the coding scheme and the left-right scale. The ideological positions measured through MP coding are quite similar to some other assessments of Latin American parties’ positions.
In the next sections we compare the MP left-right scale position to other ideology measures for several Latin American parties. Next, we discuss the classifications of salient policy issues that resulted from the analysis of six Brazilian manifestos, comparing their ideological positions, and discussing the implications of discrepancies for further analyses. Finally, we make some considerations about the weaknesses and advantages of the method and about the research agenda.
Latin American parties’ ideological positions
This section compares the RILE scale as computed using MP data (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) to other assessments of party ideology in Latin America. The MP RILE measures are available for 96 Latin American manifestos, from 1989 to 2018, and vary from -51.1 (PSOL manifesto, issued for the 2006 election) to 42.5 (PSL manifesto, issued for the 2018 election). The complete set of measures is available in the appendix
Appendix.
1a.
Means in RILE categories, for four Brazilian documents of 2006 and 2010 elections, MP and MVT coders
Category
Coder
Mean
Std. Dev.
Right: 104 - Military (positive)
MP318
0.894
0.460
MVT
0.700
0.707
Right: 201 - Freedom and human rights (positive)20
MP318
1.901
1.442
MVT
0.650
0.480
Right: 203 - Constitutionalism (positive)
MP318
0.252
0.291
MVT
0.000
0.000
Right: 305 - Political authority (positive)21
MP318
3.288
3.781
MVT
0.000
0.000
Right: 401 - Free enterprise (positive)
MP318
0.223
0.281
MVT
0.050
0.050
Right: 402 - Economic Incentives (positive)
MP318
2.212
0.885
MVT
3.275
2.205
Right: 407 - Protectionism (negative)
MP318
0.427
0.070
MVT
0.000
0.000
Right: 414 - Economic orthodoxy (positive)
MP318
1.918
0.618
MVT
0.325
0.650
Right: 505 - Welfare state limitation (positive)
MP318
0.103
0.206
MVT
0.000
0.000
Right: 601 - National way of life (positive)22
MP318
0.974
0.876
MVT
0.000
0.000
Right: 603 - Traditional morality (positive)
MP318
0.165
0.227
MVT
0.000
0.000
Right: 605 - Law and order (positive)23
MP318
4.076
0.273
MVT
4.650
1.303
Right: 606 - Social harmony (positive)24
MP318
0.937
0.713
MVT
0.050
0.050
Left: 103 - Anti-imperialism (positive)25
MP318
1.187
1.276
MVT
0.000
0.000
Left: 105 - Military (negative)
MP318
0.031
0.062
MVT
0.000
0.000
Left: 106 - Peace (positive)
MP318
0.527
0.700
MVT
0.400
0.616
Left: 107 - Internationalism (positive)
MP318
1.873
1.688
MVT
2.025
1.323
Left: 202 - Democracy (positive)26
MP318
5.289
3.253
MVT
1.800
1.804
Left: 403 - Market regulation (positive)
MP318
1.809
1.000
MVT
0.025
0.050
Left: 404 - Economic planning (positive)
MP318
1.127
0.287
MVT
0.475
0.591
Left: 406 - Protectionism (positive)
MP318
0.166
0.247
MVT
0.425
0.506
Left: 412 - Controlled economy (positive)
MP318
0.407
0.303
MVT
0.000
0.000
Left: 413 - Nationalization (positive)
MP318
0.245
0.093
MVT
0.000
0.000
Left: 504 - Welfare state expansion (positive)
MP318
9.518
1.316
MVT
11.450
5.500
Left: 506 - Education expansion (positive)
MP318
5.737
0.813
MVT
6.300
1.503
Left: 701 - Labour groups (positive)
MP318
2.639
0.962
MVT
0.950
0.929
Source: Krause et al. (2019) and MVT (2017)
1b.
Means in RILE categories, for two Brazilian documents of 2014 election, MP and MVT coders
Category
Coder
Mean
Std. Dev.
Right: 104 - Military (positive)
MP328
0.565
0.799
MVT
0.050
0.050
Right: 201 - Freedom and human rights (positive) 27
MP328
1.369
0.498
MVT
1.700
2.121
Right: 203 - Constitutionalism (positive)
MP328
0.000
0.000
MVT
0.000
0.000
Right: 305 - Political authority (positive)28
MP328
3.349
3.297
MVT
0.000
0.000
Right: 401 - Free enterprise (positive)
MP328
1.499
1.876
MVT
0.000
0.000
Right: 402 - Economic Incentives (positive)
MP328
4.557
0.371
MVT
5.750
1.061
Right: 407 - Protectionism (negative)
MP328
0.283
0.400
MVT
0.000
0.000
Right: 414 - Economic orthodoxy (positive)
MP328
1.988
0.864
MVT
0.150
0.150
Right: 505 - Welfare state limitation (positive)
MP328
0.452
0.639
MVT
0.000
0.000
Right: 601 - National way of life (positive) 29
MP328
0.398
0.076
MVT
0.000
0.000
Right: 603 - Traditional morality (positive)
MP328
0.000
0.000
MVT
0.000
0.000
Right: 605 - Law and order (positive) 30
MP328
4.035
1.325
MVT
4.250
1.768
Right: 606 - Social harmony (positive) 31
MP328
1.983
1.831
MVT
0.050
0.050
Left: 103 - Anti-imperialism (positive) 32
MP328
0.086
0.122
MVT
0.000
0.000
Left: 105 - Military (negative)
MP328
0.000
0.000
MVT
0.000
0.000
Left: 106 - Peace (positive)
MP328
0.172
0.243
MVT
0.000
0.000
Left: 107 - Internationalism (positive)
MP328
1.598
0.661
MVT
1.650
0.071
Left: 202 - Democracy (positive) 33
MP328
4.274
0.771
MVT
0.900
0.990
Left: 403 - Market regulation (positive)
MP328
0.909
0.313
MVT
1.550
2.192
Left: 404 - Economic planning (positive)
MP328
0.568
0.316
MVT
0.000
0.000
Left: 406 - Protectionism (positive)
MP328
0.229
0.163
MVT
0.350
0.350
Left: 412 - Controlled economy (positive)
MP328
0.775
1.095
MVT
0.050
0.050
Left: 413 - Nationalization (positive)
MP328
0.086
0.122
MVT
0.000
0.000
Left: 504 - Welfare state expansion (positive)
MP328
9.154
2.235
MVT
12.450
5.162
Left: 506 - Education expansion (positive)
MP328
7.887
0.287
MVT
6.750
2.192
Left: 701 - Labour groups (positive)
MP328
3.306
0.680
MVT
0.000
0.000
Source: Krause et al. (2019) and MVT (2017)
2.1.
Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Argentina
edate
party
partyname
partyabbrev
rile MP
Alcantara Saez 2012 (1-10)
Oliveira 2015 VIdeo
Huber et al. 2008 num
PREPPS (1-20)
Lopez et al 2013
D’Alessandro 2013
V-party 2020 (-5 - +5)
14/05/1989
150040
Center Alliance
13.287
4
-1.30
14/05/1989
150029
Justicialist Party of Popular Unity
FREJUPO
-3.82
4
-5.34
-0.688
14/05/1989
150301
Civic Radical Union
UCR
-19.416
3
-10.66
-0.148
14/05/1995
150202
Front for a Country in Solidarity
FREPASO
-15.217
2
-20.59
-1.198
14/05/1995
150201
Justicialist Party
PJ
1.997
7.40
4
-6.27
1.174
14/05/1995
150301
Civic Radical Union
UCR
-11.136
5.63
3
-16.06
0.038
24/10/1999
150601
Action for the Republic
AR
0.176
4
-2.10
2.251
24/10/1999
150028
Justicialist Coalition for Change
-6.644
7.78
4
-4.84
0.825
24/10/1999
150032
Alliance for Work, Justice, and Education
ALIANZA
-13.04
5.57
0.23
3
-13.09
27/04/2003
150205
Front of the popular movement
MP
4.167
-4.76
27/04/2003
150210
Alternative for a Republic of Equals
ARI
-26.135
-0.597
27/04/2003
150401
Recreate for Growth
RECREAR
-0.381
9.26
27/04/2003
150025
Front for Victory
FpV
-22.667
-0.17
-1.25
27/04/2003
150022
Front for Loyalty and Union of the Democratic Centre
FPL + UCeDé
-7.958
6.56
5.02
-0.405
28/10/2007
150027
Justice, Union and Liberty Front
FREJULI
-26.923
28/10/2007
150031
An Advanced Union
UNA
-13.213
-5.31
28/10/2007
150023
Civic Coalition
CC
-11.006
4.71
-0.269
28/10/2007
150025
Front for Victory
FpV
-20.755
4.79
-0.56
-10.71
-1.669
28/06/2009
150024
Social and Civic Agreement
ACyS
-20
28/06/2009
150220
New Encounter
-16.346
28/06/2009
150221
Neuquén People’s Movement
MPN
-12.027
28/06/2009
150222
It is possible
-18.957
28/06/2009
150025
Front for Victory
FpV
-40
-1.663
28/06/2009
150061
Alliance Union-PRO
-18.023
1.537
23/10/2011
150027
Federal Commitment
1.245
43.5
23/10/2011
150230
Popular Front
-50
29.8
23/10/2011
150026
Broad Progressive Front
FAP
-17.351
3.71
2.5
-19.91
-1.274
23/10/2011
150301
Civic Radical Union
UCR
-13.333
6.35
10.360
5.0
-21.56
0.197
23/10/2011
150025
Front for Victory
FpV
-3.425
6.37
-1.00
7.923
-1.4
-23.38
-1.677
27/10/2013
150026
Broad Progressive Front
FAP
-6.838
27/10/2013
150301
Civic Radical Union
UCR
-15.094
11.375
27/10/2013
150025
Front for Victory
FpV
0
7.083
-1.654
Sources cited in Table 1.
2.2.
Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Bolivia
edate
party
partyname
partyabbrev
rile MP
Alcantara Saez 2012 (1-10)
Oliveira 2015 VIdeo
PREPPS (1-20)
V-party 2020 (-5 - +5)
06/12/2009
151255
Social Alliance
AS
8.832
06/12/2009
151617
Autonomy for Bolivia
APB
-3.091
06/12/2009
151062
Plan Progress for Bolivia - National Convergence
PPB-CN
-38.356
2.253
06/12/2009
151610
Nationalist Revolutionary Movement
MNR
11.64
8.28
06/12/2009
151640
National Unity Front
UN
-2.381
8.28
06/12/2009
151250
Movement towards Socialism - Political Instrument for the Sovereignty of the Peoples
MAS-IPSP
-13.401
4.17
-0.26
-2.192
12/10/2014
151640
Democratic Union
UD
-3.287
12/10/2014
151540
Christian Democratic Party
PDC
-14.107
17.667
1.34
12/10/2014
151250
Movement towards Socialism - Political Instrument for the Sovereignty of the Peoples
MAS-IPSP
-17.41
-0.73
4.900
-2.192
Sources cited in Table 1.
2.3.
Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Brazil
edate
party
partyname
partyabbrev
rile MP
Zucco and Power 2019 (1-10)
Alcantara Saez 2012 (1-10)
Oliveira 2015 VIdeo
Huber et al. 2008 num
PREPPS (1-20)
Gonzales y Queirolo 2013 num
Lopez et al 2013
V-party 2020 (-5 - +5)
15/11/1989
180250
Democratic Labour Party
PDT
2
15/11/1989
180610
Democratic Social Party
PDS
5
15/11/1989
180410
New Brazil Movement
10.733
7.4
15/11/1989
180230
Popular Front Brazil
-46.855
1.5
1
15/11/1989
180309
Party of Brazilian Social Democracy
PSDB
32.836
3.9
2
03/10/1994
180710
Party of the Reconstruction of the National Order
PRONA
-27.333
03/10/1994
180230
A democratic revolution in Brazil
-27.142
2.0
1
-2.609
03/10/1994
180310
Let’s work Brazil
4.843
4.4
0.28
0.37
04/10/1998
180232
Socialist People’s Party
PPS
-15.974
3.0
1
04/10/1998
180230
Union of the People - Change Brazil
-19.905
1.9
1
-2.227
04/10/1998
180310
Go ahead Brazil
-18.318
5.9
0.33
2
0.936
06/10/2002
180320
Labour Front Coalition
-20.711
3.8
06/10/2002
180240
Hope for Brasil Coalition
-12.53
2.9
-1.639
06/10/2002
180310
Coalition Big Alliance
-10.523
6.1
4
0.881
06/10/2002
180230
Coalition Lula for President
-21.955
2.2
-0.10
1
-1.437
01/10/2006
180231
Left Front
-51.128
2.955
01/10/2006
180310
Coalition for a decent Brazil
-9.202
5.9
6.89
13.462
1.071
01/10/2006
180230
Coalition the People’s Power
-19.005
3.6
4.44
-0.29
6.370
-1.437
03/10/2010
180110
Green Party
PV
-28.302
4.6
8.158
03/10/2010
180310
Brazil can do more
-11.302
5.8
7.13
12.000
40.4
1.071
03/10/2010
180230
For Brazil to keep on changing
-13.636
3.6
4.73
0.00
5.429
13.1
-1.292
05/10/2014
180240
United for Brazil
-5.394
4.1
-1.131
05/10/2014
180310
Change, Brazil
-5.085
6.0
13.833
1.516
05/10/2014
180230
With the strength of the people
-12.048
3.7
-0.22
6.222
-1.45
07/10/2018
180620
Brazil above everything, God above everyone
42.466
18.355
3.242
07/10/2018
180230
The People happy again
-26.123
2.7
5.044
-1.897
07/10/2018
180251
Sovereign Brazil
-20.298
3.8
6.661
-1.187
07/10/2018
180310
To unite Brazil
4.474
7.1
14.772
1.416
Sources cited in Table 1.
2.4.
Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Chile
edate
party
partyname
partyabbrev
rile MP
Alcantara Saez 2012 (1-10)
Oliveira 2015 VIdeo
Huber et al 2008 num
PREPPS (1-20)
Gonzales y Queirolo 2013 num
Lopez et al 2013
V-party 2020 (-5 - +5)
14/12/1989
155620
Liberal-Socialists of Chile
-3.968
14/12/1989
155061
Democracy and Progress
21.405
5
2.656
14/12/1989
155021
Concert of Parties for Democracy
-15.152
-0.33
3
0.348
11/12/1993
155021
Concert of Parties for Democracy
-10.493
4.85
3
11/12/1993
155061
Union for Chile’s Progress
25.347
9.13
5
2.656
12/12/1999
155061
Alliance for Chile
-5.017
9.58
5
12/12/1999
155021
Concert of Parties for Democracy
-13.596
3.71
0.13
2
0.146
11/12/2005
155025
Together we can do more for Chile
PODEMOS
-36.323
11/12/2005
155021
Concert of Parties for Democracy
-19.715
2.51
0.00
2
11/12/2005
155601
Independent Democratic Union
UDI
16.892
9.53
18.100
4
2.656
11/12/2005
155602
National Renewal
RN
-0.809
7.70
16.333
5
1.67
13/12/2009
155023
New Majority for Chile
-28.325
-0.3
13/12/2009
155025
Together we can do more for Chile
PODEMOS
-41.486
-21.6
13/12/2009
155021
Concert of Parties for Democracy
-18.857
4.62
-7.7
13/12/2009
155061
Coalition for Change
-3.029
7.78
0.42
8.7
17/11/2013
155024
If you want it, Chile changes
-37.011
17/11/2013
155061
Alliance
-28.011
17.333
17/11/2013
155021
New Majority for Chile/Socialist Party of Chile
-18.974
-0.30
6.333
-1.403
19/11/2017
155501
Christian Democratic Party
PDC
-29.713
10.000
0.297
20/11/2017
155021
New Majority for Chile
-24.938
21/11/2017
155026
Broad Front
FA
-39.175
22/11/2017
155061
Chile let’s go
3.958
23/11/2017
155201
Progressive Party
PRO
-31.919
24/11/2017
155980
Independent
-2.332
Sources cited in Table 1.
2.5.
Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Uruguay
edate
party
partyname
Party abbrev
rile MP
Alcantara Saez 2012 (1-10)
Oliveira 2015 VIdeo
PREPPS (1-20)
V-party 2020 (-5 - +5)
26/10/2014
162210
Popular Assembly
UP
-43.399
1.750
26/10/2014
162510
Independent Party
PI
-0.077
9.250
26/10/2014
162410
Colorado Party
PC
-8.831
8.05
15.625
1.259
26/10/2014
162610
National Party
PN
-3.24
7.66
13.875
1.268
26/10/2014
162310
Broad Front
FA
-21.2
3.71
0.43
4.750
-1.354
Sources cited in Table 1.
.
The main classification of Latin American parties is that developed by Coppedge (1997)COPPEDGE, Michael. A classification of Latin American political parties. Kellogg Institute Working Papers, n. 244, 1997. Disponível em: Disponível em: https://kellogg.nd.edu/sites/default/files/old_files/documents/244_0.pdf Acesso em: 20 fev. 2006.
https://kellogg.nd.edu/sites/default/fil...
based on information provided by country specialists. Here we use two versions of that classification, expanded and updated by Huber et al. (2008)HUBER, Evelyne et al. Politics and social spending in latin america. The Journal of Politics, v. 70, n. 2, p. 420-436, 2008. Disponível em: Disponível em: https://www.jstor.org/stable/10.1017/s0022381608080407 Acesso em: 28 set. 2019.
https://www.jstor.org/stable/10.1017/s00...
and by González Ferrer and Quierolo Velasco (2013)GONZÁLEZ FERRER, Luis Eduardo; QUIEROLO VELASCO, Rosario. Izquierda y derecha: formas de definirlas, el caso latinoamericano y sus implicaciones. América Latina Hoy, v. 65, p. 79-105, 2013.. The classification scheme comprises five categories in the left-right dimension: left, center-left, center, center-right, and right.6
6
The original classification in Coppedge (1997) had also a religious dimension with two categories (Christian and secular) that we do not use in our comparison.
In order to compare it to the RILE scale, we converted the classification in an ordinal measure, where left corresponds to 1.0 and right takes the value of 5.0.7
7
Lorenzoni and Pérez (2013) have applied the MP method to measure ideological position of Frente Amplio, in Uruguay from 1971 to 2009. Unfortunately, MP dataset on Uruguay includes only manifestos from the 2014 election.
We also compared the RILE positions to other estimations of ideology: two surveys conducted with parliamentarian elites, two expert surveys, an alternative content analysis, and three content analysis using MP method. The first two estimations are provided by the PELA project (ALCÁNTARA SAEZ, 2012ALCÁNTARA SAEZ, Manuel. Elections in Latin America 2009-2011: a comparative analysis. Kellogg Institute Working Papers, n. 386, 2012. Disponível em: Disponível em: https://kellogg.nd.edu/sites/default/files/old_files/documents/386_0.pdf Acesso em: 17 fev. 2020.
https://kellogg.nd.edu/sites/default/fil...
))8
8
PELA - Proyecto Elites Lationoamericanas is a collection of surveys conducted with representatives in legislatures. Available at Universidad de Salamanca (https://oir.org.es/pela/)
and by Zucco and Power (2019)ZUCCO, Cesar; POWER, Timothy. Brazilian legislative surveys (Waves 1-8, 1990-2017). Harvard Dataverse, V2, UNF:6:OZ4b3ybYKksC255DkruPxg== [fileUNF] 2019. Disponível em: Disponível em: https://dataverse.harvard.edu/dataverse/bls Acesso em:04 abr. 2020.
https://dataverse.harvard.edu/dataverse/...
9
9
Brazilian Legislative Survey uses ideological position in a scale from 1 (Left) to 10 (Right) in several waves: 1990, 1993, 1997, 2001, 2005, 2009, 2013, 2017. Here we use the average position in the nearest year.
. The third one comes from the PREPPS10
10
PREPPS - The Political Representation, Executives, and Political Parties Survey is a collection of expert surveys, carried out in four waves.
, published by Wiesehomeier (WIESEHOMEIER, 2011WIESEHOMEIER, Nina. Policy positions of 47 political parties and 4 presidents in 4 Latin American countries. In: PREPPS: The political representation, executives, and political parties survey: banco de dados [Internet]. 2011. Disponível em: Disponível em: http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA Acesso em: 10 abr. 2020.
http://ninaw.webfactional.com/prepps e e...
; WIESEHOMEIER, 2015WIESEHOMEIER, Nina. Policy positions of 165 political parties and 18 presidents in 18 Latin American countries. In: PREPPS: The political representation, executives, and political parties survey: banco de dados [Internet]. 2015. Disponível em: Disponível em: http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA Acesso em: 10 abr. 2020.
http://ninaw.webfactional.com/prepps e e...
; WIESEHOMEIER; BENOIT, 2006WIESEHOMEIER, Nina; BENOIT, K. Policy positions of 146 political parties and 18 presidents in 18 Latin American countries. In: PREPPS: The political representation, executives, and political parties survey: banco de dados [Internet]. 2006. Disponível em: Disponível em: http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA Acesso em: 10 abr. 2020.
http://ninaw.webfactional.com/prepps e e...
; WIESEHOMEIER; SINGER, 2019WIESEHOMEIER, Nina; SINGER, Mathew.; RUTH-LOVELL, Saskia. Political representation, executives, and political parties survey: Data from Expert Surveys in 18 Latin American Countries, 2018-2019. IE University, University of Connecticut and GIGA Institute Hamburg. 2019. Disponível em: Disponível em: https://dataverse.harvard.edu/dataverse/prepps . Acesso em: 23 abr. 2020.
https://dataverse.harvard.edu/dataverse/...
). The fourth one, also an expert survey, is the V-PARTY (LÜHRMANN et al. 2020LÜHRMANN, A et al. Varieties of Party Identity and Organization (V-Party) Dataset V1. PROJECT., V. o. D. V.-D. 2020. Disponível em: Disponível em: https://www.v-dem.net/en/data/data/v-party-dataset/ Acesso em: 08 jan. 2021.
https://www.v-dem.net/en/data/data/v-par...
).11
11
V-PARTY - The Varieties of Party Identity and Organization (LÜHRMANN et al., 2020) is a dataset produced by the V-Dem Institute. Based on its several indicators, we used the variable Economic left-right scale (v2pariglef), which aggregates experts’ ordinal answers in a standardized interval scale varying from -5 to +5 (PEMSTEIN et al., 2020).
The fifth one creates an indicator of the economic ideology of presidents, built from dummy variables located in their platforms (OLIVEIRA, 2015OLIVEIRA, Augusto Neftali. A ideologia econômica dos presidentes na América Latina: um indicador a partir dos programas de governo eleitorais. Indicadores Econômicos FEE, v. 43, n. 1, p. 83-98, 2015.). The last two measures (D’ALESSANDRO, 2013D’ALESSANDRO, Martín. Las plataformas electorales en la Argentina moderna. América Latina Hoy, v. 65, p. 107-139, 2013. Disponível em: Disponível em: https://revistas.usal.es//index.php/1130-2887/article/view/alh201365107139 Acesso em: 10 fev. 2014
https://revistas.usal.es//index.php/1130...
; LÓPEZ; MIRANDA; VALENZUELA-GUTIÉRREZ, 2013LÓPEZ, Miguel Ángel; MIRANDA, Nicolás; VALENZUELA-GUTIÉRREZ, Pablo. Estimando el espacio politico del cono sur y Brasil: las elecciones presidenciales en el eje izquierda-derecha. PostData, v. 18, n. 2, p. 403-442, 2013. Disponível em: Disponível em: https://www.researchgate.net/publication/260837426_Estimando_el_espacio_politico_del_cono_sur_y_brasil_las_elecciones_presidenciales_en_el_eje_izquierda-derecha Acesso em: 16 jan. 2014.
https://www.researchgate.net/publication...
) are codings of Latin American political party platforms performed according to the MP methodology.
These sources analyzed a total of 96 documents, of which 18 are only present in the MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) dataset,12
12
The South America dataset has 98 lines, but two of them (documents from Brazilian PDT and PDS in 1989 election) are not coded.
meaning that these documents cannot be compared with other sources.13
13
The matching between measurements is somewhat arbitrary in some cases in which the position in the alternative scale refers to the party in a legislature elected in a different year than the presidential manifesto. In these cases, we matched the manifesto with the nearest year possible.
Graph 1 shows the number of parties whose ideology is measured by more than one source. The list of the 78 documents for which we have at least one alternative measure to compare is available in the appendix
Appendix.
1a.
Means in RILE categories, for four Brazilian documents of 2006 and 2010 elections, MP and MVT coders
Category
Coder
Mean
Std. Dev.
Right: 104 - Military (positive)
MP318
0.894
0.460
MVT
0.700
0.707
Right: 201 - Freedom and human rights (positive)20
MP318
1.901
1.442
MVT
0.650
0.480
Right: 203 - Constitutionalism (positive)
MP318
0.252
0.291
MVT
0.000
0.000
Right: 305 - Political authority (positive)21
MP318
3.288
3.781
MVT
0.000
0.000
Right: 401 - Free enterprise (positive)
MP318
0.223
0.281
MVT
0.050
0.050
Right: 402 - Economic Incentives (positive)
MP318
2.212
0.885
MVT
3.275
2.205
Right: 407 - Protectionism (negative)
MP318
0.427
0.070
MVT
0.000
0.000
Right: 414 - Economic orthodoxy (positive)
MP318
1.918
0.618
MVT
0.325
0.650
Right: 505 - Welfare state limitation (positive)
MP318
0.103
0.206
MVT
0.000
0.000
Right: 601 - National way of life (positive)22
MP318
0.974
0.876
MVT
0.000
0.000
Right: 603 - Traditional morality (positive)
MP318
0.165
0.227
MVT
0.000
0.000
Right: 605 - Law and order (positive)23
MP318
4.076
0.273
MVT
4.650
1.303
Right: 606 - Social harmony (positive)24
MP318
0.937
0.713
MVT
0.050
0.050
Left: 103 - Anti-imperialism (positive)25
MP318
1.187
1.276
MVT
0.000
0.000
Left: 105 - Military (negative)
MP318
0.031
0.062
MVT
0.000
0.000
Left: 106 - Peace (positive)
MP318
0.527
0.700
MVT
0.400
0.616
Left: 107 - Internationalism (positive)
MP318
1.873
1.688
MVT
2.025
1.323
Left: 202 - Democracy (positive)26
MP318
5.289
3.253
MVT
1.800
1.804
Left: 403 - Market regulation (positive)
MP318
1.809
1.000
MVT
0.025
0.050
Left: 404 - Economic planning (positive)
MP318
1.127
0.287
MVT
0.475
0.591
Left: 406 - Protectionism (positive)
MP318
0.166
0.247
MVT
0.425
0.506
Left: 412 - Controlled economy (positive)
MP318
0.407
0.303
MVT
0.000
0.000
Left: 413 - Nationalization (positive)
MP318
0.245
0.093
MVT
0.000
0.000
Left: 504 - Welfare state expansion (positive)
MP318
9.518
1.316
MVT
11.450
5.500
Left: 506 - Education expansion (positive)
MP318
5.737
0.813
MVT
6.300
1.503
Left: 701 - Labour groups (positive)
MP318
2.639
0.962
MVT
0.950
0.929
Source: Krause et al. (2019) and MVT (2017)
1b.
Means in RILE categories, for two Brazilian documents of 2014 election, MP and MVT coders
Category
Coder
Mean
Std. Dev.
Right: 104 - Military (positive)
MP328
0.565
0.799
MVT
0.050
0.050
Right: 201 - Freedom and human rights (positive) 27
MP328
1.369
0.498
MVT
1.700
2.121
Right: 203 - Constitutionalism (positive)
MP328
0.000
0.000
MVT
0.000
0.000
Right: 305 - Political authority (positive)28
MP328
3.349
3.297
MVT
0.000
0.000
Right: 401 - Free enterprise (positive)
MP328
1.499
1.876
MVT
0.000
0.000
Right: 402 - Economic Incentives (positive)
MP328
4.557
0.371
MVT
5.750
1.061
Right: 407 - Protectionism (negative)
MP328
0.283
0.400
MVT
0.000
0.000
Right: 414 - Economic orthodoxy (positive)
MP328
1.988
0.864
MVT
0.150
0.150
Right: 505 - Welfare state limitation (positive)
MP328
0.452
0.639
MVT
0.000
0.000
Right: 601 - National way of life (positive) 29
MP328
0.398
0.076
MVT
0.000
0.000
Right: 603 - Traditional morality (positive)
MP328
0.000
0.000
MVT
0.000
0.000
Right: 605 - Law and order (positive) 30
MP328
4.035
1.325
MVT
4.250
1.768
Right: 606 - Social harmony (positive) 31
MP328
1.983
1.831
MVT
0.050
0.050
Left: 103 - Anti-imperialism (positive) 32
MP328
0.086
0.122
MVT
0.000
0.000
Left: 105 - Military (negative)
MP328
0.000
0.000
MVT
0.000
0.000
Left: 106 - Peace (positive)
MP328
0.172
0.243
MVT
0.000
0.000
Left: 107 - Internationalism (positive)
MP328
1.598
0.661
MVT
1.650
0.071
Left: 202 - Democracy (positive) 33
MP328
4.274
0.771
MVT
0.900
0.990
Left: 403 - Market regulation (positive)
MP328
0.909
0.313
MVT
1.550
2.192
Left: 404 - Economic planning (positive)
MP328
0.568
0.316
MVT
0.000
0.000
Left: 406 - Protectionism (positive)
MP328
0.229
0.163
MVT
0.350
0.350
Left: 412 - Controlled economy (positive)
MP328
0.775
1.095
MVT
0.050
0.050
Left: 413 - Nationalization (positive)
MP328
0.086
0.122
MVT
0.000
0.000
Left: 504 - Welfare state expansion (positive)
MP328
9.154
2.235
MVT
12.450
5.162
Left: 506 - Education expansion (positive)
MP328
7.887
0.287
MVT
6.750
2.192
Left: 701 - Labour groups (positive)
MP328
3.306
0.680
MVT
0.000
0.000
Source: Krause et al. (2019) and MVT (2017)
2.1.
Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Argentina
edate
party
partyname
partyabbrev
rile MP
Alcantara Saez 2012 (1-10)
Oliveira 2015 VIdeo
Huber et al. 2008 num
PREPPS (1-20)
Lopez et al 2013
D’Alessandro 2013
V-party 2020 (-5 - +5)
14/05/1989
150040
Center Alliance
13.287
4
-1.30
14/05/1989
150029
Justicialist Party of Popular Unity
FREJUPO
-3.82
4
-5.34
-0.688
14/05/1989
150301
Civic Radical Union
UCR
-19.416
3
-10.66
-0.148
14/05/1995
150202
Front for a Country in Solidarity
FREPASO
-15.217
2
-20.59
-1.198
14/05/1995
150201
Justicialist Party
PJ
1.997
7.40
4
-6.27
1.174
14/05/1995
150301
Civic Radical Union
UCR
-11.136
5.63
3
-16.06
0.038
24/10/1999
150601
Action for the Republic
AR
0.176
4
-2.10
2.251
24/10/1999
150028
Justicialist Coalition for Change
-6.644
7.78
4
-4.84
0.825
24/10/1999
150032
Alliance for Work, Justice, and Education
ALIANZA
-13.04
5.57
0.23
3
-13.09
27/04/2003
150205
Front of the popular movement
MP
4.167
-4.76
27/04/2003
150210
Alternative for a Republic of Equals
ARI
-26.135
-0.597
27/04/2003
150401
Recreate for Growth
RECREAR
-0.381
9.26
27/04/2003
150025
Front for Victory
FpV
-22.667
-0.17
-1.25
27/04/2003
150022
Front for Loyalty and Union of the Democratic Centre
FPL + UCeDé
-7.958
6.56
5.02
-0.405
28/10/2007
150027
Justice, Union and Liberty Front
FREJULI
-26.923
28/10/2007
150031
An Advanced Union
UNA
-13.213
-5.31
28/10/2007
150023
Civic Coalition
CC
-11.006
4.71
-0.269
28/10/2007
150025
Front for Victory
FpV
-20.755
4.79
-0.56
-10.71
-1.669
28/06/2009
150024
Social and Civic Agreement
ACyS
-20
28/06/2009
150220
New Encounter
-16.346
28/06/2009
150221
Neuquén People’s Movement
MPN
-12.027
28/06/2009
150222
It is possible
-18.957
28/06/2009
150025
Front for Victory
FpV
-40
-1.663
28/06/2009
150061
Alliance Union-PRO
-18.023
1.537
23/10/2011
150027
Federal Commitment
1.245
43.5
23/10/2011
150230
Popular Front
-50
29.8
23/10/2011
150026
Broad Progressive Front
FAP
-17.351
3.71
2.5
-19.91
-1.274
23/10/2011
150301
Civic Radical Union
UCR
-13.333
6.35
10.360
5.0
-21.56
0.197
23/10/2011
150025
Front for Victory
FpV
-3.425
6.37
-1.00
7.923
-1.4
-23.38
-1.677
27/10/2013
150026
Broad Progressive Front
FAP
-6.838
27/10/2013
150301
Civic Radical Union
UCR
-15.094
11.375
27/10/2013
150025
Front for Victory
FpV
0
7.083
-1.654
Sources cited in Table 1.
2.2.
Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Bolivia
edate
party
partyname
partyabbrev
rile MP
Alcantara Saez 2012 (1-10)
Oliveira 2015 VIdeo
PREPPS (1-20)
V-party 2020 (-5 - +5)
06/12/2009
151255
Social Alliance
AS
8.832
06/12/2009
151617
Autonomy for Bolivia
APB
-3.091
06/12/2009
151062
Plan Progress for Bolivia - National Convergence
PPB-CN
-38.356
2.253
06/12/2009
151610
Nationalist Revolutionary Movement
MNR
11.64
8.28
06/12/2009
151640
National Unity Front
UN
-2.381
8.28
06/12/2009
151250
Movement towards Socialism - Political Instrument for the Sovereignty of the Peoples
MAS-IPSP
-13.401
4.17
-0.26
-2.192
12/10/2014
151640
Democratic Union
UD
-3.287
12/10/2014
151540
Christian Democratic Party
PDC
-14.107
17.667
1.34
12/10/2014
151250
Movement towards Socialism - Political Instrument for the Sovereignty of the Peoples
MAS-IPSP
-17.41
-0.73
4.900
-2.192
Sources cited in Table 1.
2.3.
Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Brazil
edate
party
partyname
partyabbrev
rile MP
Zucco and Power 2019 (1-10)
Alcantara Saez 2012 (1-10)
Oliveira 2015 VIdeo
Huber et al. 2008 num
PREPPS (1-20)
Gonzales y Queirolo 2013 num
Lopez et al 2013
V-party 2020 (-5 - +5)
15/11/1989
180250
Democratic Labour Party
PDT
2
15/11/1989
180610
Democratic Social Party
PDS
5
15/11/1989
180410
New Brazil Movement
10.733
7.4
15/11/1989
180230
Popular Front Brazil
-46.855
1.5
1
15/11/1989
180309
Party of Brazilian Social Democracy
PSDB
32.836
3.9
2
03/10/1994
180710
Party of the Reconstruction of the National Order
PRONA
-27.333
03/10/1994
180230
A democratic revolution in Brazil
-27.142
2.0
1
-2.609
03/10/1994
180310
Let’s work Brazil
4.843
4.4
0.28
0.37
04/10/1998
180232
Socialist People’s Party
PPS
-15.974
3.0
1
04/10/1998
180230
Union of the People - Change Brazil
-19.905
1.9
1
-2.227
04/10/1998
180310
Go ahead Brazil
-18.318
5.9
0.33
2
0.936
06/10/2002
180320
Labour Front Coalition
-20.711
3.8
06/10/2002
180240
Hope for Brasil Coalition
-12.53
2.9
-1.639
06/10/2002
180310
Coalition Big Alliance
-10.523
6.1
4
0.881
06/10/2002
180230
Coalition Lula for President
-21.955
2.2
-0.10
1
-1.437
01/10/2006
180231
Left Front
-51.128
2.955
01/10/2006
180310
Coalition for a decent Brazil
-9.202
5.9
6.89
13.462
1.071
01/10/2006
180230
Coalition the People’s Power
-19.005
3.6
4.44
-0.29
6.370
-1.437
03/10/2010
180110
Green Party
PV
-28.302
4.6
8.158
03/10/2010
180310
Brazil can do more
-11.302
5.8
7.13
12.000
40.4
1.071
03/10/2010
180230
For Brazil to keep on changing
-13.636
3.6
4.73
0.00
5.429
13.1
-1.292
05/10/2014
180240
United for Brazil
-5.394
4.1
-1.131
05/10/2014
180310
Change, Brazil
-5.085
6.0
13.833
1.516
05/10/2014
180230
With the strength of the people
-12.048
3.7
-0.22
6.222
-1.45
07/10/2018
180620
Brazil above everything, God above everyone
42.466
18.355
3.242
07/10/2018
180230
The People happy again
-26.123
2.7
5.044
-1.897
07/10/2018
180251
Sovereign Brazil
-20.298
3.8
6.661
-1.187
07/10/2018
180310
To unite Brazil
4.474
7.1
14.772
1.416
Sources cited in Table 1.
2.4.
Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Chile
edate
party
partyname
partyabbrev
rile MP
Alcantara Saez 2012 (1-10)
Oliveira 2015 VIdeo
Huber et al 2008 num
PREPPS (1-20)
Gonzales y Queirolo 2013 num
Lopez et al 2013
V-party 2020 (-5 - +5)
14/12/1989
155620
Liberal-Socialists of Chile
-3.968
14/12/1989
155061
Democracy and Progress
21.405
5
2.656
14/12/1989
155021
Concert of Parties for Democracy
-15.152
-0.33
3
0.348
11/12/1993
155021
Concert of Parties for Democracy
-10.493
4.85
3
11/12/1993
155061
Union for Chile’s Progress
25.347
9.13
5
2.656
12/12/1999
155061
Alliance for Chile
-5.017
9.58
5
12/12/1999
155021
Concert of Parties for Democracy
-13.596
3.71
0.13
2
0.146
11/12/2005
155025
Together we can do more for Chile
PODEMOS
-36.323
11/12/2005
155021
Concert of Parties for Democracy
-19.715
2.51
0.00
2
11/12/2005
155601
Independent Democratic Union
UDI
16.892
9.53
18.100
4
2.656
11/12/2005
155602
National Renewal
RN
-0.809
7.70
16.333
5
1.67
13/12/2009
155023
New Majority for Chile
-28.325
-0.3
13/12/2009
155025
Together we can do more for Chile
PODEMOS
-41.486
-21.6
13/12/2009
155021
Concert of Parties for Democracy
-18.857
4.62
-7.7
13/12/2009
155061
Coalition for Change
-3.029
7.78
0.42
8.7
17/11/2013
155024
If you want it, Chile changes
-37.011
17/11/2013
155061
Alliance
-28.011
17.333
17/11/2013
155021
New Majority for Chile/Socialist Party of Chile
-18.974
-0.30
6.333
-1.403
19/11/2017
155501
Christian Democratic Party
PDC
-29.713
10.000
0.297
20/11/2017
155021
New Majority for Chile
-24.938
21/11/2017
155026
Broad Front
FA
-39.175
22/11/2017
155061
Chile let’s go
3.958
23/11/2017
155201
Progressive Party
PRO
-31.919
24/11/2017
155980
Independent
-2.332
Sources cited in Table 1.
2.5.
Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Uruguay
edate
party
partyname
Party abbrev
rile MP
Alcantara Saez 2012 (1-10)
Oliveira 2015 VIdeo
PREPPS (1-20)
V-party 2020 (-5 - +5)
26/10/2014
162210
Popular Assembly
UP
-43.399
1.750
26/10/2014
162510
Independent Party
PI
-0.077
9.250
26/10/2014
162410
Colorado Party
PC
-8.831
8.05
15.625
1.259
26/10/2014
162610
National Party
PN
-3.24
7.66
13.875
1.268
26/10/2014
162310
Broad Front
FA
-21.2
3.71
0.43
4.750
-1.354
Sources cited in Table 1.
. The correlation between the RILE scale and each one of the other five measures is shown in table 1.
We can see that the RILE scale has a strong and significant correlation with two other measures: Alcántara Saez (2012)ALCÁNTARA SAEZ, Manuel. Elections in Latin America 2009-2011: a comparative analysis. Kellogg Institute Working Papers, n. 386, 2012. Disponível em: Disponível em: https://kellogg.nd.edu/sites/default/files/old_files/documents/386_0.pdf Acesso em: 17 fev. 2020.
https://kellogg.nd.edu/sites/default/fil...
and Huber et al. (2008)HUBER, Evelyne et al. Politics and social spending in latin america. The Journal of Politics, v. 70, n. 2, p. 420-436, 2008. Disponível em: Disponível em: https://www.jstor.org/stable/10.1017/s0022381608080407 Acesso em: 28 set. 2019.
https://www.jstor.org/stable/10.1017/s00...
. This result indicates that the ideological position of Latin American parties does not vary much across assessments. Legislators and experts provide measures very similar to those computed from manifestos.
On the other hand, it is noteworthy that both measures of the RILE scale provided by Latin American independent coders D’Alessandro (2013)D’ALESSANDRO, Martín. Las plataformas electorales en la Argentina moderna. América Latina Hoy, v. 65, p. 107-139, 2013. Disponível em: Disponível em: https://revistas.usal.es//index.php/1130-2887/article/view/alh201365107139 Acesso em: 10 fev. 2014
https://revistas.usal.es//index.php/1130...
and López; Miranda; Valenzuela-Gutiérrez (2013)LÓPEZ, Miguel Ángel; MIRANDA, Nicolás; VALENZUELA-GUTIÉRREZ, Pablo. Estimando el espacio politico del cono sur y Brasil: las elecciones presidenciales en el eje izquierda-derecha. PostData, v. 18, n. 2, p. 403-442, 2013. Disponível em: Disponível em: https://www.researchgate.net/publication/260837426_Estimando_el_espacio_politico_del_cono_sur_y_brasil_las_elecciones_presidenciales_en_el_eje_izquierda-derecha Acesso em: 16 jan. 2014.
https://www.researchgate.net/publication...
, which applied the same MP categories and protocol, do not correlate to the MP Rile scale at all, as lines 8 and 9 in Table 1 and Graph 2 show. Both, however, are highly correlated to other measures, although their small n hinders statistical significance. This disparity suggests a coder bias not resolvable by the MP protocol.
Another possible cause of the weak correlations is the presumable ambiguity of center parties. As Graph 3 shows, most of MP RILE measures concentrate in the center of the scale, between -30,0 and +20,0, in a scale from -100 to +100. Several alternative assessments spread their measures more widely from left to right.
Unfortunately, as Table 2 shows, most measures are available only for a small number of parties, preventing us from performing a factor analysis (which would be a great contribution to our knowledge of ideology in Latin America).
The comparison between distinct measures of party ideology in Latin America allows us to draw three important conclusions. First, some of these measures (Alcántara Saez 2012ALCÁNTARA SAEZ, Manuel. Elections in Latin America 2009-2011: a comparative analysis. Kellogg Institute Working Papers, n. 386, 2012. Disponível em: Disponível em: https://kellogg.nd.edu/sites/default/files/old_files/documents/386_0.pdf Acesso em: 17 fev. 2020.
https://kellogg.nd.edu/sites/default/fil...
; Huber et al., 2008HUBER, Evelyne et al. Politics and social spending in latin america. The Journal of Politics, v. 70, n. 2, p. 420-436, 2008. Disponível em: Disponível em: https://www.jstor.org/stable/10.1017/s0022381608080407 Acesso em: 28 set. 2019.
https://www.jstor.org/stable/10.1017/s00...
) seem to capture the same latent variable as MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
), whatever label we assign it. Second, the three applications of the MP protocol return very distinct measures of party positions at the RILE scale. Third, three measures are highly and statistically correlated to one another (BLS, PREPPS, and V-Party), suggesting that we do have good measures of ideology validated across distinct methods (legislators and expert surveys), but they do not include manifestos content analyses.
These findings have two implications. The first is methodological: survey methods produce quite similar ideological positions, regardless of who applies them, but manifesto content analysis does not. The second implication is theoretical: across the distinct faces of parties (KATZ; MAIR, 1994KATZ, Richard; MAIR, Peter. How parties organize: change and adaptation in party organizations in western democracies. London: Sage, 1994.), legislators (who belong to the party in public office) and experts (who can also consider the ideology of the party on the ground and in the central office) may identify them in the same way.
Case study: Brazilian PT and PSDB manifestos
Before the MP launched its South America dataset, its method was already being applied to Brazilian political party manifestos (TAROUCO, 2011TAROUCO, Gabriela. Brazilian parties according to their manifestos: political identity and programmatic emphases. Brazilian Political Science Review v. 5, n. 1, p. 54-76, 2011. Disponível em: Disponível em: https://www.redalyc.org/pdf/3943/394341997003.pdf Acesso em: 25 fev. 2012.
https://www.redalyc.org/pdf/3943/3943419...
) and programs for Brazilian presidential elections (MADEIRA; VIEIRA; TAROUCO, 2017MADEIRA, Rafael, VIEIRA, Soraia; TAROUCO, Gabriela. Agendas, preferências, competição: PT e PSDB em disputas presidenciais. Caderno CRH, v. 30, n. 80, p. 257-273, 2017. ). Following MP instructions (VOLKENS et al., 2013VOLKENS, A. et al. The manifesto data collection. manifesto project (MRG/CMP/MARPOR). Version 2013b. Berlin, 2013. Disponível em: Disponível em: https://manifesto-project.wzb.eu/ . Acesso em: 23 ago. 2015.
https://manifesto-project.wzb.eu/...
), we coded (MADEIRA; VIEIRA; TAROUCO, 2017MADEIRA, Rafael, VIEIRA, Soraia; TAROUCO, Gabriela. Agendas, preferências, competição: PT e PSDB em disputas presidenciais. Caderno CRH, v. 30, n. 80, p. 257-273, 2017. ) six documents issued for the 2006, 2010 and 2014 elections by presidential candidates of the Workers’ Party (Partido dos Trabalhadores - PT) and the Brazilian Social Democracy Party (Partido da Social Democracia Brasileira - PSDB). These parties polarized political competition at the presidential level for many years, and their candidates occupied the federal executive office from 1994 to 2016. None of us received MP training, nor were we submitted to its tests or hired by the project. This was an independent initiative.
The first step of hand-coded content analysis of party manifestos is the identification of the coding units. According to MP coding instructions (VOLKENS et al., 2013VOLKENS, A. et al. The manifesto data collection. manifesto project (MRG/CMP/MARPOR). Version 2013b. Berlin, 2013. Disponível em: Disponível em: https://manifesto-project.wzb.eu/ . Acesso em: 23 ago. 2015.
https://manifesto-project.wzb.eu/...
), the coding unit is each sentence segment that contains a single argument - the quasi-sentence. The quantity of identified quasi-sentences is the “n” of each document (the number of observations of units of analysis). Despite the detailed instructions on how to identify quasi-sentences, this procedure appeared to be very susceptible to coder bias. The fact that one coder can identify fewer, bigger quasi-sentences while another splits the same document into more, shorter units is revealing.
This initial procedure, when performed by distinct coders, results in diverging numbers of coding units, thus affecting the quantification of the policy content of each document and preventing further tests for inter-coder reliability. As table 3 shows, MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) coders systematically identified more and shorter units than we did.14
14
During the segmentation step, we teamed up to consolidate our individual segmentations in a final, unified one.
The next step in the analysis is the classification of each unit according to the 56 categories in the MP coding frame. These categories constitute seven domains: external relations, freedom and democracy, political system, economy, welfare and quality of life, fabric of society, and social groups.
Despite detailed instructions and decision criteria provided in the MP coding manual, it is possible that distinct coders classify a same unit into different categories. This is the primary source of reliability problems in hand-coded content analysis. We conducted double-blind coding and checked our classifications against each other through the Kappa test of inter-coder reliability (COHEN, 1960COHEN, Jacob. A coefficient of agreement for nominal scales. Educational and Psychological Measurement, XX, n. 1, p. 37-46, 1960.), reaching high scores of agreement (MADEIRA; VIEIRA; TAROUCO, 2017MADEIRA, Rafael, VIEIRA, Soraia; TAROUCO, Gabriela. Agendas, preferências, competição: PT e PSDB em disputas presidenciais. Caderno CRH, v. 30, n. 80, p. 257-273, 2017. ).15
15
The values of the kappa index for the three pairs of MVT coders are 0.88 (TxV), 0.68 (VxM), and 0.66 (MxT). According to Landis and Koch (1977) and Bonnardel (2001), kappa values equal or greater than 0.81 indicate excellent agreement, and values between 0.61 and 0.80 are good.
The MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) coder, instead, worked by herself, after taking the MP training and having passed a pre-hiring test of reliability. Unfortunately, the distinct set of quasi-sentences resulting from the two distinct unitizing processes prevents a reliability test between MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) and MVT coding results.
An additional source of divergence is that we adopted the original standard scheme of 56 categories (BUDGE et al., 2001BUDGE, Ian et al. Mapping policy preferences: estimates for parties, electors, and governments 1945-1998. New York: Oxford University Press, 2001. 274 p. ), adding a set of specific sub-categories (ALONSO; VOLKENS; GÓMEZ, 2012ALONSO, Sonia; VOLKENS, Andrea; GÓMEZ, Braulio. Análisis de contenido de textos políticos. Un enfoque cuantitativo. Madrid: Centro de Investigaciones Sociologicas, 2012.).16
16
These subcategories were created in order to accommodate the specificities of new democracies, like Eastern and Central European countries and Mexico. They were then used to analyze Latin American party manifestos.
The MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) coders, instead, used a new set of sub-categories, added in the last version of the coding manual (WERNER; LACEWELL; VOLKENS, 2014WERNER, A.; LACEWELL, O.; VOLKENS, A. Manifesto coding instructions. (5th revised edition), 2014. Disponível em: Disponível em: https://manifesto-project.wzb.eu/information/documents/handbooks . Acesso em 23 ago. 2015.
https://manifesto-project.wzb.eu/informa...
).
Fortunately, the distinct sets of subcategories do not affect their clustering within domains, whose proportions in the documents can, then, be compared. Graphs 4 and 5 show that results agree on locating both PT and PSDB documents within a shared public agenda: economy and welfare issues concentrate most of the space in their programs for presidential elections in Brazil, from 2006 to 2014, according to the two coding results. Moreover, in both cases, the political system is the third priority category. It is also worth mentioning that the other domains are far less frequently used by both parties as observed in the two classifications (about 10%). The only exception is the uncoded units of PT’s 2010 manifesto by MVT.
Proportion of domains according to Madeira et al. (2017)MADEIRA, Rafael, VIEIRA, Soraia; TAROUCO, Gabriela. Agendas, preferências, competição: PT e PSDB em disputas presidenciais. Caderno CRH, v. 30, n. 80, p. 257-273, 2017. .
The differences are mostly related to categories in the “Freedom and Democracy” domain. It is also worth noting the disagreement between the two coder teams regarding the proportion of text that does not fit into any category. Uncoded sentences are much more frequent in the MVT coding than in the MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) coding. This is important because it illustrates how individual interpretations and personal bias can compromise the reliability in this kind of analysis.17
17
This difference may be related to a supposed emphasis that MP instructors put, while training the coders, on avoiding the ‘uncoded’ category, as we were recently warned about.
Table 4 shows the differences of means for each domain in the full set of documents according to the analysis of each coder team.
The main difference occurs in the proportion of domain 2. The MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) team identified many more statements on freedom and democracy than the MVT coders in the same set of texts.
The classifications for each sentence are not comparable between the two coding results because the sentences are not equal, as we said above. However, we can compare the proportions attributed by each team to each category in each document. Here we adapt two indicators of proportionality often used in institutional studies, in order to measure the congruence between the two coding results at the level of categories.
The first measure is the index of coalescence (AMORIM NETO, 2000AMORIM NETO, Octavio. Gabinetes presidenciais, ciclos eleitorais e disciplina legislativa no Brasil. Dados, v. 43, n. 3, 2000. Disponível em: Disponível em: https://www.scielo.br/j/dados/a/3xyyXsZD8NdvxFcSLJjbKrN/?lang=pt Acesso em: 24 nov. 2021
https://www.scielo.br/j/dados/a/3xyyXsZD...
) that is used to measure how much the distribution of portfolios in coalition governments is proportional to party seat share in the legislature. The index varies from 0 (no correspondence) to 1 (perfect correspondence). For our purposes, values near one in the coalescence index indicate a high similarity between the two coding results.
The second measure is an adaptation of the index of disproportionality (GALLAGHER, 1991GALLAGHER, Michael. Proportionality, disproportionality and electoral systems. Electoral Studies, v. 10, n. 1, p. 33-51, 1991.) used to estimate how much the assignment of legislative seats is proportional to the vote share of each party. That original index varies from 0 (perfectly proportional) to 100 (totally disproportional). For our purposes, values near to zero in the index of disproportionality indicate good similarity between the two coding results. Thus, we computed our index of proportionality by subtracting the original one from 100. To fit better in the graphs, we divided this index by 100, so it varies from 0 to 1, from the least to the most proportional.
We used both measures of congruence as a sort of alternative reliability test. A document’s two coding results are considered to be more congruent when both indexes (coalescence and proportionality) are high. We argue that congruent coding results could support a claim of reliability of the method even when the coding units are not the same. We calculated both indices for three distinct proportions: the proportion of each category in the entire set of sentences; the proportion of each category in the set of sentences coded with a valid category (excluding uncoded ones); and the proportion of each domain. Table 5 shows both measures of congruence between the two coding results for the six Brazilian documents.
Not surprisingly, for every document coding that we compared, the congruence is greater for domains than for categories. The exclusion of uncoded sentences also improves both measures of congruence. It is interesting to note that the highest and lowest congruences are found in programs for the same election. In 2010, the PSDB’s program was coded by the two teams with the most similar proportion of categories, while the PT’s program showed the most distinct proportions. Graph 6 shows how the congruence between the two codings varies among documents and over time.
The congruence between proportions of categories (excluded the uncoded) in each of the two measures is rather appreciable. The index of proportionality is always higher than 0.85, thus much closer to one than to zero. The index of coalescence is always higher than 0.5, reaching 0.66 in 2010. If the proportions of categories are not so different, we should expect to find similar results for the scales built on those categories. The next section describes the disparities in the left-right positions.
Implications: comparing ideological positions
Up to this point, we have discussed the reliability of the method of text unit coding. Seemingly, the hand-coded content analysis of party manifestos produces quite different results when applied by distinct coders, even when they follow the same rules. However, the greater or lesser relevance of these disparities depends on their implications for substantive analysis. In this section, we try to validate MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) coding results by comparing the positions in the RILE scale to MVT coding of Brazilian manifestos.
The MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) database encompasses the RILE scale, a left-right scale based on a specific set of categories.18
18
Laver and Budge (1992) built the RILE scale using a series of exploratory factors to identify possible combinations of variables that would reflect either the left or the right position.
A manifesto position in the scale is computed by deducting the percentage of left categories from the percentage of the right categories. The RILE scale varies from -100 (if all units were coded with left categories) to +100 (if the whole text was coded with right categories).
The availability of two codings (MP and MVT) of the same six Brazilian documents enables us to compare each manifesto’s values on that scale. Graph 7 shows both measures for the six documents.
Graph 7 shows how distinct are the RILE scale positions resulting from the two codings. Although the two measures locate all documents in the left field (negative values in the RILE scale), there is an important difference in PT programs’ positions for the 2006 and 2010 elections. While the RILE scale applied to the MP coding shows the party moving seven points from left to right over time, the same scale applied to the Brazilian coding shows PT’s return in 2014 to practically the same position of 2006, after a deepening of its left position in 2010. As for the PSDB documents, the main difference is also in the change from 2006 to 2010: despite the similarity between the MP and MVT measures for 2006, the MVT scale shows a much deeper displacement toward left than the MP scale.
The scores in the RILE scale measured for the two codings are more distant in some documents (PSDB 2014, PSDB 2010, and PT 2006) than in the others. However, both measures result in changes over time in the same direction. The PSDB and PT movements toward the center from 2010 to 2014 are detected in both assessments. The same happens to the PSDB, toward left from 2006 to 2010. The only divergence in direction is the PT’s movement from 2006 to 2010.
The distinct measures of RILE positions have implications on how we analyze the Brazilian party system dynamics. Although both measures point to similar positions for the 2014 PT program, the contrasting results for its positions in 2006 lead to opposite interpretations about ideological changes and competitive strategies in the presidential elections in Brazil. MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) assessments would support interpretations that PT has consistently softened its left platform during its presidential mandates. Alternatively, the MVT measures would support interpretations about the ideological convergence of the two main parties in Brazilian presidential elections within a bipolar mode of competition.
The divergences do not seem to be related to the coders. While the MVT team kept the same three coders for all elections and documents, the MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) coder was replaced for the 2014 documents’ analysis. However, there are considerable divergences in the coding of documents of all three elections.
So, where the differences come from? We searched for different proportions of RILE categories between the MP and MVT coders. While the same MVT coders coded the six documents, in MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) , two different coders worked on different documents. They are labeled in the MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) dataset as coders 318 and 328, and worked on the four documents of 2006 and 2010 elections and on the two documents of 2014 election, respectively. Since the MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) coders worked on different documents, we cannot compare them to each other. Tables showing the differences of means between coders for each category of the RILE scale are available in the appendix
Appendix.
1a.
Means in RILE categories, for four Brazilian documents of 2006 and 2010 elections, MP and MVT coders
Category
Coder
Mean
Std. Dev.
Right: 104 - Military (positive)
MP318
0.894
0.460
MVT
0.700
0.707
Right: 201 - Freedom and human rights (positive)20
MP318
1.901
1.442
MVT
0.650
0.480
Right: 203 - Constitutionalism (positive)
MP318
0.252
0.291
MVT
0.000
0.000
Right: 305 - Political authority (positive)21
MP318
3.288
3.781
MVT
0.000
0.000
Right: 401 - Free enterprise (positive)
MP318
0.223
0.281
MVT
0.050
0.050
Right: 402 - Economic Incentives (positive)
MP318
2.212
0.885
MVT
3.275
2.205
Right: 407 - Protectionism (negative)
MP318
0.427
0.070
MVT
0.000
0.000
Right: 414 - Economic orthodoxy (positive)
MP318
1.918
0.618
MVT
0.325
0.650
Right: 505 - Welfare state limitation (positive)
MP318
0.103
0.206
MVT
0.000
0.000
Right: 601 - National way of life (positive)22
MP318
0.974
0.876
MVT
0.000
0.000
Right: 603 - Traditional morality (positive)
MP318
0.165
0.227
MVT
0.000
0.000
Right: 605 - Law and order (positive)23
MP318
4.076
0.273
MVT
4.650
1.303
Right: 606 - Social harmony (positive)24
MP318
0.937
0.713
MVT
0.050
0.050
Left: 103 - Anti-imperialism (positive)25
MP318
1.187
1.276
MVT
0.000
0.000
Left: 105 - Military (negative)
MP318
0.031
0.062
MVT
0.000
0.000
Left: 106 - Peace (positive)
MP318
0.527
0.700
MVT
0.400
0.616
Left: 107 - Internationalism (positive)
MP318
1.873
1.688
MVT
2.025
1.323
Left: 202 - Democracy (positive)26
MP318
5.289
3.253
MVT
1.800
1.804
Left: 403 - Market regulation (positive)
MP318
1.809
1.000
MVT
0.025
0.050
Left: 404 - Economic planning (positive)
MP318
1.127
0.287
MVT
0.475
0.591
Left: 406 - Protectionism (positive)
MP318
0.166
0.247
MVT
0.425
0.506
Left: 412 - Controlled economy (positive)
MP318
0.407
0.303
MVT
0.000
0.000
Left: 413 - Nationalization (positive)
MP318
0.245
0.093
MVT
0.000
0.000
Left: 504 - Welfare state expansion (positive)
MP318
9.518
1.316
MVT
11.450
5.500
Left: 506 - Education expansion (positive)
MP318
5.737
0.813
MVT
6.300
1.503
Left: 701 - Labour groups (positive)
MP318
2.639
0.962
MVT
0.950
0.929
Source: Krause et al. (2019) and MVT (2017)
1b.
Means in RILE categories, for two Brazilian documents of 2014 election, MP and MVT coders
Category
Coder
Mean
Std. Dev.
Right: 104 - Military (positive)
MP328
0.565
0.799
MVT
0.050
0.050
Right: 201 - Freedom and human rights (positive) 27
MP328
1.369
0.498
MVT
1.700
2.121
Right: 203 - Constitutionalism (positive)
MP328
0.000
0.000
MVT
0.000
0.000
Right: 305 - Political authority (positive)28
MP328
3.349
3.297
MVT
0.000
0.000
Right: 401 - Free enterprise (positive)
MP328
1.499
1.876
MVT
0.000
0.000
Right: 402 - Economic Incentives (positive)
MP328
4.557
0.371
MVT
5.750
1.061
Right: 407 - Protectionism (negative)
MP328
0.283
0.400
MVT
0.000
0.000
Right: 414 - Economic orthodoxy (positive)
MP328
1.988
0.864
MVT
0.150
0.150
Right: 505 - Welfare state limitation (positive)
MP328
0.452
0.639
MVT
0.000
0.000
Right: 601 - National way of life (positive) 29
MP328
0.398
0.076
MVT
0.000
0.000
Right: 603 - Traditional morality (positive)
MP328
0.000
0.000
MVT
0.000
0.000
Right: 605 - Law and order (positive) 30
MP328
4.035
1.325
MVT
4.250
1.768
Right: 606 - Social harmony (positive) 31
MP328
1.983
1.831
MVT
0.050
0.050
Left: 103 - Anti-imperialism (positive) 32
MP328
0.086
0.122
MVT
0.000
0.000
Left: 105 - Military (negative)
MP328
0.000
0.000
MVT
0.000
0.000
Left: 106 - Peace (positive)
MP328
0.172
0.243
MVT
0.000
0.000
Left: 107 - Internationalism (positive)
MP328
1.598
0.661
MVT
1.650
0.071
Left: 202 - Democracy (positive) 33
MP328
4.274
0.771
MVT
0.900
0.990
Left: 403 - Market regulation (positive)
MP328
0.909
0.313
MVT
1.550
2.192
Left: 404 - Economic planning (positive)
MP328
0.568
0.316
MVT
0.000
0.000
Left: 406 - Protectionism (positive)
MP328
0.229
0.163
MVT
0.350
0.350
Left: 412 - Controlled economy (positive)
MP328
0.775
1.095
MVT
0.050
0.050
Left: 413 - Nationalization (positive)
MP328
0.086
0.122
MVT
0.000
0.000
Left: 504 - Welfare state expansion (positive)
MP328
9.154
2.235
MVT
12.450
5.162
Left: 506 - Education expansion (positive)
MP328
7.887
0.287
MVT
6.750
2.192
Left: 701 - Labour groups (positive)
MP328
3.306
0.680
MVT
0.000
0.000
Source: Krause et al. (2019) and MVT (2017)
2.1.
Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Argentina
edate
party
partyname
partyabbrev
rile MP
Alcantara Saez 2012 (1-10)
Oliveira 2015 VIdeo
Huber et al. 2008 num
PREPPS (1-20)
Lopez et al 2013
D’Alessandro 2013
V-party 2020 (-5 - +5)
14/05/1989
150040
Center Alliance
13.287
4
-1.30
14/05/1989
150029
Justicialist Party of Popular Unity
FREJUPO
-3.82
4
-5.34
-0.688
14/05/1989
150301
Civic Radical Union
UCR
-19.416
3
-10.66
-0.148
14/05/1995
150202
Front for a Country in Solidarity
FREPASO
-15.217
2
-20.59
-1.198
14/05/1995
150201
Justicialist Party
PJ
1.997
7.40
4
-6.27
1.174
14/05/1995
150301
Civic Radical Union
UCR
-11.136
5.63
3
-16.06
0.038
24/10/1999
150601
Action for the Republic
AR
0.176
4
-2.10
2.251
24/10/1999
150028
Justicialist Coalition for Change
-6.644
7.78
4
-4.84
0.825
24/10/1999
150032
Alliance for Work, Justice, and Education
ALIANZA
-13.04
5.57
0.23
3
-13.09
27/04/2003
150205
Front of the popular movement
MP
4.167
-4.76
27/04/2003
150210
Alternative for a Republic of Equals
ARI
-26.135
-0.597
27/04/2003
150401
Recreate for Growth
RECREAR
-0.381
9.26
27/04/2003
150025
Front for Victory
FpV
-22.667
-0.17
-1.25
27/04/2003
150022
Front for Loyalty and Union of the Democratic Centre
FPL + UCeDé
-7.958
6.56
5.02
-0.405
28/10/2007
150027
Justice, Union and Liberty Front
FREJULI
-26.923
28/10/2007
150031
An Advanced Union
UNA
-13.213
-5.31
28/10/2007
150023
Civic Coalition
CC
-11.006
4.71
-0.269
28/10/2007
150025
Front for Victory
FpV
-20.755
4.79
-0.56
-10.71
-1.669
28/06/2009
150024
Social and Civic Agreement
ACyS
-20
28/06/2009
150220
New Encounter
-16.346
28/06/2009
150221
Neuquén People’s Movement
MPN
-12.027
28/06/2009
150222
It is possible
-18.957
28/06/2009
150025
Front for Victory
FpV
-40
-1.663
28/06/2009
150061
Alliance Union-PRO
-18.023
1.537
23/10/2011
150027
Federal Commitment
1.245
43.5
23/10/2011
150230
Popular Front
-50
29.8
23/10/2011
150026
Broad Progressive Front
FAP
-17.351
3.71
2.5
-19.91
-1.274
23/10/2011
150301
Civic Radical Union
UCR
-13.333
6.35
10.360
5.0
-21.56
0.197
23/10/2011
150025
Front for Victory
FpV
-3.425
6.37
-1.00
7.923
-1.4
-23.38
-1.677
27/10/2013
150026
Broad Progressive Front
FAP
-6.838
27/10/2013
150301
Civic Radical Union
UCR
-15.094
11.375
27/10/2013
150025
Front for Victory
FpV
0
7.083
-1.654
Sources cited in Table 1.
2.2.
Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Bolivia
edate
party
partyname
partyabbrev
rile MP
Alcantara Saez 2012 (1-10)
Oliveira 2015 VIdeo
PREPPS (1-20)
V-party 2020 (-5 - +5)
06/12/2009
151255
Social Alliance
AS
8.832
06/12/2009
151617
Autonomy for Bolivia
APB
-3.091
06/12/2009
151062
Plan Progress for Bolivia - National Convergence
PPB-CN
-38.356
2.253
06/12/2009
151610
Nationalist Revolutionary Movement
MNR
11.64
8.28
06/12/2009
151640
National Unity Front
UN
-2.381
8.28
06/12/2009
151250
Movement towards Socialism - Political Instrument for the Sovereignty of the Peoples
MAS-IPSP
-13.401
4.17
-0.26
-2.192
12/10/2014
151640
Democratic Union
UD
-3.287
12/10/2014
151540
Christian Democratic Party
PDC
-14.107
17.667
1.34
12/10/2014
151250
Movement towards Socialism - Political Instrument for the Sovereignty of the Peoples
MAS-IPSP
-17.41
-0.73
4.900
-2.192
Sources cited in Table 1.
2.3.
Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Brazil
edate
party
partyname
partyabbrev
rile MP
Zucco and Power 2019 (1-10)
Alcantara Saez 2012 (1-10)
Oliveira 2015 VIdeo
Huber et al. 2008 num
PREPPS (1-20)
Gonzales y Queirolo 2013 num
Lopez et al 2013
V-party 2020 (-5 - +5)
15/11/1989
180250
Democratic Labour Party
PDT
2
15/11/1989
180610
Democratic Social Party
PDS
5
15/11/1989
180410
New Brazil Movement
10.733
7.4
15/11/1989
180230
Popular Front Brazil
-46.855
1.5
1
15/11/1989
180309
Party of Brazilian Social Democracy
PSDB
32.836
3.9
2
03/10/1994
180710
Party of the Reconstruction of the National Order
PRONA
-27.333
03/10/1994
180230
A democratic revolution in Brazil
-27.142
2.0
1
-2.609
03/10/1994
180310
Let’s work Brazil
4.843
4.4
0.28
0.37
04/10/1998
180232
Socialist People’s Party
PPS
-15.974
3.0
1
04/10/1998
180230
Union of the People - Change Brazil
-19.905
1.9
1
-2.227
04/10/1998
180310
Go ahead Brazil
-18.318
5.9
0.33
2
0.936
06/10/2002
180320
Labour Front Coalition
-20.711
3.8
06/10/2002
180240
Hope for Brasil Coalition
-12.53
2.9
-1.639
06/10/2002
180310
Coalition Big Alliance
-10.523
6.1
4
0.881
06/10/2002
180230
Coalition Lula for President
-21.955
2.2
-0.10
1
-1.437
01/10/2006
180231
Left Front
-51.128
2.955
01/10/2006
180310
Coalition for a decent Brazil
-9.202
5.9
6.89
13.462
1.071
01/10/2006
180230
Coalition the People’s Power
-19.005
3.6
4.44
-0.29
6.370
-1.437
03/10/2010
180110
Green Party
PV
-28.302
4.6
8.158
03/10/2010
180310
Brazil can do more
-11.302
5.8
7.13
12.000
40.4
1.071
03/10/2010
180230
For Brazil to keep on changing
-13.636
3.6
4.73
0.00
5.429
13.1
-1.292
05/10/2014
180240
United for Brazil
-5.394
4.1
-1.131
05/10/2014
180310
Change, Brazil
-5.085
6.0
13.833
1.516
05/10/2014
180230
With the strength of the people
-12.048
3.7
-0.22
6.222
-1.45
07/10/2018
180620
Brazil above everything, God above everyone
42.466
18.355
3.242
07/10/2018
180230
The People happy again
-26.123
2.7
5.044
-1.897
07/10/2018
180251
Sovereign Brazil
-20.298
3.8
6.661
-1.187
07/10/2018
180310
To unite Brazil
4.474
7.1
14.772
1.416
Sources cited in Table 1.
2.4.
Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Chile
edate
party
partyname
partyabbrev
rile MP
Alcantara Saez 2012 (1-10)
Oliveira 2015 VIdeo
Huber et al 2008 num
PREPPS (1-20)
Gonzales y Queirolo 2013 num
Lopez et al 2013
V-party 2020 (-5 - +5)
14/12/1989
155620
Liberal-Socialists of Chile
-3.968
14/12/1989
155061
Democracy and Progress
21.405
5
2.656
14/12/1989
155021
Concert of Parties for Democracy
-15.152
-0.33
3
0.348
11/12/1993
155021
Concert of Parties for Democracy
-10.493
4.85
3
11/12/1993
155061
Union for Chile’s Progress
25.347
9.13
5
2.656
12/12/1999
155061
Alliance for Chile
-5.017
9.58
5
12/12/1999
155021
Concert of Parties for Democracy
-13.596
3.71
0.13
2
0.146
11/12/2005
155025
Together we can do more for Chile
PODEMOS
-36.323
11/12/2005
155021
Concert of Parties for Democracy
-19.715
2.51
0.00
2
11/12/2005
155601
Independent Democratic Union
UDI
16.892
9.53
18.100
4
2.656
11/12/2005
155602
National Renewal
RN
-0.809
7.70
16.333
5
1.67
13/12/2009
155023
New Majority for Chile
-28.325
-0.3
13/12/2009
155025
Together we can do more for Chile
PODEMOS
-41.486
-21.6
13/12/2009
155021
Concert of Parties for Democracy
-18.857
4.62
-7.7
13/12/2009
155061
Coalition for Change
-3.029
7.78
0.42
8.7
17/11/2013
155024
If you want it, Chile changes
-37.011
17/11/2013
155061
Alliance
-28.011
17.333
17/11/2013
155021
New Majority for Chile/Socialist Party of Chile
-18.974
-0.30
6.333
-1.403
19/11/2017
155501
Christian Democratic Party
PDC
-29.713
10.000
0.297
20/11/2017
155021
New Majority for Chile
-24.938
21/11/2017
155026
Broad Front
FA
-39.175
22/11/2017
155061
Chile let’s go
3.958
23/11/2017
155201
Progressive Party
PRO
-31.919
24/11/2017
155980
Independent
-2.332
Sources cited in Table 1.
2.5.
Compared measures of ideological positions (only parties with other measures than RILE scale from MP): Uruguay
edate
party
partyname
Party abbrev
rile MP
Alcantara Saez 2012 (1-10)
Oliveira 2015 VIdeo
PREPPS (1-20)
V-party 2020 (-5 - +5)
26/10/2014
162210
Popular Assembly
UP
-43.399
1.750
26/10/2014
162510
Independent Party
PI
-0.077
9.250
26/10/2014
162410
Colorado Party
PC
-8.831
8.05
15.625
1.259
26/10/2014
162610
National Party
PN
-3.24
7.66
13.875
1.268
26/10/2014
162310
Broad Front
FA
-21.2
3.71
0.43
4.750
-1.354
Sources cited in Table 1.
.
Another aspect of this difference is that, although there is room for positions ranging from -100 to +100, in fact, the distance between the PT and the PSDB is small (their positions vary between -5 and -20) according to the two assessments, showing that the manifestos effectively occupy a narrow range of the left-right dimension.
From the 26 categories in the RILE scale, MP and MVT teams found very different proportions of four right categories and four left categories. MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) coders identified left-right content in the documents systematically more often than the MVT team, whose means in these categories are always smaller, or even zero in several cases.
Differences reach more categories when we compare MVT codings to those conducted by MP318 than when compared to codings by MP328. This is possibly related to the differences showed in graph 7, since MP318 coded 2006 and 2010 documents, exactly those for which the values in the RILE scale showed major differences between MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) and MVT calculations.
The category that generates the main differences between MVT and both MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) coders is “701 - Labour groups”, defined as “Favorable references to all labor groups, the working class, and unemployed workers in general. Support for trade unions and calls for the good treatment of all employees, including: More jobs; Good working conditions; Fair wages; Pension provisions etc.” (WERNER; LACEWELL; VOLKENS, 2014WERNER, A.; LACEWELL, O.; VOLKENS, A. Manifesto coding instructions. (5th revised edition), 2014. Disponível em: Disponível em: https://manifesto-project.wzb.eu/information/documents/handbooks . Acesso em 23 ago. 2015.
https://manifesto-project.wzb.eu/informa...
). This category is used much more often by the two MP coders than by the MVT team. Unfortunately, we cannot compare measured parties’ position between MP coders, since they did not code the same documents.
The comparison between the scales resulting from the two independent codings of Brazilian manifestos leads us to question their adoption as proxies for parties’ ideology. The Brazilian case is an example of the challenges posed by hand-coded content analysis. Coders express disparate interpretations, and, therefore, the scales computed from the distinct codings show distinct ideological positions for the same documents. Discrepant measures of ideology cause discrepant interpretations of the party competition. Having only one coding of each document (as is the case for the MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) measures) hinders us from knowing the magnitude of that discrepancy.
What makes the analysis of the manifestos comparable (JORGE; FARIA; SILVA, 2020JORGE, Vladymir Lombardo; FARIA, Alessandra Maia Terra de; SILVA, Mayra Goulart da. Posicionamento dos partidos políticos brasileiros na escala esquerda-direita: dilemas metodológicos e revisão da literatura. Revista Brasileira de Ciência Política, n. 33, 2020. Disponível em: Disponível em: https://www.scielo.br/j/rbcpol/a/XNBnwhWwbSsMPFrj4zmHQsG/?lang=pt Acesso em: 20 jan. 2021.
https://www.scielo.br/j/rbcpol/a/XNBnwhW...
) is also what allows us to grasp its fragility: the coding protocol published and openly available to any researcher. Following the same techniques and categories used by MP, we found in Brazil, years before the launching of their dataset for South America, different proportions of contents in the same texts.19
29
Comprises subcategories 601_1 and 601_2 used by MP coders.
Other authors also reached distinct results in the analysis of Latin American manifestos (D’ALESSANDRO, 2013D’ALESSANDRO, Martín. Las plataformas electorales en la Argentina moderna. América Latina Hoy, v. 65, p. 107-139, 2013. Disponível em: Disponível em: https://revistas.usal.es//index.php/1130-2887/article/view/alh201365107139 Acesso em: 10 fev. 2014
https://revistas.usal.es//index.php/1130...
; LÓPEZ; MIRANDA; VALENZUELA-GUTIÉRREZ, 2013LÓPEZ, Miguel Ángel; MIRANDA, Nicolás; VALENZUELA-GUTIÉRREZ, Pablo. Estimando el espacio politico del cono sur y Brasil: las elecciones presidenciales en el eje izquierda-derecha. PostData, v. 18, n. 2, p. 403-442, 2013. Disponível em: Disponível em: https://www.researchgate.net/publication/260837426_Estimando_el_espacio_politico_del_cono_sur_y_brasil_las_elecciones_presidenciales_en_el_eje_izquierda-derecha Acesso em: 16 jan. 2014.
https://www.researchgate.net/publication...
). We ascribe these differences to the known weaknesses of the method: the unavoidable bias of human coders. Bias does not belittle the hard work of so many MP coders, nor is it exclusive to this method. However, we argue that the bias must be identified- and reduced - through intercoder reliability tests.
Beyond these methodological concerns, we hope to shed some light on the debate about concepts of left and right. The weak correlations among some distinct measures in Latin American data make us wonder about the different contents being measured. For instance, in Latin America, measures of ideological positions regarding economic issues (OLIVEIRA, 2015OLIVEIRA, Augusto Neftali. A ideologia econômica dos presidentes na América Latina: um indicador a partir dos programas de governo eleitorais. Indicadores Econômicos FEE, v. 43, n. 1, p. 83-98, 2015.) do not match either experts’ surveys measures (WIESEHOMEIER, 2011WIESEHOMEIER, Nina. Policy positions of 47 political parties and 4 presidents in 4 Latin American countries. In: PREPPS: The political representation, executives, and political parties survey: banco de dados [Internet]. 2011. Disponível em: Disponível em: http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA Acesso em: 10 abr. 2020.
http://ninaw.webfactional.com/prepps e e...
; WIESEHOMEIER, 2015WIESEHOMEIER, Nina. Policy positions of 165 political parties and 18 presidents in 18 Latin American countries. In: PREPPS: The political representation, executives, and political parties survey: banco de dados [Internet]. 2015. Disponível em: Disponível em: http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA Acesso em: 10 abr. 2020.
http://ninaw.webfactional.com/prepps e e...
; WIESEHOMEIER; BENOIT, 2006WIESEHOMEIER, Nina; BENOIT, K. Policy positions of 146 political parties and 18 presidents in 18 Latin American countries. In: PREPPS: The political representation, executives, and political parties survey: banco de dados [Internet]. 2006. Disponível em: Disponível em: http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA Acesso em: 10 abr. 2020.
http://ninaw.webfactional.com/prepps e e...
; WIESEHOMEIER; SINGER, 2019WIESEHOMEIER, Nina; SINGER, Mathew.; RUTH-LOVELL, Saskia. Political representation, executives, and political parties survey: Data from Expert Surveys in 18 Latin American Countries, 2018-2019. IE University, University of Connecticut and GIGA Institute Hamburg. 2019. Disponível em: Disponível em: https://dataverse.harvard.edu/dataverse/prepps . Acesso em: 23 abr. 2020.
https://dataverse.harvard.edu/dataverse/...
) or that of the RILE scale (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
; D’ALESSANDRO, 2013D’ALESSANDRO, Martín. Las plataformas electorales en la Argentina moderna. América Latina Hoy, v. 65, p. 107-139, 2013. Disponível em: Disponível em: https://revistas.usal.es//index.php/1130-2887/article/view/alh201365107139 Acesso em: 10 fev. 2014
https://revistas.usal.es//index.php/1130...
; LÓPEZ; MIRANDA; VALENZUELA-GUTIÉRREZ, 2013LÓPEZ, Miguel Ángel; MIRANDA, Nicolás; VALENZUELA-GUTIÉRREZ, Pablo. Estimando el espacio politico del cono sur y Brasil: las elecciones presidenciales en el eje izquierda-derecha. PostData, v. 18, n. 2, p. 403-442, 2013. Disponível em: Disponível em: https://www.researchgate.net/publication/260837426_Estimando_el_espacio_politico_del_cono_sur_y_brasil_las_elecciones_presidenciales_en_el_eje_izquierda-derecha Acesso em: 16 jan. 2014.
https://www.researchgate.net/publication...
). So, if we need to mobilize political party ideology as a variable, the first challenge before choosing a measure is how to define it.
Unlike the MP (KRAUSE et al., 2019KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b . Acesso em: 26 dez. 2020.
https://doi.org/10.25522/manifesto.mpdss...
) measures, other measures validate each other. The surprisingly high correlation between two of our compared measures is worth mentioning. Measures based on an expert survey (WIESEHOMEIER, 2011WIESEHOMEIER, Nina. Policy positions of 47 political parties and 4 presidents in 4 Latin American countries. In: PREPPS: The political representation, executives, and political parties survey: banco de dados [Internet]. 2011. Disponível em: Disponível em: http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA Acesso em: 10 abr. 2020.
http://ninaw.webfactional.com/prepps e e...
; WIESEHOMEIER, 2015WIESEHOMEIER, Nina. Policy positions of 165 political parties and 18 presidents in 18 Latin American countries. In: PREPPS: The political representation, executives, and political parties survey: banco de dados [Internet]. 2015. Disponível em: Disponível em: http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA Acesso em: 10 abr. 2020.
http://ninaw.webfactional.com/prepps e e...
; WIESEHOMEIER; BENOIT, 2006WIESEHOMEIER, Nina; BENOIT, K. Policy positions of 146 political parties and 18 presidents in 18 Latin American countries. In: PREPPS: The political representation, executives, and political parties survey: banco de dados [Internet]. 2006. Disponível em: Disponível em: http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA Acesso em: 10 abr. 2020.
http://ninaw.webfactional.com/prepps e e...
; WIESEHOMEIER; SINGER, 2019WIESEHOMEIER, Nina; SINGER, Mathew.; RUTH-LOVELL, Saskia. Political representation, executives, and political parties survey: Data from Expert Surveys in 18 Latin American Countries, 2018-2019. IE University, University of Connecticut and GIGA Institute Hamburg. 2019. Disponível em: Disponível em: https://dataverse.harvard.edu/dataverse/prepps . Acesso em: 23 abr. 2020.
https://dataverse.harvard.edu/dataverse/...
) and a parliamentary survey (ZUCCO; POWER, 2019ZUCCO, Cesar; POWER, Timothy. Brazilian legislative surveys (Waves 1-8, 1990-2017). Harvard Dataverse, V2, UNF:6:OZ4b3ybYKksC255DkruPxg== [fileUNF] 2019. Disponível em: Disponível em: https://dataverse.harvard.edu/dataverse/bls Acesso em:04 abr. 2020.
https://dataverse.harvard.edu/dataverse/...
) are almost the same for 10 Brazilian political parties. In both cases, Brazilian respondents (both political scientists and legislators respectively) were simply asked to locate parties on the left-right scale, with no explanation about its meaning.
Concluding remarks and research agenda
When we think of ways to analyze ideologies and classify ideological positions, we must always bear in mind the fact that ideology cannot be directly observed. There is no concrete metric or measure directly observable and capable of accounting for (and exhausting) the ideological classification of any political agent (leaders, parties, governments etc.). In addition to these difficulties, the ideological framework is a complex phenomenon that involves different dimensions of reality, such as: classifications based on agents’ actions (what they do effectively), classifications based on agents’ speech (what they say/affirm) and classifications based on their reputation (how others see a particular agent and locate her on the ideological spectrum). We assume that all these dimensions are important, and that none of them exhaust the topic.
That said, the most fruitful strategy for scholars, in our view, is not to discuss the best methodology or approach, or which one better explains this phenomenon. Rather, it is to superimpose and to compare the different strategies for measuring ideological positioning, so as to produce knowledge from the contrasts and approaches identified in the different analyses. The literature provides many strategies that attempt to measure ideological positioning. Each has advantages and disadvantages.
Examining and analyzing the rich (and complex) mosaic formed by the overlap of these studies is as important as analyzing and discussing the scope and limitations of each strategy by itself. Starting from very different points (and taking distinct paths), each approach presents us with the delimitation of one of the (numerous) manifestations of ideology as a political phenomenon.
Hand-coded content analysis of manifestos is very promising, but reliability is not its leading feature. The disparities that we found suggest that: 1) there is room for improvement of manual coding of manifestos content; 2) caution is needed when drawing conclusions about ideology based on the scales constructed with the codifications.
Although we acknowledge the relevance of the content analysis of manifestos, the question is how to seize the advantages of the hand-coded procedures and minimize their problems. Human coding allows for quantifying qualitative data and deductively describing political positions expressed in texts in terms of theoretically built categories. Human coders can identify meanings and interpret complex writings. Hand-coded analysis may feed dictionaries for automated analysis of other materials, that would be too numerous or too long to be coded by hand.
However, hand-coded analysis is vulnerable to human limitations and bias. In addition to different physical and mental capacities, distinct coders often disagree about what they read. Because of that, the reliability of the method depends on several challenges. The first is the quality of the coding scheme, coding instructions, and coding criteria. Another challenge is the evaluation of the coding results and, eventually, the revision of the coding work.
MP does an excellent job on the first challenge but can improve on the second one. Without inter-coder reliability tests for each document, the application of the data is vulnerable to a level of error that we cannot estimate. This is what we found by comparing MP measures to other ones.
Adopting inter-coder reliability tests for each document would, of course, make the process more expensive and slower, but would assure that we can seize the advantages of the method with more confidence. Keeping only one coder for each document certainly improves the viability of such an ambitious project, but we are not sure that the results are worthwhile. Mistaken measures of political positions could compromise the study of their effects, for example. Speed and volume are advantages that we should seek in automated analysis, not in hand-coded analysis.
We are aware of the great value of the MP project contribution and of the vast comparative possibilities it opens to political science (JORGE; FARIA; SILVA, 2020JORGE, Vladymir Lombardo; FARIA, Alessandra Maia Terra de; SILVA, Mayra Goulart da. Posicionamento dos partidos políticos brasileiros na escala esquerda-direita: dilemas metodológicos e revisão da literatura. Revista Brasileira de Ciência Política, n. 33, 2020. Disponível em: Disponível em: https://www.scielo.br/j/rbcpol/a/XNBnwhWwbSsMPFrj4zmHQsG/?lang=pt Acesso em: 20 jan. 2021.
https://www.scielo.br/j/rbcpol/a/XNBnwhW...
). Even so, we believe that, as for any other proxy, hand-coded analysis of manifestos must be carefully used, taking its limitations into account. With this paper we expect to contribute to advance knowledge, by pointing out those limitations, at least with regard to Latin American parties. Regarding the research agenda, the main task for future research is to compare the MP coding results to a computer-based analysis, to measure its strength vis-a-vis a non-human classification.
References
- ALCÁNTARA SAEZ, Manuel. Elections in Latin America 2009-2011: a comparative analysis. Kellogg Institute Working Papers, n. 386, 2012. Disponível em: Disponível em: https://kellogg.nd.edu/sites/default/files/old_files/documents/386_0.pdf Acesso em: 17 fev. 2020.
» https://kellogg.nd.edu/sites/default/files/old_files/documents/386_0.pdf - ALONSO, Sonia; VOLKENS, Andrea; GÓMEZ, Braulio. Análisis de contenido de textos políticos. Un enfoque cuantitativo. Madrid: Centro de Investigaciones Sociologicas, 2012.
- AMORIM NETO, Octavio. Gabinetes presidenciais, ciclos eleitorais e disciplina legislativa no Brasil. Dados, v. 43, n. 3, 2000. Disponível em: Disponível em: https://www.scielo.br/j/dados/a/3xyyXsZD8NdvxFcSLJjbKrN/?lang=pt Acesso em: 24 nov. 2021
» https://www.scielo.br/j/dados/a/3xyyXsZD8NdvxFcSLJjbKrN/?lang=pt - BONNARDEL, Philippe. The Kappa coefficient: The measurement of interrater agreement when the ratings are on categorical scales. The case of two raters. Le Coefficient Kappa. 2001. Disponível em: Disponível em: http://kappa.chez-alice.fr/kappa.txt Acesso em: 24 nov. 2011.
» http://kappa.chez-alice.fr/kappa.txt - BUDGE, Ian et al. Mapping policy preferences: estimates for parties, electors, and governments 1945-1998. New York: Oxford University Press, 2001. 274 p.
- BUDGE, Ian; MEYER, Thomas. Understanding and validating the left-right scale (RILE). In: VOLKENS, Andrea; BARA, Judith; BUDGE, Ian et al. (ed.). Mapping policy preferences from texts: statistical solutions for manifesto analysts. Oxford: Oxford University Press, 2013.
- COHEN, Jacob. A coefficient of agreement for nominal scales. Educational and Psychological Measurement, XX, n. 1, p. 37-46, 1960.
- COPPEDGE, Michael. A classification of Latin American political parties. Kellogg Institute Working Papers, n. 244, 1997. Disponível em: Disponível em: https://kellogg.nd.edu/sites/default/files/old_files/documents/244_0.pdf Acesso em: 20 fev. 2006.
» https://kellogg.nd.edu/sites/default/files/old_files/documents/244_0.pdf - D’ALESSANDRO, Martín. Las plataformas electorales en la Argentina moderna. América Latina Hoy, v. 65, p. 107-139, 2013. Disponível em: Disponível em: https://revistas.usal.es//index.php/1130-2887/article/view/alh201365107139 Acesso em: 10 fev. 2014
» https://revistas.usal.es//index.php/1130-2887/article/view/alh201365107139 - GALLAGHER, Michael. Proportionality, disproportionality and electoral systems. Electoral Studies, v. 10, n. 1, p. 33-51, 1991.
- GONZÁLEZ FERRER, Luis Eduardo; QUIEROLO VELASCO, Rosario. Izquierda y derecha: formas de definirlas, el caso latinoamericano y sus implicaciones. América Latina Hoy, v. 65, p. 79-105, 2013.
- HUBER, Evelyne et al. Politics and social spending in latin america. The Journal of Politics, v. 70, n. 2, p. 420-436, 2008. Disponível em: Disponível em: https://www.jstor.org/stable/10.1017/s0022381608080407 Acesso em: 28 set. 2019.
» https://www.jstor.org/stable/10.1017/s0022381608080407 - JORGE, Vladymir Lombardo; FARIA, Alessandra Maia Terra de; SILVA, Mayra Goulart da. Posicionamento dos partidos políticos brasileiros na escala esquerda-direita: dilemas metodológicos e revisão da literatura. Revista Brasileira de Ciência Política, n. 33, 2020. Disponível em: Disponível em: https://www.scielo.br/j/rbcpol/a/XNBnwhWwbSsMPFrj4zmHQsG/?lang=pt Acesso em: 20 jan. 2021.
» https://www.scielo.br/j/rbcpol/a/XNBnwhWwbSsMPFrj4zmHQsG/?lang=pt - KATZ, Richard; MAIR, Peter. How parties organize: change and adaptation in party organizations in western democracies. London: Sage, 1994.
- KRAUSE, W. et al. Manifesto Corpus. Version: 2018-2. WZB Berlin Social Science Center, 2018. Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2018b Acesso em: 16 out. 2019.
» https://doi.org/10.25522/manifesto.mpdssa.2018b - KRAUSE, W. et al. The Manifesto Data Collection: South America (WZB), W. B. f. S. Berlin, 2019. Disponível em: Disponível em: https://doi.org/10.25522/manifesto.mpdssa.2019b Acesso em: 26 dez. 2020.
» https://doi.org/10.25522/manifesto.mpdssa.2019b - LANDIS, J. R.; KOCH, G. G. The measurement of observer agreement for categorical data. Biometrics, v. 33, n. 1, p. 159-174, 1977.
- LAVER, M.; BUDGE, I. Party policy and government coalitions. Nova Iorque: St. Martin’s Press, 1992.
- LÓPEZ, Miguel Ángel; MIRANDA, Nicolás; VALENZUELA-GUTIÉRREZ, Pablo. Estimando el espacio politico del cono sur y Brasil: las elecciones presidenciales en el eje izquierda-derecha. PostData, v. 18, n. 2, p. 403-442, 2013. Disponível em: Disponível em: https://www.researchgate.net/publication/260837426_Estimando_el_espacio_politico_del_cono_sur_y_brasil_las_elecciones_presidenciales_en_el_eje_izquierda-derecha Acesso em: 16 jan. 2014.
» https://www.researchgate.net/publication/260837426_Estimando_el_espacio_politico_del_cono_sur_y_brasil_las_elecciones_presidenciales_en_el_eje_izquierda-derecha - LORENZONI, Miguel; PÉREZ, Verónica. Cambios y continuidades de la izquierda en Uruguay: un análisis a partir de las propuestas programáticas del Frente Amplio 1971-2009. Revista Uruguaya de Ciencia Política, v. 22, n. 1, p. 61-81, 2013.
- LÜHRMANN, A et al. Varieties of Party Identity and Organization (V-Party) Dataset V1. PROJECT., V. o. D. V.-D. 2020. Disponível em: Disponível em: https://www.v-dem.net/en/data/data/v-party-dataset/ Acesso em: 08 jan. 2021.
» https://www.v-dem.net/en/data/data/v-party-dataset/ - MADEIRA, Rafael, VIEIRA, Soraia; TAROUCO, Gabriela. Agendas, preferências, competição: PT e PSDB em disputas presidenciais. Caderno CRH, v. 30, n. 80, p. 257-273, 2017.
- MP - The Manifesto Project/MARPOR (Manifesto Research on Political Representation): bancos de dados [Internet]. Berlin. German Science Foundation (DFG). 1979 --. Disponível em: Disponível em: https://manifesto-project.wzb.eu/ Acesso em: 17 out. 2021.
» https://manifesto-project.wzb.eu/ - OLIVEIRA, Augusto Neftali. A ideologia econômica dos presidentes na América Latina: um indicador a partir dos programas de governo eleitorais. Indicadores Econômicos FEE, v. 43, n. 1, p. 83-98, 2015.
- PEMSTEIN, Dan et al. The V-Dem Measurement Model: latent variable analysis for cross-national and cross-temporal expert-coded data. V-Dem working paper series, n. 21, 2020.
- TAROUCO, Gabriela. Brazilian parties according to their manifestos: political identity and programmatic emphases. Brazilian Political Science Review v. 5, n. 1, p. 54-76, 2011. Disponível em: Disponível em: https://www.redalyc.org/pdf/3943/394341997003.pdf Acesso em: 25 fev. 2012.
» https://www.redalyc.org/pdf/3943/394341997003.pdf - TAROUCO, Gabriela; MADEIRA, Rafael. Esquerda e direita no sistema partidário brasileiro: análise de conteúdo de documentos programáticos. Revista Debates, v. 7, n. 2, Porto Alegre, p. 93-114, 2013.
- VOLKENS et al. Scope, range, and extent of manifesto project data usage: a survey of publications in eight high-impact journals. Berlin: WZB Berlin Social Science Center, 2015. Handbook for Data Users and Coders. version I. Disponível em: Disponível em: https://manifesto-project.wzb.eu/datasets/mpduds# Acesso em: 27 out. 2019.
» https://manifesto-project.wzb.eu/datasets/mpduds# - VOLKENS, A. et al. The manifesto data collection (MRG/CMP/MARPOR), M. P. Berlin 2019.
- VOLKENS, A. et al. The manifesto data collection. manifesto project (MRG/CMP/MARPOR). Version 2013b. Berlin, 2013. Disponível em: Disponível em: https://manifesto-project.wzb.eu/ Acesso em: 23 ago. 2015.
» https://manifesto-project.wzb.eu/ - WERNER, A.; LACEWELL, O.; VOLKENS, A. Manifesto coding instructions. (5th revised edition), 2014. Disponível em: Disponível em: https://manifesto-project.wzb.eu/information/documents/handbooks Acesso em 23 ago. 2015.
» https://manifesto-project.wzb.eu/information/documents/handbooks - WIESEHOMEIER, Nina. Policy positions of 47 political parties and 4 presidents in 4 Latin American countries. In: PREPPS: The political representation, executives, and political parties survey: banco de dados [Internet]. 2011. Disponível em: Disponível em: http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA Acesso em: 10 abr. 2020.
» http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA - WIESEHOMEIER, Nina. Policy positions of 165 political parties and 18 presidents in 18 Latin American countries. In: PREPPS: The political representation, executives, and political parties survey: banco de dados [Internet]. 2015. Disponível em: Disponível em: http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA Acesso em: 10 abr. 2020.
» http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA - WIESEHOMEIER, Nina; BENOIT, K. Policy positions of 146 political parties and 18 presidents in 18 Latin American countries. In: PREPPS: The political representation, executives, and political parties survey: banco de dados [Internet]. 2006. Disponível em: Disponível em: http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA Acesso em: 10 abr. 2020.
» http://ninaw.webfactional.com/prepps e em: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/Z5DESA - WIESEHOMEIER, Nina; SINGER, Mathew.; RUTH-LOVELL, Saskia. Political representation, executives, and political parties survey: Data from Expert Surveys in 18 Latin American Countries, 2018-2019. IE University, University of Connecticut and GIGA Institute Hamburg. 2019. Disponível em: Disponível em: https://dataverse.harvard.edu/dataverse/prepps Acesso em: 23 abr. 2020.
» https://dataverse.harvard.edu/dataverse/prepps - ZUCCO, Cesar; POWER, Timothy. Brazilian legislative surveys (Waves 1-8, 1990-2017). Harvard Dataverse, V2, UNF:6:OZ4b3ybYKksC255DkruPxg== [fileUNF] 2019. Disponível em: Disponível em: https://dataverse.harvard.edu/dataverse/bls Acesso em:04 abr. 2020.
» https://dataverse.harvard.edu/dataverse/bls
-
5
Former versions of this paper were presented at the 3rd LAPolMeth Meeting in 2019, at the 43º ANPOCS Annual Meeting in 2019, and in a research seminar at the DCP/UFPE also in 2019. We thank the participants of these events and the anonymous RBCP reviewers for their helpful comments. Any remaining mistakes are our full responsibility.
-
6
The original classification in Coppedge (1997)COPPEDGE, Michael. A classification of Latin American political parties. Kellogg Institute Working Papers, n. 244, 1997. Disponível em: Disponível em: https://kellogg.nd.edu/sites/default/files/old_files/documents/244_0.pdf Acesso em: 20 fev. 2006.
https://kellogg.nd.edu/sites/default/fil... had also a religious dimension with two categories (Christian and secular) that we do not use in our comparison. -
7
Lorenzoni and Pérez (2013)LORENZONI, Miguel; PÉREZ, Verónica. Cambios y continuidades de la izquierda en Uruguay: un análisis a partir de las propuestas programáticas del Frente Amplio 1971-2009. Revista Uruguaya de Ciencia Política, v. 22, n. 1, p. 61-81, 2013. have applied the MP method to measure ideological position of Frente Amplio, in Uruguay from 1971 to 2009. Unfortunately, MP dataset on Uruguay includes only manifestos from the 2014 election.
-
8
PELA - Proyecto Elites Lationoamericanas is a collection of surveys conducted with representatives in legislatures. Available at Universidad de Salamanca (https://oir.org.es/pela/)
-
9
Brazilian Legislative Survey uses ideological position in a scale from 1 (Left) to 10 (Right) in several waves: 1990, 1993, 1997, 2001, 2005, 2009, 2013, 2017. Here we use the average position in the nearest year.
-
10
PREPPS - The Political Representation, Executives, and Political Parties Survey is a collection of expert surveys, carried out in four waves.
-
11
V-PARTY - The Varieties of Party Identity and Organization (LÜHRMANN et al., 2020LÜHRMANN, A et al. Varieties of Party Identity and Organization (V-Party) Dataset V1. PROJECT., V. o. D. V.-D. 2020. Disponível em: Disponível em: https://www.v-dem.net/en/data/data/v-party-dataset/ Acesso em: 08 jan. 2021.
https://www.v-dem.net/en/data/data/v-par... ) is a dataset produced by the V-Dem Institute. Based on its several indicators, we used the variable Economic left-right scale (v2pariglef), which aggregates experts’ ordinal answers in a standardized interval scale varying from -5 to +5 (PEMSTEIN et al., 2020PEMSTEIN, Dan et al. The V-Dem Measurement Model: latent variable analysis for cross-national and cross-temporal expert-coded data. V-Dem working paper series, n. 21, 2020.). -
12
The South America dataset has 98 lines, but two of them (documents from Brazilian PDT and PDS in 1989 election) are not coded.
-
13
The matching between measurements is somewhat arbitrary in some cases in which the position in the alternative scale refers to the party in a legislature elected in a different year than the presidential manifesto. In these cases, we matched the manifesto with the nearest year possible.
-
14
During the segmentation step, we teamed up to consolidate our individual segmentations in a final, unified one.
-
15
The values of the kappa index for the three pairs of MVT coders are 0.88 (TxV), 0.68 (VxM), and 0.66 (MxT). According to Landis and Koch (1977)LANDIS, J. R.; KOCH, G. G. The measurement of observer agreement for categorical data. Biometrics, v. 33, n. 1, p. 159-174, 1977. and Bonnardel (2001)BONNARDEL, Philippe. The Kappa coefficient: The measurement of interrater agreement when the ratings are on categorical scales. The case of two raters. Le Coefficient Kappa. 2001. Disponível em: Disponível em: http://kappa.chez-alice.fr/kappa.txt . Acesso em: 24 nov. 2011.
http://kappa.chez-alice.fr/kappa.txt... , kappa values equal or greater than 0.81 indicate excellent agreement, and values between 0.61 and 0.80 are good. -
16
These subcategories were created in order to accommodate the specificities of new democracies, like Eastern and Central European countries and Mexico. They were then used to analyze Latin American party manifestos.
-
17
This difference may be related to a supposed emphasis that MP instructors put, while training the coders, on avoiding the ‘uncoded’ category, as we were recently warned about.
-
18
Laver and Budge (1992)LAVER, M.; BUDGE, I. Party policy and government coalitions. Nova Iorque: St. Martin’s Press, 1992. built the RILE scale using a series of exploratory factors to identify possible combinations of variables that would reflect either the left or the right position.
-
29
Comprises subcategories 601_1 and 601_2 used by MP coders.
-
1
Para replicação dos dados: https://doi.org/10.7910/DVN/CM3LXP
-
19
In the present paper, we compute the MP original RILE scale (BUDGE; MEYER, 2013BUDGE, Ian; MEYER, Thomas. Understanding and validating the left-right scale (RILE). In: VOLKENS, Andrea; BARA, Judith; BUDGE, Ian et al. (ed.). Mapping policy preferences from texts: statistical solutions for manifesto analysts. Oxford: Oxford University Press, 2013.) using the original set of categories, although elsewhere (TAROUCO; MADEIRA, 2013TAROUCO, Gabriela; MADEIRA, Rafael. Esquerda e direita no sistema partidário brasileiro: análise de conteúdo de documentos programáticos. Revista Debates, v. 7, n. 2, Porto Alegre, p. 93-114, 2013.; MADEIRA; VIEIRA; TAROUCO, 2017MADEIRA, Rafael, VIEIRA, Soraia; TAROUCO, Gabriela. Agendas, preferências, competição: PT e PSDB em disputas presidenciais. Caderno CRH, v. 30, n. 80, p. 257-273, 2017. ) we had proposed an adaptation of the left-right scale.
-
20
Comprises subcategories 201_1 and 201_2 used by MP coders.
-
21
Comprises subcategories 305_1, 305_2, 305_3, 305_4, 305_5 and 305_6 used by MP coders.
-
22
Comprises subcategories 601_1 and 601_2 used by MP coders.
-
23
Comprises subcategories 605_1 and 605_2 used by MP coders.
-
24
Comprises subcategories 606_1 and 606_2 used by MP coders.
-
25
Comprises subcategories 103_1 and 103_2 used by MP coders.
-
26
Comprises subcategories 202_1, 202_2, 202_3 and 202_4 used by MP coders.
-
27
Comprises subcategories 201_1 and 201_2 used by MP coders.
-
28
Comprises subcategories 305_1, 305_2, 305_3, 305_4, 305_5 and 305_6 used by MP coders.
-
30
Comprises subcategories 605_1 and 605_2 used by MP coders.
-
31
Comprises subcategories 606_1 and 606_2 used by MP coders.
-
32
Comprises subcategories 103_1 and 103_2 used by MP coders.
-
33
Comprises subcategories 202_1, 202_2, 202_3 and 202_4 used by MP coders.
Appendix.
Publication Dates
-
Publication in this collection
08 Dec 2021 -
Date of issue
2022
History
-
Received
10 Feb 2021 -
Accepted
28 June 2021