Framing effects on foreign policy: experimental evidence from emerging countries and the Argentine-Brazilian rivalry

Framing effects on foreign policy: experimental evidence from emerging countries and the Argentine-Brazilian rivalry Civil society plays an increasingly important role in the formulation of foreign policy in emerging countries. This article investigates whether public opinion is sensitive to framing effects regarding foreign policy. Data from a survey experiment with a sample of 1,530 students at the Universidad de Buenos Aires and the Universidad Nacional de Avellaneda, we find that participants are sensitive to framing effects on foreign affairs. The interviewees changed their preferences when stimulated by information regarding Brazilian economic growth and military expenditure in comparison with Argentina. In turn, this effect was more pronounced among a) people who tend to stay less informed regarding foreign affairs and b) individuals who are more nationalistic.


Introduction
Public opinion has a specific influence on the formulation of public policy (Howlett, 2000) in democratic regimes where rulers rely on newer forms of public consultation, are accountable to voters, and need legitimacy (Echegaray, 2001). Thus, although foreign policy cannot be considered a "traditional" public policy (Lentner, 2006), the perception that public opinion has regarding this subject has become a matter of concern for academics and policymakers (Faria, 2008).
Historically, it was thought that due to the average citizen's lack of interest in international affairs (Lippmann, 1932;Mueller, 2002), public opinion had little influence over foreign policy formulation (Jacobs and Page, 2005). Nevertheless, over time certain issues of a greater sensitivity, such as the number of deaths in military operations (Aldrich et al., 2006, p. 478), have captured the public's attention and repeatedly risen to importance on the electoral agenda (Holsti, 1992). As a consequence, this has increased the influence of public opinion on foreign policy formulation in developed countries (Foyle, 2004).
With regard to emerging countries, processes such as economic liberalization have meant that civil society is increasingly taken into account in discussions about foreign policy. A clear example is the Brazilian case, in which just after the end of the military dictatorship and the subsequent democratization of the public sphere, foreign policy formulation has become increasingly complex, now including civil society actors such as trade unions, NGOs, business associations, and academics (Milani and Pinheiro, 2013). In this regard, it has been noted that the average Brazilian voter has begun to associate his/her economic welfare with the results achieved by the government in foreign affairs (Lopes and Faria, 2014), which has led to the creation of an agenda in public opinion regarding foreign policy (Lopes, 2011). Our article aims at contributing to the public opinion literature surrounding perceptions of foreign policy issues in emerging countries. More particularly, this study investigates whether these perceptions are sensitive to framing effects. The main objective of our work is to understand the sensitivity of Argentines to framing about Brazil's growing power. To this end, we applied an experimental survey in Argentina to a convenience sample composed of 1,530 students belonging to the Universidad de Buenos Aires (59% of the sample) and the Universidad Nacional de Avellaneda (41%). We subsequently looked at whether new information about Brazil's economic growth and military spending changed respondents' opinions on the best strategy Argentina might adopt in relation to Brazil: align itself with its neighbor or seek new regional partners to counterbalance Brazil's power.
Our findings show that when respondents receive information about Brazil with no point of comparison with Argentina (see Appendix I), the impact on their opinions is null. However, when Brazil's level of growth and military spending are contrasted with Argentina's (see Appendix II), the interviewees prove to be sensitive to framing effects, with an inclination toward thinking that Argentina should seek new partners to counterbalance Brazilian power. In turn, this effect is even greater among people who reported that they did not stay regularly informed about international affairs and among those who believed that Argentina has a greater role in the international arena. This finding shows that survey experiments can generate effects that are only observable among particular subgroups of our samples (Barabas and Jerit, 2010, p. 226).
Our article is structured as follows. First, we contextualize our research problem, presenting a historical evolution of the relationship between Argentina and Brazil and pouring through the public opinion research aimed at capturing Argentines perceptions of their neighbor. Then we delve into the theoretical framework, addressing the debate about framing effects and drawing on a few survey experiments on foreign policy issues conducted in the United States. Next, we explain our methodology and present our survey results without considering the different framings. We then go on to discuss the results of our experimental research and, finally, our conclusions, providing suggestions for a future research agenda.

From rivalry to regional hegemony
Brazil is the most important country in Argentina's contemporary foreign policy (Russell and Tokatlian, 2011). However, throughout their history, the two countries-the largest in South America-have gone through phases of rivalries and alliances, only stabilizing their relationship in 1985 with the rapprochement between presidents Alfonsin and Sarney in Foz do Iguaçu. Since then, Brazil has been seen as Argentina's closest ally and vice-versa (Selcher, 1985;Hirst, 1987;Bandeira, 1987).
In economic terms, bilateral trade grew exponentially between the two with the creation of Mercosur in 1991. As Figure 1 shows, Argentina enjoyed a trade boom during the 90s, with the trend reversing after the deep crisis of 2001, when Brazil started to set the pace for the countries' commercial relationship. Today, even though 20% of Argentine exports go to the Brazilian market, Argentina absorbs only 8% of its neighbor's exports, which enjoy both a higher aggregate value and a higher level of industrialization (Bernal-Meza, 2008). On the other hand, analyzing the historical evolution of both countries using the National Material Capabilities Index 2 , we can see that the power gap between the two has grown steadily since 1956 when the Kubitschek administration came into power (1956)(1957)(1958)(1959)(1960)(1961) (Sikkink and Wolfson, 1993) (see Figure 2). Taking as a reference the theoretical framework of the Realist school, when it comes to foreign policy, regional middle powers like Argentina have two choices regarding regional hegemonic power: stand with it or counterbalance its power. While the first strategy consists in following the regional hegemon's policies, counterbalancing can be approached in two ways: (a) externally, by combining material capabilities with countries in the same situation; or (b) internally, by mobilizing its own resources to more effectively resist the hegemonic country.
Historically, Argentina tried to counterbalance Brazilian power following strategy (b). Nevertheless, by the mid 1980s, when Brazil achieved a clear military superiority, Argentina once again was forced to choose between two foreign policy strategies regarding its neighbor: align itself with Brazil to achieve greater relevance in the international arena, or align itself with other regional countries in order to counterbalance Brazil's relevance decreased to 29% in 2015, but it has still remained the most chosen response.

Figure 3 Perception about Brazilian relevance (%)
Source: Elaborated by the authors using data from the Argentine Council for International Relations (2015). 3 Along with Latinobarometer research (<http://www.latinobarometro.org/>) and the work focused on elites carried out by the Nupri-USP (Albuquerque, 2000(Albuquerque, , 2001, this is the only initiative that consistently conducts traditional surveys in order to capture Argentine perceptions regarding international affairs. 4 2,408 people and 192 opinion leaders interviewed between November 15th and December 15 th , 2001. 5 1,616 people and 175 opinion leaders interviewed between October 8 th and October 23 rd , 2006. 6 1.606 people and 170 opinion leaders interviewed between November 17th and November 28 th , 2010. 7 1.600 people and 100 opinion leaders interviewed between July and August of 2015.
At this point, it is worth asking ourselves how sensitive Argentines' perceptions on foreign policy are to framing effects. Drawing on the literature on framing effects, in the next section we present our own theoretical framework together with some experiments applied in the United States that focus their attention on specific foreign policy issues.

Theoretical framework: framing effects and foreign policy
Research on framing effects can be found throughout the Social Sciences, as well as in various subfields of the Political Science field (Druckman, 2001, p. 226). In fact, the high interest in the topic has often been accompanied by a lack of clarity in the definition of what exactly framing effects are and how they influence public opinion (Nelson, Oxley and Clawson, 1997). To account for this conceptual vagueness, this article defines framing effects as "small changes in the presentation of an event or topic, but which have large effects on the perception that the public has regarding these issues" (Chong and Druckman, 2007, p. 104).
Since its inception, the specialized literature has shown that the way in which a question is worded has the ability to influence respondents' answers (Rugg andCantril, 1942, 1944;Payne, 1951;Schuman andPresser, 1977, 1981). However, with the passing of time and advances in the area, framing effects have come to be classified into two main groups: (a) those stimuli that, while presenting the same data (but in a different way), alter interviewees' preferences; (b) those stimuli that focus their attention on certain topics and cause interviewees to focus on particular issues at a moment at which they are forming opinions 8 (Druckman, 2001, p. 228-230). Our work is in line with the first type of research, which tries to analyze how the same piece of information, when presented differently, can affect an interviewee's opinions.
Among the classic works that use this approach, the foundational article by Tversky and Kahneman (1981) is certainly among the most noteworthy. Through an experiment involving 307 students from the University of British Columbia, the authors show that when deaths resulting from a given health program were presented relative to the number of patients, respondents were more likely to support the initiative than when they were stimulated simply by the information in absolute terms. In contrast, when the given information focused on the lives that could be saved, the framing effect reversed, and the students showed themselves to be more supportive in relation to the program than when they were stimulated by the information in absolute terms. Drawing on this 8 A classic example of this type of research-one that also takes into account the question order effect-is the work carried out by Hyman and Sheatsley (1950). Through an experiment applied in 1948, the authors show that when first exposed to the possibility that the USSR allowed American journalists to enter the Soviet Union, the interviewees showed themselves to be more receptive to the belief that the United States should do the same with their communist counterparts. On the other hand, those who first were requested to express their opinion about the possibility of Soviet journalists working in the US presented lower acceptance rates when compared to the other group. research, a great number of studies since have tried to analyze how framing effects can affect public opinion perceptions on foreign policy issues 9 .
Undoubtedly the topic that has been discussed most among foreign policy issues, mainly in the American literature, has been war and the impact of casualties on public opinion perceptions (Gartner, 2008). Boettcher and Cobb found that when combat casualties are presented relative to enemy casualties, the negative effect of the latter information is activated and support for the war increases (Boettcher and Cobb, 2006, p. 849). However, other recent research into framing effects related to this topic has presented different results. Americans are concerned not only with the proportion of casualties, but also with their distribution across society (Kriner and Shen, 2013). In turn, these casualties are tolerated more if a future victory is perceived as likely (Gelpi, Feaver and Reifler, 2009). Thus, we can affirm that framing effects are not homogeneous and may be influenced by individual, as well as situational, factors (Boettcher and Cobb, 2009, p. 692).
On the other hand, other authors have focused their attention on issues such as international trade and the propensity of citizens to support economic integration processes. Hiscox (2006) observed for the American case that those people stimulated by framing that included potential loss of jobs were less likely to support trade with other countries than those who did not receive the job loss framing. The author found that less educated people are more sensitive to framing effects, demonstrating again that these effects are not homogeneous and that it is always important to take into consideration heterogeneous effects across different subgroups of the population (Hiscox, 2006). In a related work, after replicating Hiscox's experimental survey in Argentina, Ardanaz, Murillo and Pinto (2013) found that individuals' material conditions factor into explaining both their perceptions on trade policy and, therefore, their sensitivity to framing effects.
Given what we have analyzed so far, we can conclude that: (a) the question regarding the degree of sensitivity of public opinion in relation to framing effects on foreign policy issues is still a matter of discussion; (b) there are studies that have addressed specific foreign policy issues, but few have investigated matters of bilateral relations; (c) almost all of the existing literature has focused its attention on public opinion in developed countries, mainly especially the United States.
The present work furthers this research agenda by providing evidence from an emerging country, Argentina, on how public opinion there reacts to information regarding the growth and military spending of the country's main ally, Brazil. Ultimately our goal is to see if these stimuli affect respondents' perceptions. If so, this would reveal public opinion's sensitivity to framing effects related to foreign policy.

Methodological design
The use of experimental methods has recently expanded into the fields of Political Science and International Relations (Hyde, 2010, p. 73). In this regard, although there is a significant history of experimental work in this area (Morton and Williams, 2010, p. 9), the discipline, as a whole, has seen the resurgence of this methodology in recent years, becoming a widely accepted approach (Druckman et al., 2006, p. 634). Nevertheless, as pointed out by Turgeon and Rennó, this type of methodology is still underused in Brazil and Latin America (Turgeon and Rennó, 2010, p. 147).
In summary, the great advantage of experimental research is that it allows us to manipulate our independent variable, while controlling for the remaining factors that could influence our dependent variable (Neuman and Robson, 2004, p. 205;Babbie, 2013, p. 238). Thus, in contrast with observational research such as cross-sectional and panel surveys (Gaines, Kuklinski and Quirck, 2007, p. 1-2), experimental studies allow us to isolate the effect of the variable we want to analyze and, therefore, are more suitable for answering some specific research questions, particularly those involving causal mechanisms (Morton and Williams, 2010, p. 12).
Our experimental data comes from a survey applied at the Universidad de Buenos Aires (UBA) and the Universidad Nacional de Avellaneda (UNdAv) between April 7 th and May 30 th , 2014 10 to a convenience sample composed of 1,530 students in the Human Sciences 11 . We chose this type of sample because, despite having some limitations, it has been shown to behave in the same way as general population samples (Druckman and Kam, 2009). Moreover, given the ease of recruiting students, using this type of sample offered an efficient way to test our initial hypotheses before generalizing our theory to the general population (Dasgupta and Hunsinger, 2008, p. 94).
With 308,000 students 12 , UBA is the oldest and most prestigious university in Argentina 13 . Before entering the university, students must pass a common course that lasts at least one year and has a 40% dropout rate. In turn, professors and the system as a whole are very demanding compared to other universities, especially for those students who have financial limitations and who must work. Thus, only 23% of UBA students graduate.
On the other hand, located in Avellaneda, on the outskirts of the city of Buenos Aires, UNdAv was founded in 2011 and saw its first graduates in 2015. With 9,311 10 A pre-test was applied in early December of 2013 to ensure that the questions were clear enough for all participants. 100 students answered the questionnaire, and the question that tried to capture political affinity in relation to the government had to be modified, since in its original format it generated a rejection superior to 10%. 11 Of those 1,530 respondents, 13 were excluded from the final sample because they chose not to submit the questionnaire. 12 Census applied in 2011 <http://www.uba.ar/institucional/censos/Estudiantes2011/estudiantes%202011.pdf>. 13 If we were to only consider the indicators "academic reputation" and "reputation among employers" from the QS ranking, UBA would be the best-positioned university in Latin America. students-most of them first-generation college attendees 14 -this university seeks to fulfill a social commitment, serving as a tool for the reintegration of young people who were excluded from the university system, preparing them to enter the labor market. In this sense, the courses offered are quite diverse, responding to a social environment different from that of traditional universities such as the UBA (see Table 1).
With the aim of obtaining a more accurate representation of Argentine society, we conceived of our sample including students from both universities, rather than simply students from either an elite institution (UBA) or a public university with a specific profile such as UNdAv. To conduct the survey, we came into contact with professors in the Human Sciences at both universities with the help of the academic staff. With their approval, we scheduled times at which we could interrupt their lessons and carry out the survey. We avoided giving participants prior knowledge about the research in order to prevent them from actively seeking information about international affairs before taking the survey.
Students were simply invited to participate in a public opinion poll. To incentivize participation, students were offered two vouchers (worth AR $1,000) for books. As suggested by McDermott (2014), in order to maintain anonymity, participants' personal information was collected on a separate sheet of paper and kept apart from the answers.
After answering five questions about foreign policy, Argentina, and its relevance in the international arena, participants were randomly assigned to one of the two experimental treatments or to the control group 15 . Both treatments provided real information about Brazilian GDP growth over the last ten years, the sum of money spent by Brasilia on military expenditures, and the number of troops the country has. The only difference was that one of the texts presented the information in an absolute way (see Appendix I), while the other did so relative to Argentina (see Appendix II). As suggested by Boettcher and Cobb (2009, p. 684), a third of our sample did not receive any stimulus, thus serving as the control group necessary to estimate the exact impact of each type of framing. The following procedure was used to assign participants to the three different groups: the absolute treatment was coded as 1, the relative as 2, and the control group as 0. Then we randomized the distribution of these 3 numbers among 900 (UBA) and 650 (UNdAv) cases and mixed the questionnaires according to this random distribution.
Immediately after these stimuli, respondents were questioned about which strategy would be more suitable for Argentina: (a) align with Brazil or (b) seek new regional partners to counterbalance Brazilian power. Also, to more broadly capture Argentinian public opinion perceptions regarding international issues, participants had to tell us which they felt were the three most relevant countries in the international arena, the three countries they believed would be the most important in the next ten years. We also asked respondents to tell us if they believed Brazil was a leader in South America and if they think Argentina should imitate its behavior. General results: align with a future global power and a regional leader Next, we will present the results of the survey without considering the treatment effects. First, as we can see in Table 2, more than 77% of those interviewed believe that Argentina should align with Brazil and not seek regional partners in order to counterbalance its power.  16 Respondents placed themselves on a scale of 1 to 10, where 1 was extreme left and 10 extreme right. 17 Respondents placed themselves on a scale of 1 to 10, where 1 meant that they totally disagreed with the government's policies and 10 meant that they totally agreed. 18 Respondents placed Argentina on a scale of 1 to 10 in terms of its relevance in the international arena, with 1 being of little relevance and 10 being very relevant. 19 Respondents had to answer how often they informed themselves about international affairs, having the options: daily, 2 to 3 times a week, once a week or not usually.
Second, we can see that Argentinians consider there to be only two major global powers today. When consulted on which are the three most relevant countries in the international arena 20 (see Table 3), the United States and China stand out well above the others, with 92% and 73% of the responses respectively. It is noteworthy that Russia was the third most selected country with 33%; we believe this result can be explained by the proximity of the implementation of the survey to events happening in Ukraine and to the annexation of Crimea to the Russian Federation 21 . Finally, only 24% of respondents mentioned Brazil as a current global power, taking fifth place, behind Japan. When asked which would be the three most powerful countries in the next ten years 22 , the answers appear to be more evenly distributed (see Table 4). China retains almost the same percentage (68%) but now is the most chosen country, slightly above  Moving forward with questions that tried to capture Argentine perceptions on international affairs, we note again that there is a strong consensus about Brazil being the most important South American country. Generally, almost 65% of respondents believed that Brazil is the regional leader (Table 5), although this perception is inversely proportional to the perception of Argentina's relevance in the international arena (see Figure 4). To calculate this last probability, we took the marginal probabilities of a logistic regression and plotted them to facilitate their interpretation (see Figure 4). Thus, we see that the probability of Brazil being considered a regional leader falls from 80% to 40% as we move from respondents who believe that Argentina is irrelevant to the international arena (value 0), to those who think it is most relevant (value 10).

Figure 4 Plotted probability of Brazil being seen as a regional leader
Source: Elaborated by the authors using Clarify (Tomz et al., 2003) Finally, 44% of the respondents appeared to believe that Argentina should hold Brazil up as a model, although for this question the variation of responses among different subgroups was considerable. Those participants who had posited themselves on the left of the ideological spectrum tended to believe to a lesser extent that the Argentinian government should follow Brazil's example (see Figure 5). On the other hand, compared to those who declared support for Kirchner's policies, those with a negative view of the Kirchner administration tended to believe more strongly that Argentina should imitate its neighbor. Like Figure 4, Figure 5 is a visual representation of the marginal probabilities calculated using logistic regression. This is probably one of the most important findings of this section, as it confirms Russel and Tokatlian's hypothesis, which suggest that the Brazilian image has been historically used in Argentinian domestic debates as an "inverted mirror," featuring Brazil as a rising power and Argentina as a country in decline (Russell and Tokatlian, 2011, p. 258). This does not necessarily mean that people who used to support Kirchner's administrations had a negative view of Brazil, but it does suggest that those who were opponents of Kirchner's government used Brazil's success in order to criticize the government in power.

Framing effects
Analyzing the results of the survey with the differences between each treatment group and the control group in consideration, we can see that framing effects indeed changed participants' perceptions, albeit in a heterogeneous way.
Argentines who received absolute information regarding Brazilian growth, its military spending, and its number of soldiers proved to have perceptions similar to those who received no stimulus (control group). Meanwhile, those who received information comparing Argentina to its neighbor tended toward the idea that Argentina should seek OPINIÃO PÚBLICA, Campinas, vol. 22, nº 1,abril, 2016 209 new regional partners to counterbalance Brazilian power (26% vs. 18%) (see Table 6).

FRAMING EFFECTS ON FOREIGN POLICY
These findings support the theory that in order to fully grasp what large numbers mean, it is necessary to have a reference point closer to our reality (Tversky and Kahneman, 1981). Table 6 Difference between treatments and control groups As a robustness measure, we compared the treatments, including controls, with individual characteristics through a logistic regression. As we can see in Figure 6, the previous findings hold even after including controls. These variables are: the ideology of the person (a categorical variable ranging from 1 to 10, with10 representing the extreme right), how often the person informs themselves about foreign affairs (ordinal variable, ranging from 0 to 3), a variable about the person's opinion on Argentina's role in the world (categorical variable ranging from 1 to 10, 10 being "very important"), opinion about the current government (categorical variable ranging from 1 to 10, 10 being "very favorable"), family income (ordinal variable), university degree (fixed effects for careers), the person being nationalist or not (dummy variable based on self-declared perception), gender (dummy, 1 being "man"), and a dummy for the university (UBA or Undav).  A second empirical strategy consisted of analyzing whether the treatment effect was more or less significant for certain subgroups of our sample. What we observe is that among people who usually get little information about international affairs, the relative treatment was 64% more effective than the one we had previously observed. That is, 46% of uninformed people chose to counterbalance Brazilian power after receiving the comparative treatment. In turn, among nationalists this effect was subtly higher than the one for the general population-just 4.6% higher than the 26% associated with the general sample (see Table 7).

Argentina should counterbalance Brazil
Thus, this finding reinforces the idea that while some framings can have no effect on public opinion perceptions, others may have an effect that is conditional on or amplified by respondents' characteristics (Knoll, Redlawsk andSanborn, 2011). Donovan andJalleh (1999) suggest that the degree of involvement of individuals with regard to the issue addressed may alter their sensitivity toward framing. To summarize the authors' argument, while negative framing would be more effective than positive framing among people with a high degree of involvement, the relationship reverses when respondents' interest is low, making positive framings more effective.

Conclusions
Is public opinion sensitive to framing effects on foreign policy issues? It depends.
Drawing on novel experimental data, we arrive at some findings that contribute to an ongoing discussion in the literature and present empirical evidence from emerging countries that until now have received little attention.
As other authors have pointed out (Flachaire and Hollard, 2008;Barabas and Jerit, 2010, p. 226), survey experiments generate effects that are observable only among particular subgroups of our samples and, due to this, cannot be extrapolated to the entire population.
Our findings show that the frequency with which people stay informed about international issues is a determining factor in explaining how sensitive they are to framing effects on foreign affairs. In this sense, the more a person declares that they stay informed, the less sensitive he/she will be to framing effects on foreign policy issues. With this in mind, we might ask ourselves from which source individuals are receiving information about international affairs. The literature suggests that big media plays a gatekeeping role regarding foreign policy issues (Hill, 2003, p. 275) since it is the main channel through which most of the population receives information on themes (Puglisi and Snyder, 2011) with which we do not have personal, daily contact (Soroka, 2003, p. 28 On the other hand, we also find that the more relevance a person gives to his country in the international arena, the more susceptible he/she is to information that shows its weakness compared to other countries. This finding is interesting if we think from Putnam's logic of two-level games (1988). Politicians might try to influence the electorate and gain an advantage in the domestic debate using a nationalist discourse.
Building on this idea, we also corroborate Russel and Tokatlian's hypothesis, which suggests that the Argentinian opposition had used the Brazilian image as an "inverted mirror," presenting Brazil as an emerging power and Argentina as a country in decline in order to criticize the previous government (Russel and Tokatlian, 2011, p. 258

Abstract
Framing effects on foreign policy: experimental evidence from emerging countries and the Argentine-Brazilian rivalry Civil society plays an increasingly important role in the formulation of foreign policy in emerging countries. This article investigates whether public opinion is sensitive to framing effects regarding foreign policy. Data from a survey experiment with a sample of 1,530 students at the Universidad de Buenos Aires and the Universidad Nacional de Avellaneda, we find that participants are sensitive to framing effects on foreign affairs. The interviewees changed their preferences when stimulated by information regarding Brazilian economic growth and military expenditure in comparison with Argentina. In turn, this effect was more pronounced among a) people who tend to stay less informed regarding foreign affairs and b) individuals who are more nationalistic.
Keywords: framing effects; survey experiment; foreign policy; Argentina; Brazil