SciELO - Scientific Electronic Library Online

vol.7 issue1Human security paradigms and economic crisis in first year of the North Caucasus Federal DistrictTwo to tango: an analysis Brazilian-Argentine relations author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand




Related links


Brazilian Political Science Review

On-line version ISSN 1981-3821

Bras. Political Sci. Rev. vol.7 no.1 São Paulo  2013 



Public policy and media frames: the debate over migration in Brazil



Augusto Veloso Leão

Institute of International Relations, University of São Paulo (USP), Brazil




The main goal of this article is to provide a good basis to assess the way media frames are embedded in a wider social scenario, and how public and political preferences can be researched through media debate. The methodology adopted allows for an analysis of newspaper articles that can show broader trends of the debate and serve as a thermometer to measure public debate. It can further highlight details and enable in-depth analyses of media discourse.
The article explores the interconnectedness of media debate and policy-making process by means of the analysis of articles published in a Brazilian newspaper between 2009 and 2010. Quantitative and qualitative methodologies are employed to identify the frames used in media and to offer insights of their relationship with the political debate. The ongoing political debate has raised attention to the issue of migration, with a great numbers of actors expressing very diverse points of view. A broader public debate has been initiated and some portions of it find voice in different means of communication. The paper argues that changes in the public and in the media debate are a response to changes in the political debate, while at the same time the first two also help to outline the latter.

Keywords: migration; public policy; media debate



Migration public policies have received increasing attention in public debate in the last few years, especially in countries with a large migrant worker population. We can observe that the debate over migration in the last decade showed a greater focus on national security issues, constituting the main reasoning behind the proposition of more rigid entry and permanence laws for migrants (Weiner, 1993; Rudolph, 2003). Weiner (1993) and Rudolph (2003) argue that migrants can be perceived as threats to the geopolitical security, to the production and accumulation of wealth, and to social stability and cohesion, especially by bringing or creating political conflicts or through the social and economic burden they can represent. The beginning of the twenty-first century also saw terrorist attacks with the participation of migrants, further increasing the perceived threats that migration poses to national security.

Examples of public policies that aim at enhancing national responses to the mass entrance of migrants and, particularly, at countering irregular migration, include the Normative Resolution 97/2012 act from the Brazilian National Immigration Council (CNIg), which restricts the number of humanitarian visas conceded to Haiti nationals to 1200 per year, the European Union Directive 2008/115/CE, initiated by the European Commission and establishing the "common standards and procedures for returning illegally staying third-country nationals" (also known as Return Directive), or the Senate Bill 1070, of the Arizona state in the United States of America, intending "to discourage and deter the unlawful entry and presence of aliens and economic activity by persons unlawfully present" in the country (its most polemical provisions are currently blocked by a preliminary injunction issued by a federal judge). All of these policies have been highly criticised internally, by other countries, and by international institutions because of their discriminatory provisions and because they create the possibility of migrant human rights abuse. The fact that these policies were approved is a response to parts of the domestic population who feel themselves threatened by the growing migrant population, either economically or because of cultural differences.

In Brazil, the current law defining migrant policy was issued in 1980 (Law 6815/1980) and reflects an exaggerated preoccupation with national security – Ventura and Illes (2010) credit this preoccupation to how the military government at the time perceived the Cold War. The national security concerns are already evident in its first articles, which restrict the entry and permanence of migrants for safeguarding national interests, namely, national security and the protection of the national work force. At the moment, the Brazilian Chamber of Deputies, the lower federal chamber, is discussing a proposed law from 2009 (PL 5655/2009), by the Ministry of Justice which is designed as a substitute to the 1980 law. The proposal presents several improvements towards a better protection of migrants' human rights, and a public audience to discuss the project that was approved in late 2012 should occur in the beginning of 2013. The proposed law reinforces the idea that all migrants are warranted their rights and constitutional guarantees, as well as access to education, health, and to the labour benefits system, irrespective of their migrant status. Yet the proposed law also gives an even greater focus to the duties of migrant workers, creating several mechanisms to stop their entry and restraining certain rights (some aspects of political participation, for example) and grants the Executive branch enlarged powers to act in the control of migrant workers. According to Ventura and Illes (2010: 2), the proposed law "introduces a large number of rights on general and abstract terms" and "an even larger number of specific regulatory apparatus which restrict the meaning or extent of those rights". The authors further state that the programmatic mention of migrant workers' rights without some level of definition can be used to render those rights ineffective in daily life, particularly because the proposed law does not specify the roles of federal, state, and municipal governments in guaranteeing those rights.

During the discussions on the proposed law, the Brazilian National Immigration Council (CNIg), a department of the Ministry of Labour in charge of policymaking and immigration coordination1, has presented the government's "National Policy of Immigration and the Protection to the Migrant Workers" in May 2010. The policy is marked by a strong preoccupation with the migrants' human rights and their integration into the Brazilian market and society, responding directly to some concerns of the migrant population in Brazil and establishing a row of concrete actions to enforce the protection of this section of the population's rights. One such action is the recommendation to sign and ratify the "United Nations Convention on the Protection of the Rights of All Migrant Workers and Members of Their Families" in order to reaffirm Brazil's commitment to the protection of migrant workers internationally. The National Policy also introduces a technical qualification program for public officials working in the registration of migrants to promote a better orientation of migrant workers' rights and a more humane relationship, one of the most common complaints from migrants who ought to be registered in the Federal Police Office from the Ministry of Justice. Furthermore, the document recommends the development of research and enquiries on the situation of migrants and of the migrants' networks, stated as a necessary condition for improving and proposing adequate public policies addressing the issue.

This research uses newspaper articles on migration public policies and on migrants in order to help identify some of the major frames – ideas that help us give meanings to a theme – on the issue of migration in the media debate. We chose to analyse Folha de S. Paulo because it is Brazil's most widely read national newspaper and because it represents an important source of opinion diffusion and leadership, informing much of the political, public, and media debate in Brazil. The analysis of one newspaper enables the creation of a large panorama of the most important frames in the different debates examined, even though, corresponding to the rationale we present below, the inclusion of more newspapers could increase the diversity of points of views captured. The objective of this analysis is to verify if the references to migrants in the media debate can be linked to the development of a public policy that either proposes better control and policing towards migrants or the protection of migrants' human rights, advocating for a better social and economic integration of this group. A total of 126 newspaper articles from 2009 and 2010 were examined in this study.


Two Different Perceptions

Migration is a complex phenomenon that has its roots in different economic, political, social, and environmental issues (Castles, 1986; 2006; Freeman, 1995; Hugo, 1996; Mattar, 2012). Its complexity in forms and motivations causes migration to be perceived differently by each country and to be seen, at the same time and by both the sending and the host countries, as an opportunity for growth and a problem.

Remittances sent by migrants to developing countries have transferred more than 300 billion dollars in 2010 (World Bank, 2012) and, consequently, are an important part of the family income in the sending countries. Furthermore, migrant workers have an important economic role in supplementing the workforce and causing better economic allocative efficiency and an increase in productivity, especially in countries or regions with scarce labour or an ageing population (Freeman, 1995). The significant cultural gains from the promotion of an exchange between host and migrant cultures are also positive.

However, migration also has negative consequences to both sides. To the sending country, the "brain drain" effect (in which a significant part of the specialised workforce moves to a developed country), combined with a lack of motivation for seeking specialisation caused by low wages, obstructs economic development, and increases dependence on external capital (including capital sent by emigrants to their families)2, thus encouraging further emigration (Rosenzweig, 2005). Host countries have to deal with irregular immigration, threat of unemployment (especially between poorer working classes), lowering of wages, competition over social benefits and resources, and concerns regarding the inclusion of culturally different groups. Moreover, changes in values and national identity are frequently cited as negative consequences arising from the presence of culturally different populations.

The perceived difficulties therefore affect both the local and the migrant populations – the latter being, in many cases, excluded socially, economically, and politically from the community they live in. Koopmans and Statham (1999) describe two main challenges to the contemporary nation state: the post-national challenge, in which universal values prevail over the internally accepted and determined ones; and the multicultural challenge, in which the internal plurality of a community weakens the state's capacity to maintain social order. The importance given to universal values and a community's internal plurality affect how the legitimacy of a government's actions is perceived. In most cases, these actions are perceived as failing or illegitimate once the main cultural values change and problems with the cultural integration of migrants occur.

The loss of legitimacy in the contemporary nation state is also discussed by Hurrell (2007). Legitimacy, for that author, is expressed in five different dimensions: shared understanding of justice, specialised and specialist knowledge, reason-giving in light of shared principles inside the political community, the process and procedure of its operations ("input legitimacy"), and the effectiveness of those actions ("output legitimacy") (Hurrell, 2007: 80-91). The importance of legitimate actions from a government increases in so far as the distance between citizen and political actors becomes greater. Legitimating its actions is a strategic attitude in the national political game, and also internationally because it implies the acceptance of the governors' authority and the citizens' disposition in following the established rules (Hurrell, 2007: 78). It is a grave issue when both local and migrant populations perceive the government's actions as illegitimate. To the native population, this happens when state actions cannot solve the problems brought by immigration (its effectiveness) or the principles of social order are challenged by the migrant culture. To the migrant population, state actions can be perceived as illegitimate when they are not based on the same principles of justice or when their political community is based in different reason-giving conducts, added by the fact that the migrants are rarely allowed participation in the processes and procedures for decision-making (input legitimacy). In Brazil, illegitimacy is often perceived because government actions lack effectiveness in registering or controlling migrants and in giving them access to their rights. A common argument is that because Brazilian boundaries are extremely large and porous, the government is unable to account for irregular entrances. For migrants, lack of effectiveness is perceived when official information is contradictory, registration processes are demanding and slow (a migrant identity card takes approximately one year to be manufactured), and the government fails to guarantee access to justice, health or education. Because migrants are denied access to most political rights in Brazil, they also experience a lack of input legitimacy.

The controversy about legitimacy loss leads to various reactions from governments and societies regarding the integration of the growing migrant community, two of which we shall analyse here. We focus our discussion on two frames for the meanings attributed to migration and migrants: one that encompasses the preoccupations with national security and one extending over the preoccupations regarding migrants' human rights.

The increase of migrant populations initiates discussions inside a political community that can be seen in national security terms, which relates to the first frame we analyse. The national security frame focuses on the burden migrant populations impose on internal social and cultural cohesion. The presence of immigrants is seen as the focal point of social and economic problems in the country, with the cultural differences between groups leading to a perception of the decay of national identity values. Usually, this frame highlights problems such as irregular migration and competition for social benefits and stresses the idea of a loss of governmental control over the migrant population.

To the second frame, the source of social and economic hardship is not ascribed to the presence of migrant populations, but instead is blamed on society's and the government's failure in properly integrating those migrants in the social, economic, and political domains, mostly by disrespecting the principles of universal human rights. This lack of integration and the disrespect for migrants' human rights is the main force driving the social conflicts between native and migrant populations. This frame emphasises that the migrant population does not share the same political principles and reason-giving conducts as the native population, consequentially causing a perceived lack of legitimacy in the government's actions. It could also suggest the integration of the migrant population as a solution for the social and political conflicts brought to light by the increase in the migrant population.

It is important to recognise that the focus given to each frame is related to a given community's concept of citizenship and its organising principles. Citizenship is a central concept to the study of migration because it defines who is entitled to have their rights guaranteed in a nation state. In most cases, migrants are entitled to economic, social, and cultural rights but are barred from the majority of political rights, and because citizenship cannot be understood in the absence of political rights, migrants cannot be considered citizens until they naturalise in their host country (Reis, 2007). We can relate two broad concepts of citizenship within the frames presented. If citizenship is related to belonging to a specific social group and affiliating oneself with a muster of national identity, it gives space for a stark differentiation between "us" and "them" and places incentives to adopt the national security frame. On the other hand, if citizenship is related to the possession of the legal rights of individuals in the nation state, there is more space for the adoption of the migrants' human rights frame.

The first concept of citizenship – communitarian citizenship – states that a mutual responsibility agreement and a shared common goods concept are possible through a social process of constructing a political community around common history, culture, language, and territory (Bellamy and Warleigh-Lack, 1998: 459-460). Through this process, individuals and political communities are able to establish who is entitled to rights, how the rights are to be provided, and also consolidates individual commitment to a political organisation – the nation-state – that guarantees the production and distribution of public goods. The second concept – cosmopolitan citizenship – reinforces the moral obligation of individuals towards all other human beings and is based on the existence of bonds that create a human community, taking precedence over one's nationality. This Kantian ideal of universal citizenship is based on solidarity ties between all humans rather than on the existence of a world government. Linklater (1998: 26) reminds us that the concept of citizenship can have its moral dimension expanded, thereby facilitating the defence of universal human rights, even in the absence of transformations in the political boundaries of the nation-states or the constitution of a global government.

The principles of how a community should organise itself are also important in order to give prominence to one or the other frame in the migration debate. Demant (2009) presents a study that can be useful in thinking about the preferences of different societies when integrating migrants. The debate about how many different groups can live together inside a political community may admit to a classification along its two main characteristics: the possibility of the existence of different cultures inside the same space and the tendency of those groups to be in contact each other (Demant, 2009: 2). A certain community might give preference to fewer and culturally similar groups to live together, making up a more cohesive society ("cultural monism") or to the coexistence of various groups, thereby forging a diverse society ("cultural pluralism"). On the other hand, communities might understand that cultural identity is relatively immutable ("cultural essentialism") or that different cultures can combine different elements between them ("cultural evolutionism"). From the crossing of those two dimensions, four main routes appear to deal with migrant population: societies which are "monist-essentialist" prefer little or no diversity and do not believe in changes in cultural values, rejecting migrants and giving preference to barring their entry, while "monist-evolutionists", although also favouring the coexistence of culturally similar groups, understand that migrants can be assimilated into the national culture. "Pluralist-essentialists" can live with a diversity of cultural groups, but do not believe in big cultural changes, preferring multicultural policies and creating structures to guarantee autonomy and differentiation between groups3, while "pluralist-evolutionists" also admit the existence of different cultural groups but believe that cultural values can admit changes, leading to an integrationist perspective with the creation of a new culture with elements from both the native and the migrant cultures. Arguably, it is not the mere presence of migrants that guide the composition of preferences, as migrants from culturally similar groups might experience unhindered acceptance even in monist-essentialist societies. In all kinds of societies there may exist a preferred type of migrant for whom integration can present no great difficulties – such as highly skilled workers, investors, or people from a determined cultural group.

In general, "monist-essentialist" communities would be more prone to adopt the national security frames, while "pluralist-evolutionists" would be more prone to adopt the migrants' human rights frames. The other two groups have a relatively more ambiguous relationship towards the two frames, but could be more prone to adopt the national security frames as the migrant community grows significantly once the hegemonic position of the native culture (or the egalitarian relationship between them in the case of pure multiculturalism) is challenged by the migrant culture.

Brazil is a nation formed by many different migration flows (Portuguese and West Africans in the sixteenth to the eighteenth centuries, Europeans, Middle-Easterners, and Japanese in the nineteenth and twentieth century, and more recently, Chinese and South Americans since the last decades of the twentieth century), and contributions from each migrant culture have been weaved together to form a rather mixed and integrated national culture where the boundaries of all these different contributions are now blurred (Prado Júnior, 1994; Holanda, 1995; Ribeiro, 1995; Freyre, 2006). Accordingly, we can recognise tendencies for a cosmopolitan citizenship concept, for example, in the adoption of jus sanguinis and jus soli as a citizenship right at birth, or in the low requirements for naturalisation, as well as a "pluralist-evolutionist" approach to cultural differences. Disputably, we can also identify some negative reactions to the integration of some cultural groups in Brazilian history, for example in the resistance to integrating the African or the Asian migrants and, more recently, the South Americans, while there have been expressions of preference for specific kinds of migrants, especially high-skilled workers.


Public Debate and Media

This work means to clarify how the media debate could be linked to the process of decision-making around public migration policies. In this section, we first explain how the media debate can help us understand the relationship between political actors and citizens in modern democracies, and also how the use of particular frames in the media debate is related to the public responses to different policies. We later present the main findings from the analysis of the media debate over migration in Brazil.

In his "two-track model", Habermas (1997) explains how public debate can be linked with the proposition of public policies: discussions arising from the public debate are introduced by the mass media, themselves a source of information about social demands for governors. Nowadays, mass media have become the main space for political actors to communicate with citizens in modern democracies, with both groups receiving and giving information through the main means of communication. Citizens define their political preferences through information publicised in the media and by means of periodic elections, convert those preferences into political power, choosing governors and the policies they want them to adopt. Correspondingly, political actors present their points of view to citizens, obtain publicity for their actions, and also receive information about citizen preferences through the media. Responding adequately to those preferences help governors be elected to political positions or keep them in their current positions (Habermas, 1997).

In our understanding, the media is a privileged locus for visibility, with great reach and resonance in the public debate and the place where symbolical disputes are discussed by different social and political actors around certain understandings over the facts and information (Gamson, 1992; Van Gorp, 2005; Porto, 2007). According to Gamson,

we should think of them [the mass media] as the site of a complex symbolic contest over which interpretations will prevail. This cultural system encounters thinking individuals, and political consciousness arises from the interweaving of these levels (Gamson, 1992: xi-xii).

The most significant political actors contend for a place to present their understandings and frames inside the mass media system. Maia (2009) clarifies that public visibility in the mass media is important for the performance of four processes influencing the proposition of public policies:

• defining public issues and proposing solutions for those issues;

• collecting citizen's preferences and organising mobilisation around them;

• introducing public demands to elected representatives;

• organising legislative pressure and monitoring the political representatives' actions (Maia, 2009: 87)

Visibility in the mass media and, therefore, the chance to present one's preferential frame involves much more than the simple understanding of an issue. In other words, this understanding is directly linked to the ways the issue is addressed. This means that a certain understanding about one issue will have a role in shaping citizens' demands and priorities, in their comprehension of how those demands affect their daily lives, and in thinking of possible ways those demands could be addressed by the political representatives who possess political decision-making power. If the most prominent frames in the mass media place immigration as an issue related to the economic situation, the debate will probably concentrate on labour legislation, social security, or the role of migrants in maintaining the productivity level in an economic system. If the discourse in the media debate is concentrated on social-cultural issues, the debate will probably also have a focus on the integration of the migrant and host cultures, and the impact of the migrant culture in the national identity. Additionally, the media debate plays an important part because it feeds discussion in other places too, for example in the political institutions or the public debate that happens in other forums.

The method of content analysis used in this research aims to identify the frames used in the media debate in order to observe which opinions coming from the public or political debates reach the media. We especially focus on the way migrants are represented in this discourse and the suggestions on how the migrant movements should be dealt with. Here, we identify and contextualise themes and frames presented in the news coverage from 2009 and 2010 and then observe how they are inserted in the broader social-political scheme. This aids in clarifying the perceptions about the issues and their resonance in the proposition of public policies.

Frames and the process of framing by the mass media have gained importance for studies in politics and communication since the work of Erving Goffman (1986) and Tversky and Kahneman (1981; 1984). In an earlier work, Bateson (1972) highlighted the use of frames to focus on the inclusion of some piece of information in the presentation of a message while excluding others and to stress the existence of premises that guide us in the interpretation of the message (Bateson, 1972: 144). Goffman (1986) picked up on this concept to discuss the way frames help us organise our daily lives, offering "keys" and "keyings" on the type of interaction for each case, for example helping us determine whether something is a fight or a play of a fight. On their side, Tversky and Kahneman used an experimental psychology study to show how the presentation of some issues determines one's preferences by confronting people with the same options who are "framed" as positive or negative and evaluating the outcomes of their decision. The results suggest that the "framing of an action sometimes affects the actual experience of its outcomes" (Tversky and Kahneman, 1981: 458). As a consequence, presenting an issue in a different manner "could influence the decision made [...] without distorting or suppressing information, and merely by the framing of outcomes and contingencies" (Tversky and Kahneman, 1981: 346). The relative importance of the frame in shaping decisions is central to the study we propose here, as we are looking at the interplay between media debate and policymaking processes.

While those authors concentrated more on conceptual reflections on frames and framing, Gamson and Modigliani (1994; 1989) and Iyengar (1990) focused on how media frames help define the descriptions and evaluations of a determined issue. Gamson and Modigliani used focus groups to analyse how media frames related to individual understandings of themes such as affirmative action policies (1994) and the controversy around benefits and costs from nuclear power (1989), while Iyengar (1990) studied how conceptions of the causes of poverty shifted from societal to individual responsibility depending on the way it was framed. Iyengar experimented with media frames that could either have an episodic frame – an article with a concrete and personal focus, for example, a family's personal experience of poverty – or a thematic one – an abstract and impersonal article with background information, for example, on the poverty rate and the outcomes of different policies (Iyengar, 1990: 22) –, finding that episodic frames tended to place the focus of responsibility for poverty on individuals, while the thematic frames favoured societal responsibility in surveys after media exposure.

Describing the growing interest in the theme, Entman (1993) notices an astonishing diversity in the uses of the concept of framing. The author discusses many disputed conceptualisations of framing, while noting its relevance in defining problems, diagnosing causes, making moral judgements, and suggesting remedies (Entman, 1993: 52). Entering a central debate to the framing processes, Entman stresses that frames in the mass media are not simply manipulated by individuals and that culture can be perceived as a stock of commonly invoked frames (1993: 53) that are then used by journalists in the media (Gitlin, 1980; Maia et al., 2008). Entman ascribes the inclusion/exclusion tension of frames a central role in the concept and, according to him, "exclusion of interpretations by frames is as significant to its outcomes as inclusion" (1993: 54). In the study proposed here, we assume that different frames to an issue exist in our society, but the work of the media in only highlighting some of them is central in increasing their publicity in the public and political debate, while the ones excluded remain partially invisible in all three forums.

Scheufele (1999) proposed an organisation of the field according to the manner in which a frame is analysed and observes that frames can be investigated from two main perspectives. In one perspective, frame studies can focus on how the individual or the media use different frames. Alternatively, they can focus on the frame either as the outcome of a series of events (dependent variable) or as the effect leading to some other process (independent variable) (Scheufele, 1999: 109-110). Scheufele then develops a model of framing that attempts at highlighting the inputs, processes, and outcomes involved in the framing of an issue for both the media and the audience and makes an effort to place different studies along this process model, helping recognise each study's contribution in a systematic and coherent manner. The research proposed here focuses mainly on the outcomes of the media processes of framing, while also looking briefly at the inputs for this process when we reflect about how public policy can inform media debate.

In sum, the concept of frame used in this research is close to that of Gamson and Modigliani, who define it as "a central organising idea or story line that provides meaning to an unfolding strip of events, weaving a connection among them. The frame suggests what the controversy is about, the essence of the issue" (Gamson and Modigliani, 1994: 376). For our study, we place the creation and existence of frames in their wider social cultural background, but we stress the selection process from mass media actors when publicising an issue.

In order to identify frames in newspapers articles, we start by following the steps of Vimieiro and Dantas (2009), who suggested that frames can be observed in news coverage from two of the frames' main characteristics in the social reality: their semantic organisation function and their structure for the social world function. The two authors tried to put into operation the frame characteristics pointed out by Reese (2001), and Gamson and Modigliani (1989)4. The first function, organisation, develops the understandings associated to a certain theme, for instance, if the theme is presented in a positive or a negative manner or in terms of gains and losses (a cognitive organisation of frames). According to Reese (2001), there is also a cultural dimension to organisation, referring to the shared cultural values activated when the issue is presented, thereby allowing the reader to make connections to a broader setting instead of just the specific case. The author uses the examples of "Cold War" or "War on Drugs" to explain the cultural dimension of the organisation function. The second function, structure, is related to the symbols activated in relation to the issue, as such symbols have a great emphasis in the process of attachment to larger cultural ideas. Reese argues that some structures can be very explicit, but sometimes they also are intricate and implicit. In journalism, structures of meaning can be identified by the inclusion/exclusion of information (Reese, 2001:17). In the case of migration, we could mention the inclusion or exclusion of information about the importance of the migrant workforce to the recent economic situation.

In practical terms, Vimieiro and Dantas (2009) suggest that we can recognise frames in news articles by identifying actors referred to, illustrating images, terms defining the issue, proposals for solutions, metaphors, and moral judgements. A similar model for the analysis of political debates in the means of communication is proposed by Porto (2007) in order to clarify the "interpretive frames" and the symbolic disputes in which they are embedded. The author identifies five dimensions that help identify the connections between interpretive frames and the issue at hand:

• definition  of the issue (a);

• assignments of responsibilities and causes (b);

• evaluations of the importance of the issue or event (c);

• arguments over consequences (d);

• proposals for solutions (e). (Porto, 2007: 31)

We can see that three of those dimensions are also in the research by Vimieiro and Dantas (2009), but the evaluation of the importance of the issue and the arguments over the consequences will be included in this research as migration issues receive a lot of attention described along those dimensions.

To describe the debate in newspaper articles, we are also looking for additional information to assess the importance journalists give the theme, which, in turn, reflects how relevant the issue is in the public debate. The section in which the article is printed indicates the context in which the issue is presented (for instance, an article about migration could be in the economy or in the crime sections), opinion editorials organise and summarise some of the information, and the greater relevance of an issue brings it to the front pages of the newspaper. In this sense, we are looking for information in the news articles that qualifies the frames but also helps us place the public debate about migration:


Table 1



The research was performed with all articles published in the Brazilian newspaper Folha de S. Paulo in 2009 and 2010 which mentioned migration, the situation of migrant workers in Brazil, the situation of Brazilian migrant workers in other countries, and which debated migration public policies. The period reflects the initial discussions of the Proposed Law 5655/2009 in the lower legislative chamber and the publication of the "National Policy of Immigration and the Protection to Migrant Workers". Folha de S. Paulo was chosen because it was the best-selling daily newspaper with nationwide issue in Brazil at the time5, and the articles were selected using a keyword search6 in the newspaper online archive. In total, 270 articles mentioned the selected keywords, 36 of which mentioned themes excluded from this research (migration movements inside Brazilian borders or foreign investment). In addition to the categories mentioned above, three theme-specific categories were created to help qualify the articles: a category to help qualify the article as related to the frame "national security" or "migrants' human rights", a category to indicate a very famous case in Brazil during the period regarding the Italian activist Cesare Battisti7, and a category to identify whether the article made reference to the Proposed Law 5655/2009 or to the National Policy on Immigration and Protection of Migrant Worker.

The great controversy caused by the Battisti case attracted much media interest and was the theme of 106 news articles in the two years studied. After being analysed and codified, those articles were excluded from the research results. Articles about the case accounted for nearly half of the news articles produced, and the judicial dispute was prominent among them. Because they dealt with a single and specific case and debates focused more on the processes of extradition and the status of this political refugee, this coverage created a bias in the media debate about migration in Brazil. Therefore the research corpus is composed of 126 news articles.

The analysis level adopted here was the category actors. Information given by each actor was codified using the different categories presented above8. For example, the article "Law that grants amnesty to illegal foreigners in Brazil is approved", on 03/07/2009, was codified using three different actors. The journalist presented one "argument over consequences" is that the immigrants would now have access to health, education and work, while president Lula presented a "moral judgment" that states that migration policy in rich countries is unfair; and the Minister of Justice, Tarso Genro, presented a "definition of the issue" when he stated that migration policies in certain countries criminalise migrants. All three statements were identified with the frame "migrants' human rights". The choice of using actors on the analysis level allows for a more detailed description of the debate, making it possible to see the actors behind each point of view expressed and evaluate the contributions of each group of actors to the debate.

In all 126 news articles, 138 different actors presented 291 statements classified under one of the textual categories: definitions of the issue, causes of the issue, evaluation over the importance of the issue, metaphors and examples, moral judgments, arguments over consequences, and proposals for solutions. The most frequent themes of the statements were Politics, with 42% of the articles referring to it (124 statements), followed by Economy with 24% (71), Crime with 12% (35) and Demography with 10% (28). Themed under "Demography", for example, there is the article "Brazil did in decades what Europe took centuries to do" (06/09/2009), which discusses the fast process of ageing and the reduction of birth rates in the Brazilian population and the need to count on international migration which, the journalist points out, has been associated with cultural changes, social conflicts, and xenophobic reactions (definition of the issue).

Journalists were by far the most frequent group to present statements. As many as 67% (197) of the 291 statements were made by journalists, followed by Specialists representing 16% (49) of the statements, members of the Executive, 9% (27), and Civil Society organisations, 2% (7). It is remarkable that migrants were almost absent from the discussion, only presenting 4 statements (1%), as well as Brazilian citizens, who presented only one statement on the issue of migration between 2009 and 2010. From the total statements, 46% (133) referred to the national security frame, 29% (85) referred to the human rights frame, while 25% (73) of the statements refer to neither of the two frames. One example of the statements referring to the national security frame appears in the article "Terrorism in the Brazilian agenda" (16/06/2009), in an opinion editorial where a specialist in terrorism, André Luís Woloszyn, states that the increase and ease of international migration is one of the causes of the current expansion of terrorist networks (causes of the issue).

Looking at the frames data, we can clearly see the prominence of statements that can be qualified under the frame of national security rather than the ones under migrants' human rights. However, different political and public events have a large influence on the frames presented (see graph 1). We speculate that the global economic crisis may account for the larger concentration of the national security frame in the first months of 2009. In the same sense, in October 2009 there is a prominence for the migrants' human rights frame, mostly due to the publication of a study advocating for migration as a fundamental human right by the United Nations Development Program (UNDP, 2009). Graph 1 shows the distribution of statements divided by each frame:

The quantitative data is presented clustered in months to show the broader patterns involving the frames. Furthermore, it also helps to aggregate them when looking for information over the two frames, because not all articles featured statements that could be related to one or the other frame. This situation reveals the value of expanding the research to include a larger timeframe or articles from other national newspapers in order to have more robust and reliable data.

The quantitative data showed an interesting relationship between news sections and frames. The section that concentrates the majority of national security frames is "Money" (Economy section), with 57% (24) of statements in this section with the security frame against 10% (4) of human rights frames. Articles in this section are highly marked by discussions referring to the global economic crisis as well as national security concerns raised by economic hardship. The foreign issues section, "World", has 54% (36) of statements with the security frame against 36% (24) of human rights frames. Conversely, the "Power" section, comprising Brazilian Politics issues and the opinion page, is much more balanced. In this section, 42% (41) of statements presented the national security frame and 38% (37) the human rights frame. This scenario is also supported by the qualitative analysis which shows that the sections "Money" and "World" referred mainly to discussions over the economic crisis abroad, discussed other countries' propositions to expand regulations over migrants, and published news about the fiercer tensions with irregular immigrants in foreign countries, including Brazilian emigrants. The section "Power" mostly had news about the Brazilian political debate and migrant regulation and integration in Brazil. As the country has fewer conflicts with its migrants and was not so seriously affected by the global economic crisis, news in this section more often referred to the increasing need for migrant labour and discussions about changing the regulations to provide more security for migrant workers and investors.

In the next paragraphs, the discussion concentrates on two categories: the actors and the textual categories. The analysis of the statements presented by each actor reveals some interesting features of the public debate and, at the same time, shows the way journalists organise their messages in the news.

Specialists (professors, analysts, researchers, etc.) had a fair participation in the debate: they are the second group most frequently invited into the media debate, followed by members of the executive branch. News articles usually use the statements from specialists to introduce the debate to the readers, especially in the categories definition of the issue, causes of the issue, and arguments over consequences. This is the case of the political scientist Demetrius Papademetriou, who speaks about how the 2008 economic crisis is different from its precedents at the end of the twentieth century and the beginning of the twenty-first century (definition of the issue) because it is more severe and happens in a time with bigger migration flows (causes). Presenting solutions for the issue is a task given to political actors, especially members of the executive and legislative branches like a senator from the U.S. Congress, Chuck Grassley, quoted in his proposition to dismiss migrant workers first (solution) in order to guarantee the protection of national labour (definition). The civil society, although given limited participation, was called to debate causes and consequences of the issue, as did the secretary of the organisation "Continental Cry from the Excluded", Luiz Bassegio, stating that unemployed migrants prefer to stay in Brazil because of health and education services and in the hope of finding work (causes).

As stated, journalists themselves were the most recurrent group in the news articles. Their participation was more evenly distributed in all categories, but they were mostly in charge of discussing the causes of the issue, presenting definitions, presenting metaphors and examples, and presenting arguments over consequences. Immigrants and emigrants rarely had the opportunity to present their problems personally, or even to contribute to the discussion over possible solutions. Their quotes are mostly limited to anecdotal statements, retelling situations from their lives or illustrating the situations presented by journalists. Because of that they rarely present any textual category. For instance, the Japanese immigrant to Brazil Yukio Yamashita uses the metaphor of confinement to talk about his early life in rural migrant working colonies in the beginning of the twentieth century. We can note that migrants are almost absent from this discussion, and even though they are the focus point of the debate they have strikingly little participation in the textual categories.

These journalistic choices reflect how the media debate is organised (solutions are presented by political actors, migrants present examples and metaphors from their daily life) and give us insights on how the public debate, inside and outside the media, organises itself. However, the presence of journalists as the most recurrent group in the news articles and in the presentation of textual categories reminds us of their job in mediating and selecting the information in the newspapers. Traquina (2001) alerts us to the fact that journalists are subject to cultural and institutional influences, and he also calls attention to the circumstances around the production of the piece (resources, availability of sources and time, for example) in the construction of printed discourse (Tuchman, 1978; Scheufele, 1999). Acknowledging the role of journalists in newspapers, we can still use the articles as thermometers to measure the broader trends of the public discussion around an issue, especially in indicating the prevailing trends and points of view in the debate.

With the quantitative and qualitative analysis of the newspaper articles, we can also discuss some findings about the two frames of national security and migrants' human rights and how they appear to the public in the newspaper. The first observation is the lack of articles mentioning the debate, the legislative process, possible outcomes or the results of the Proposed Law 5655/2009 and the National Policy on Immigration and Protection of Migrant Worker – resulting also in the lack of discussion about the inconsistencies between the premises of both. Only one article mentioned the Proposed Law, in defence of a new regulation pertaining to human trafficking. According to the National Secretary of Justice, Romeu Tuma Júnior, one of the consequences of giving incentives to regular migration is diminishing human trafficking (arguments over consequences), as the latter is closely connected with irregular migration flows (causes of the issue).

We have already said that the national security frame has preponderance over the migrants' human rights frame, but very interestingly there are differences in the textual categories associated with each frame, meaning that the structure of the debate changes between both frames. The national security group makes major use of the categories "causes of the issue", "definition of the issue" and "metaphors and examples", and furthermore mentions several times that the recent economic crisis is the main reason behind migration policies that suggest more control over migration flows. In statements related to this frame, there is also a preponderance of negative references and metaphors to describe immigrants, for instance "illegal", "irregular", "black sheep", or "invaders". On the other hand, in the articles related to the migrants' human rights frame, most statements come under the "arguments over consequences", "definition of the issue" or "proposals for solutions" category. The most quoted statements tend to explain the advantages of inclusive public policies for migrants (arguments over consequences) to advocate for the respect of human rights as a possible solution for the issue of integration or to discuss the proposition of voting rights to migrants (proposals for solutions).

The analysis of the media debate about migration points to a restricted discussion involving a small number of actors which can in turn mean that there are limited points of view in the debate about migration in Brazil. The debate is mostly focused on specific events such as the publication of reports, analysis of political or economic scenarios, or police operations that trigger a small amount of publicity to the theme of migration. Much of the news coverage in the period actually discussed policy changes in other countries or significant cases involving migrants outside Brazil. Another sign that the media debate is still underdeveloped in Brazil is that the concerned population – migrants – are almost always absent from the discussions. Despite these limitations, it is an important feature of the media debate that the actors who take part in it perform the same roles that they already occupy in the political or the public debate. For example, specialists are called to introduce and analyse the situation, political actors to present solutions, and the civil society to debate causes of the issue and its consequences. In the same sense, although the specific events regarding the political debate on the Proposed Law and the National Policy have not been discussed in the media, other political and public events have fed the discussions in the newspaper, which further connects the media debate to the other two debates.


Concluding Remarks

We claim that the present method for quantitative and qualitative analysis of the media debate about migration in the newspaper Folha de S. Paulo in 2009 and 2010 is useful to show some broad tendencies of the recent public debate over the issue and show some opinions coming from it. The combination of those two types of analysis is a good contribution to recognise variations on the debate and identify how migration is shown in the newspapers. The importance of the political and economic context in the choice of themes and frames in the news indicates that the media debate is closely related with the debates going on in public and the political debate – the asserted influence of the 2008 economic crisis on the prevalence of the national security frames is one example. Ideally, the expansion of the period studied and the number of newspapers analysed can provide more information about diversity in the media debate.

It is also a very important finding that the roles played by specific groups of actors in the media debate are in line with their presumed roles in the political and public debates. We believe that this fact reinforces our supposition that the media debate works in connection with the public debate and can be a good site for the analysis of current and past opinions and points of view about an issue.

Regarding the debate on human rights, the great majority of references are about human rights already guaranteed to migrants, such as access to the labour market, education or justice, and the debate lacks discussions about the expansion of rights, for instance, the right to vote. Even though the news articles in this study do not directly mention the legislation or the public policies for migration, we can observe a link between them to the extent that elements from both the Proposed Law and the National Policy are also constant in the media debate. The importance of international human rights regimes and the necessity for a clear integration policy for migrants – elements from the National Policy – are mentioned many times in the news coverage. The same is true for statements that make the case for broader governmental control over migrant workers or the precedence of the national citizens' rights over migrants' rights. Both are components of the Proposed Law and of some news articles, especially at the beginning of the year 2009 when the legislative discussion started in the Congress and the effects of the 2008 economic crisis could be felt more intensively.



BATESON, Gregory. (1972), A Theory of Play and Fantasy, In: Steps to an Ecology of Mind. Chicago: University of Chicago Press, pp. 138-148.         [ Links ]

BELLAMY, Richard., and WARLEIGH-LACK, Andrew. (1998), From an Ethics of Integration to an Ethics of Participation: Citizenship and the Future of the European Union, Millennium: A Journal of International Studies, vol. 27, n° 3, pp. 447-470.         [ Links ]

BRASIL, Câmara dos Deputados. (2009), Projeto de Lei 5655/2009,  <         [ Links ]>

BRASIL, Câmara dos Deputados. (1980), Lei 6815/1980, <>         [ Links ].

BRASIL. Conselho Nacional de Imigração (2012), Resolução Normativa 97/2012, <,%20DOU%2013jan12-1.doc>         [ Links ].

BRASIL, Ministério do Trabalho, Conselho Nacional de Imigração. (2010), Política Nacional de Imigração e Proteção ao(a) Trabalhador(a) Migrante ("National Policy on Immigration and Protection to the Migrant Worker"), 12 de Maio de 2010.         [ Links ]

CASTLES, Stephen. (1986), The Guest-worker in Western Europe – An Obituary, International Migration Review, vol. 20, nº 4, pp. 761-778.         [ Links ]

_______. (2006), Guestworkers in Europe: A Resurrection?. International Migration Review, vol. 40, nº 4, pp. 741-766.         [ Links ]

DEMANT, Peter R. (2009), Het Islamdebat in Nederland: van exceptionalisme tot convergentie?, Res Publica, vol. 51, nº 2, pp. 239-256.         [ Links ]

ENTMAN, Robert M. (1993), Framing: Toward Clarification of a Fractured Paradigm, Journal of Communication, vol. 43, nº 4, pp. 51-58.         [ Links ]

FOLHA DE S. PAULO. (2009-2010) São Paulo, Online. Archive available in: <>         [ Links ]

FREEMAN, Gary P. (1995), Modes of immigration politics in liberal democratic states. International Migration Review, vol. 29, nº 4, pp. 881-913.         [ Links ]

FREYRE, Gilberto. (2006), Casa Grande & Senzala. São Paulo: Editora Global, 51st edition.         [ Links ]

GAMSON, William A. (1992), Talking Politics. Cambridge: Cambridge University Press.         [ Links ]

_______., and MODIGLIANI, Andre. (1989), Media Discourse and Public Opinion on Nuclear Power: A constructionist approach, American Journal of Sociology, vol. 95, nº 1, pp. 1-37.         [ Links ]

________., and MODIGLIANI, Andre. (1994), The Changing Culture of Affirmative Action, In: Equal Employment Opportunity, edited by Paul Burnstein. New York: Aldine de Gruyter.         [ Links ]

GITLIN, Todd (1980), The whole World is Watching. Berkeley: University of California Press.         [ Links ]

GOFFMAN, Erving. (1986), Frame Analysis: an essay on the organization of experience. York: The Maple Press.         [ Links ]

HABERMAS, Jürgen. (1997), Direito e democracia: entre facticidade e validade. Volume I. Rio de Janeiro: Tempo Brasileiro.         [ Links ]

HOLANDA, Sérgio Buarque de. (1995), Raízes do Brasil. São Paulo: Companhia das Letras, 26th edition.         [ Links ]

HURRELL, Andrew. (2007), On Global Order: Power, Values and the Constitution of International Society. New York: Oxford University Press.         [ Links ]

HUGO, Graeme. (1996), Environmental concerns and international migration, International Migration Review, vol. 30, nº 1, pp. 105-131.         [ Links ]

IYENGAR, Shanto. (1990), Framing Responsibility for Political Issues: The Case of Poverty, Political Behaviour, vol. 12, nº 1, pp. 19-40.         [ Links ]

KOOPMANS, Ruud., and STATHAM, Paul. (1999), Challenging the Liberal Nation-State? Postnationalism, Multiculturalism, and the Collective Claims-Making of Migrants and Ethnic Minorities in Britain and Germany, American Journal of Sociology, vol. 105, nº 3, pp. 652-696.         [ Links ]

LIJPHART, Arend. (1968), The Politics of Accommodation: Pluralism and Democracy in the Netherlands. Berkeley: University of California Press.         [ Links ]

LINKLATER, Andrew. (1998), Cosmopolitan Citizenship, Citizenship Studies, vol. 2, nº 1, pp. 23-41.         [ Links ]

MAIA, Rousiley C. M. (2009), Atores da sociedade civil e ação coletiva: relações com a comunicação de massa, Lua Nova. Revista de Cultura e Política, vol. 76, pp. 87-118.         [ Links ]

______., DINIZ, Aline., VELOSO LEÃO, Augusto., SANTOS, Débora., OLIVEIRA, Vanessa., and GUIMARAES, Victor. (2008), Mídia e enquadramentos em ambientes competitivos: a troca pública de razões. In: XXXII Encontro Anual da Anpocs - Associação Nacional de Pós Graduação e Pesquisa em Ciências Sociais, Caxambu, MG: Grupo de Trabalho em Democracia, Comunicação Política e Eleições.         [ Links ]

MATTAR, Marina R.M. (2012), Migrações ambientais: o caso dos pequenos países insulares, Universitas Relações Internacionais, vol. 10, nº 1, pp. 111-121.         [ Links ]

PORTO, Mauro P. (2007), Framing Controversies: Television and the 2002 Presidential Election in Brazil, Political Communication, vol. 24, nº 1, pp. 19-36.         [ Links ]

PRADO JUNIOR, Caio. (1994), Formação do Brasil Contemporâneo. São Paulo: Editora Brasiliense, 23rd editon.         [ Links ]

REESE, Stephen D. (2001), Prologue – Framing Public Life: a bridging model for media research. In: Framing Public Life: Perspectives on Media and our Understanding of the Social World, edited by Stephen Reese, Oscar H. Gandy and August E. Grant. Mahwah: Lawrence Erlbaum, pp. 7-31.         [ Links ] 

REIS, Rossana R. (2007), Políticas de Imigração na França e nos Estados Unidos. São Paulo: Aderaldo & Rothschild.         [ Links ]

RIBEIRO, Darcy. (1995), O Povo Brasileiro: a formação e o sentido do Brasil. São Paulo: Companhia das Letras.         [ Links ]

ROSENZWEIG, Mark. (2005), Consequences of Migration for Developing Countries. United Nations Expert Group Meeting on International Migration and Development. New York, 6-8 July, <>         [ Links ].

RUDOLPH, Christopher. (2003), Security and the Political Economy of International Migration, The American Political Science Review, vol. 97, nº 4, pp. 603-620.         [ Links ]

SCHEUFELE, Dietram A. (1999), Framing as a theory of media effects, Journal of Communication, vol. 49, nº 1, pp. 103-122.         [ Links ]

TRAQUINA, Nelson. (2001), O Estudo do Jornalismo no Século XX. São Leopoldo: Editora Unisinos.         [ Links ]

TUCHMAN, Gaye. (1978), Making the news. Nova York: Free Press.         [ Links ]

TVERSKY, Amos., and KAHNEMAN, Daniel. (1981), The Framing of Decisions and the Psychology of Choice, Science, vol. 211, nº 4481, pp. 453-458.         [ Links ]

_______. (1984), Choices, Values and Frames, American Psychologist, vol. 39, nº 4, pp. 341-350.         [ Links ]

UNITED NATIONS. (1990), International Convention on the Protection of the Rights of All Migrant Workers and Members of their Families, <>         [ Links ]

UNITED NATIONS DEVELOPMENT PROGRAM. (2009), Overcoming Barriers: Human mobility and development, Human Development Report 2009, <>         [ Links ].

VAN GORP, Baldwin. (2005), Where is the frame? Victims and Intruders in the Belgian Press Coverage of the Asylum Issue, European Journal of Communication, vol. 20, nº 4, pp. 484-507.         [ Links ]

VELOSO LEÃO, Augusto. (2011), Governança Regional das Migrações: a lei de imigração na União Europeia, Alemanha, Áustria e Suíça. 3rd Annual Meeting of Brazilian Association of International Relations, São Paulo, SP.         [ Links ]

VENTURA, Deisy., and ILLES, Paulo. (2010), Estatuto do estrangeiro ou lei de imigração?, Le Monde Diplomatique Brasil, vol. 4, nº 37, pp. 14-15.         [ Links ]

VIMIEIRO, Ana Carolina., and DANTAS, Marcela. (2009), Entre o implícito e o explícito: proposta de análise de enquadramentos da mídia, Lumina, vol. 3, nº 2, pp. 1-16.         [ Links ]

WEINER, Myron. (1993), Security, Stability, and International Migration, International Security, vol. 17, nº 3, pp. 91-126.         [ Links ]

WORLD BANK. (2012), Annual Remittances Data: Inflows, <>         [ Links ].



Submitted in February 2012
Accepted in March 2013




1 In the Proposed Law 5655/2009, the CNIg would be renamed the National Council of Migration and its responsibilities would be expanded to also act in the protection of emigrated Brazilians and the policymaking regarding both the immigration and emigration of peoples.
2 The World Bank published, in April 2010, a series of data showing that developing countries are significantly more dependent on international remittances than developed countries. In 2008, developing countries have, on average, received an amount of remittances corresponding to 2% of their Gross National Product (GNP), while in developed countries it corresponded to 0.2% of their GNP on average. The data also shows some developing countries that are extremely dependent on international remittances such as Lebanon (where the remittances corresponded to 20.4% of the country's GNP in 2008), Honduras (20.4%), and Haiti (20.3%). Data were obtained from estimates by the World Bank based in the International Monetary Fund's Balance of Payments Statistics Yearbook 2008, (World Bank, 2010).
3 The concept of consociational politics, from Lijphart (1968) is one example of how a society could organise itself as "pluralist-essentialist". Variations in the multicultural policies could give way for the hegemony of one cultural group, in which case this "pluralist-essentialist" society could be called "communalist".
4 Originally, Reese described six characteristics of frames: organising, principles, shared, persistent, symbolically, structure ("Frames are organising principles that are socially shared and persistent over time, that work symbolically to meaningfully structure the social world") (Reese, 2001:11-12). [Emphasis added].
5 The Instituto Verificador de Circulação (IVC), which inspects the print publications in Brazil places Folha de S. Paulo as the best-selling daily newspaper between 1986 and 2009. In 2010, the newspaper Super Notícias, a popular tabloid, became the best-selling daily newspaper in Brazil, but it only prints local issues in the city of Belo Horizonte. At the time of this research, Folha de S. Paulo was the only large national newspaper with an online archive search tool that enables keyword query in specific time periods.
6 Searched keywords (in Portuguese): migrant; migrants; immigrant; immigrants; emigrant; emigrants; migration; migrations; immigration; immigrations; emigration; emigrations; foreigner; foreigners (migrante; migrantes; imigrante; imigrantes; emigrante; emigrantes; migração; migrações; imigração; imigrações; emigração; emigrações; estrangeiro; estrangeiros; estrangeira; estrangeiras).
7 Cesare Battisti is an Italian citizen arrested in Brazil at the end 2007, charged with four murders in Italy since 1987. The crimes were committed at the end of the 1970s, a time when Battisti was supposedly a member of an armed extreme left group, which Battisti denies. The activist was accepted as a refugee in France until 2004, when the French government issued his extradition to Italy. The activist then escaped to Brazil. At the end of 2008, the Minister of Justice in Brazil gave him the status of political refugee, claiming a "proven fear of persecution because of his political ideas" and in opposition to a decision in favor of his extradition by the National Committee for Refugees. The case received a lot of attention in Brazil and in Italy and has been analysed in different aspects by the Brazilian Supreme Court, which, in the end, gave the president the final word about the extradition. President Lula rejected the extradition in his last day in office in 2010, and this decision was once again analysed and ratified by the Supreme Court in 2011. Battisti now has a refugee status in Brazil and was released after staying in prison for more than 4 years in Brazil.
8 We performed an agreement test with a sample of the news articles. In total, the analysis of 12 news articles and 120 textual categories were compared between the two independent raters. The raw agreement coefficient scored 0.6, while the Cohen's kappa coefficient rated 0.58, suggesting moderate agreement between raters.

Creative Commons License All the contents of this journal, except where otherwise noted, is licensed under a Creative Commons Attribution License