Assessing some measures of online deliberation^*

Abstract

The empirical turn in deliberative democracy has fostered the development of different methodological procedures. Within this literature, studies focusing on the internet have gained increasing attention. The belief that the internet may help solve some of the deliberative deficits of democracies has propelled an interest in the potential benefits and problems of online discourse. This article seeks to discuss some of the methods that have been advocated for the study of online deliberation to point out three of their weaknesses: (01) the establishment of misleading distinctions; (02) the neglect of the implications of the deliberative system; and (03) the disregard of some specificities of the internet.

Online deliberation; deliberative democracy; empirical turn; methodology; internet

The empirical turn in deliberative democracy has brought a renewed vitality to this field of research. After a decade of fruitful philosophical development, the 21st century witnessed a growing interest in rethinking conceptual frameworks through empirical inquiry, thus pushing the frontiers of deliberative theory in new directions. Habermas (2005) has endorsed this move, and Kies (2010) attributes this validation to the lack of empirical evidence on the grounds of a highly abstract perspective.

It is therefore not a surprise that Habermas has very recently strongly appreciated and encouraged the efforts accomplished by researchers from around the world to operationalize and test the criteria and presuppositions of the deliberative democratic model in different contexts of discursive interaction (KIES, 2010, p. 34).

This turn has fostered the development of methods applicable to different types of discursive arenas aimed at tackling diverse problems (BLACK et al., 2009; DRYZEK, 2008). Most studies seek to assess either the deliberativeness of specific types of interaction (KELLY, 2008; STEINER et al., 2004; WESSLER, 2008) or the effects of these processes on citizens, decision making and society in the broad sense (DELI CARPINI et al., 2004). There are investigations devoted to tracking preference change (DRYZEK and NIEMEYER, 2006), the exposure to opposite perspectives (LEV-ON and MANIN, 2009; MUTZ, 2006), and processes of social learning provoked by deliberation (KANRA, 2009). There is also significant interest in the design of participatory experiments¹ and the role of the media².

Within this profuse literature, studies focusing on the internet have been gaining attention. The belief that the internet may help solve some of the deliberative deficits of democracies has fueled an interest in the potential benefits and problems of online discourse. The aim of this article is to present some of the methods that have been advocated for the study of online deliberation to point out some of their weaknesses. An element of these weaknesses emerges from problems in the empirical literature on deliberative democracy in general, while other issues are specific to texts related to the online phenomena.

This article begins with a very brief review of online deliberation, followed by the presentation of some analytical approaches utilized to study this topic. The following sections provide a discussion of three weaknesses of the previous approaches to online deliberation: (01) the establishment of misleading distinctions, (02) the neglect of the implications of the deliberative system, and (03) the disregard of some specificities of the internet. It must be clear that it is not my aim, in this article, to advocate deliberative democracy against its critics or to deal with the many relevant criticisms raised against this democratic perspective. There is an extensive literature covering this debate³. My goal is to foster the advancement of a debate within the deliberative approach, contributing to the development of this literature in its own grounds.

Online deliberation and its measures

Online deliberation is one of the main areas of interest among the most innovative research on deliberative democracy (DAVIES, 2009). Following the excitement evident in studies from the early 1990s that anticipated the emergence of a new public sphere on the internet, and on a more critical perspective, several scholars have tried to understand online practices by examining them through the lens of deliberation⁴.

Briefly, deliberation has been understood as a process reflecting a public give-and-take of reasons marked by the equality of its participants. Deliberation is a dialogical practice during which social actors seek to convince each other through a discursive exchange. Based on the Habermasian theory, this idea was developed over the last two decades, becoming one of the most productive areas on political theory (DRYZEK, 2007)⁵.

The most recent literature addressing the deliberative approach has highlighted the fact that deliberation should be viewed as a process resulting from the overlap of several arenas and discursive moments⁶. These studies have also emphasized that the give-and-take of reasons that serve as the foundation of deliberation occur through a variety of communicative formats, including the presentation of ideas in an emotional way. In addition, deliberation does not require that participants become altruistic beings without particular preferences (MANSBRIDGE et al., 2010; MENDONÇA and SANTOS, 2009). The deliberative process only requires a public clash of discourses that induces reflection in a non-coercive way and promotes a connection between particular experiences and more general principles (DRYZEK, 2006, p. 52).

Online forums may function as arenas that play a role in broader discursive processes, thus nurturing public deliberation (COLEMAN and MOSS, 2012). Far from compromising the benefits of face-to-face group meetings, computer mediated communication may prove especially useful for deliberative work (PRICE, 2009, p. 37). Raphaël Kies (2010) also argues that there is no original contradiction between the internet and deliberation, although some scholars claim that the former can only foster frivolous and empty interactions. Despite his criticism of several approaches to online deliberation, Arthur Lupia (2009) is another scholar who admits its potential: "Online deliberation […] is promising because of its ability to bring people together for the purpose of information exchange without the difficulties caused by physical distances between participants" (LUPIA, 2009, p.60).

In the search for online possibilities regarding deliberation, deliberative democrats have conducted a wide range of investigations. Among the pioneering works in this field are Wilhelm's (2000) investigation about Usenet in the United States, Jensen’s (2003) comparison of a Usenet group (dk.politik), and a government sponsored forum (Nordpol.dk) in Denmark, and Dahlberg's (2001) studies on the renowned experience of Minnesota E-Democracy. The research conducted by Graham and Witschge (2003) is also significant in these early stages of online deliberation research. The investigators focused on a British governmental website (UK Online), which, at the time, had around 20,000 posts.

An increase in the number of studies about online deliberation has led to an emergence of analyses with varying focuses. There are studies about: the design of forums (DAVIES and CHANDLER, 2012; SÆBØ et al., 2009; WRIGHT and STREET, 2007); the deliberativeness of online arenas (HAMLETT, 2002; JANSSEN and KIES, 2005; KIES, 2010; SAMPAIO et al., 2011; STROMER-GALLEY, 2007; WALES et al., 2010); the comparison between online and conventional media spheres (GERHARDS and SCHÄFER, 2010); the role of the internet in promoting contact between opposing perspectives (LEV-ON and MANIN, 2009; MUTZ, 2006; WOJCIESZAK and MUTZ, 2009); the use of online consultations (COLEMAN and SHANE, 2012; DAVIES and CHANDLER, 2012; FISHKIN, 2009; KIES, 2010; SHANE, 2009); and the potential impact resulting from these processes (FRESCHI and METE, 2009). These investigations have offered rich methodological approaches, which vary greatly depending on the type of research problem being addressed. In this article, it would be impossible to provide fair coverage of each of these routes. I will therefore focus on some of the most influential analytical frameworks. However, throughout the discussion I will make quick references to specific aspects of other methodological proposals that will not be featured here.

The first approach I wish to focus on is the Discourse Quality Index(DQI), proposed by Steiner et al. (2004) for the study of parliamentary deliberations. Following its establishment, this technique was developed further, and has been applied to the comprehension of other discursive spheres (BÄCHTIGER et al., 2009; PEDRINI, 2012). Praised by Habermas (2005) as 'splendid', and increasingly used in empirical studies, the DQI has become the most renowned method for the micro-analysis of deliberation, and is one of the most influential approaches for scholars of online deliberation (KIES, 2010). Such influence should not be seen as a misuse of the DQI, because it has been advocated by its proponents: "the DQI can be applied easily and reliably to a wide range of deliberative contexts" (STEENBERGEN et al., 2003, p, p. 22).

The first version of DQI is based on six Habermasian principles: (01) open participation; (02) justification of assertions; (03) consideration of the common good; (04) respectful treatment; (05) attempt to reach a rationally motivated consensus; and (06) authenticity. To transform these principles into operable variables, Steenbergen et al. (2003) propose the following framework (Table 01):

Table 01
Categories for assessing deliberation according to DQI

Criteria	Categories
Participation	(0) Interruption of a speaker
	(1) Normal participation is possible
Level of Justification	(0) No justification
	(1) Inferior justification: incomplete inference
	(2) Qualified justification: complete inference
	(3) Sophisticated justification: more than one complete justification
Content of Justifications	(0) Explicit statement concerning particular interests
	(1) Neutral statement
	(2a) Explicit statement of the common good in utilitarian terms
	(2b) Explicit statement of the common good in terms of the Rawlsian difference principle
Mutual Respect	For the groups:
	(0) No respect
	(1) Implicit respect
	(2) Explicit respect
	Toward the demands of others:
	(0) No respect
	(1) Implicit respect
	(2) Explicit respect
	Towards counterarguments:
	(0) Counterarguments ignored
	(1) Counterarguments included but degraded
	(2) Counterarguments included — neutral
	(3) Counterarguments included and valued
Constructive Politics	(0) Positional politics
	(1) Alternative proposal
	(2) Mediating proposal

• Source: Adapted from Steenbergen et al. (2003, pp. 27-30)

After the DQI's initial formulation, the approach has been expanded upon and updated.Bächtiger et al. (2009) put forth three theoretical reformulations: (01) the consideration of alternative forms of communication (labeled Type 02 Deliberation) besides only contemplating rational discourse; (02) the establishment of standards for considering an interaction as deliberative; (03) the adoption of a sequential approach that does not anticipate encountering all of the features of deliberation throughout the whole process. Such reformulations affect the analytical matrix advanced by these scholars.

Equality is measured "by counting the frequency of participation as well as by counting its volume (measured by the number of words)" (BÄCHTIGER et al., 2009, p. 05). Five levels of justification are considered, with the inclusion of one additional category: sophisticated justification (in depth), which means that "a problem is examined in a quasi-scientific way from various viewpoints" (BÄCHTIGER et al., 2009, p, p. 05). The variable Respect and Agreement "measures whether speakers degrade (0), treat neutrally (01), value (02), or agree (03) with positions and counterarguments" (BÄCHTIGER et al., 2009, p, p. 06). A variable identified as Interactivity assesses mutual references between arguments. Concerning Constructive Politics,Bächtiger et al. (2009) establish four categories; in addition to the three factors already suggested in the original version of the DQI, the authors believe consensus appeals should also be considered.

To evaluate Type 02 Deliberation, the scholars consider the possibility of deliberative negotiations, arguing that the use of threats and promises "allows to empirically distinguish between 'deliberative' and 'non-deliberative' forms of negotiation" (BÄCHTIGER et al., 2009, p. 08). Lastly, this revised version of the DQI measures the use of story-telling as a source of justification.

The idea of looking at the sources of justification was expanded on by Jennifer Stromer-Galley (2007), who developed one of the most influential frameworks for scholars dedicated to the comprehension of online deliberation. Her micro-analytic approach is based on six elements: (01) reasoned opinion expression, (02) references to external sources when articulating opinions, (03) expressions of disagreement and hence exposure to diverse perspectives, (04) equal levels of participation during the deliberation, (05) coherence with regard to the structure and topic of deliberation, and (06) engagement among participants with each other (STROMER-GALLEY, 2007, p. 04).

Stromer-Galley (2007) develops a complex coding scheme that begins with dividing units of discourse into four categories that specify the type and aim of that particular component: problem (focused on the issue), metatalk (talk about talk), process (talk about the process) or social (talk that fosters or hinders social bonds). The next step involves tracingthoughts within the segmented units. "The 'thought' is the unit of analysis for which the deliberations are coded" (STROMER-GALLEY, 2007, p. 22). These thoughts should be understood withinturns that may Start a new topic, Respond on topic, Respond to moderator or Continue self.

Thoughts that express the problem focused on the arena are coded as expressing anOpinion, an Agreement, aDisagreement, a Fact or aQuestion. Thoughts representing metatalk are divided into manifestations of Conflict, Consensus, Clarification of one's own positionor Clarification of someone else's position. Thoughts regarding process can point to Technical Problems associated with the process, which may include Technical Benefits, Deliberation Process, Deliberation Problems or Deliberation Positive.Lastly, thoughts coded as social can be designated as Salutations, Apologies, Praise or Chitchat. Problem and Metatalk thoughts that are on topic are further coded according to their valence: For; For-but; Against; Against-but orUnsure. In addition, when these thoughts are expanded on, the source of such elaboration is coded as Personal Experience, Briefing Documents, Mass Media or Other Participants. In contrast to the DQI, "the elaboration measure did not categorize the types of reasons offered, the quality of the reasons, nor the accuracy or factual nature of the reasons" (STROMER-GALLEY, 2007, p, p. 10).

Equality among participants "was measured by counting the frequency of participation and by volume—measured by number of words" (STROMER-GALLEY, 2007, p. 11). The measurement of engagement included not only the levels of responsivity, but also the formulation of non-rhetorical questions. Ultimately, the model fostered by Stromer-Galley (2007) has several similarities with the DQI, and thus advances a matrix for a micro-content analysis.

A third proposal focusing on this type of micro analysis was recently presented by Raphäel Kies (2010), who gathered elements from different models, including those used by the DQI and by Stromer-Galley (2007). His indicators are presented in Table 02, below:

Table 02
Indicators for assessing deliberation according to the Kies (2010)

Criteria	Indicators
Inclusion	Evaluation of the ease of access to the online forum, on the basis of connectivity, ICT skills and discursive rules
Discursive equality	Assessment of discursive concentration and the level of control of the debate
Reciprocity	Measurement of the proportion of posts that are within a thread and the proportion that start a new thread, in addition to the assessment of the extent to which posts take into consideration opinions previously presented
Justification	Evaluation of whether the opinions are justified or not and how complex justifications are. It should also be observed he depth of justifications, which is measured by coding the use of internal (based on personal viewpoints) or external (based on facts) justifications
Reflexivity	Content analysis points to apparent cases of reflexivity. Surveys and interviews help demonstrate more internal processes
Empathy	Measurement of cases of disrespect and surveys and interviews that ask users about levels of respect
Sincerity	Assessment of apparent cases of insincerity. Surveys and interviews indicate the participants’ perceptions of the intensity of each other's sincerity
Plurality	Evaluation of sociodemographic profiles of participants and their political involvements
External Impact	Signs of extension of the discussion to an external agenda. Participation of political personalities in the forum. Users participate in other discussion spaces. There are concrete outcomes.

• Source: Kies (2010, p, pp. 56-57).

Kies's (2010) model presents some important differences in relation to the two frameworks that were previously mentioned. One important distinction is the consideration of elements that point to the external impact of the forum. The forum is not investigated as a contained environment. A second difference, which is related to the first, addresses the use of surveys and interviews in addition to content analysis. These methods help to promote a more complex picture of processes.

Surveys are also at the heart of methodological procedures adopted by some studies seeking to investigate the exposure of internet users to other perspectives. Wojcieszak and Mutz (2009) used a sample of 1,028 respondents from a large national survey in the United States to investigate if, how and when Americans discuss politics online. The main goal of their survey was to observe the extent to which political talk arising in various types of online groups serve to expose participants to like-minded views as opposed to challenging them via exposure to disagreement (WOJCIESZAK and MUTZ, 2009, p. 43). Participants who had been engaged in online political discussions were asked whether they usually agreed or disagreed with the other participants' views. This research was developed utilizing some aspects initially addressed by Diana Mutz (2006). She attempted to study real world informal conversations through "several representative national surveys that included information on Americans' networks of political discussion" (MUTZ, 2006, p, p. 21). The conclusions of this study suggest "cross-cutting exposure is negatively related to participation" (MUTZ, 2006, p, p. 112).

Although a discussion of the results of these investigations would be valuable, this article will focus on some of the weaknesses of the methods applied. In doing so, my analysis obviously points to the potential shortcomings in the findings that were reached. Each of the following sections focuses on presenting one of such weakness.

The establishment of misleading distinctions

Micro approaches to online deliberation seem fascinated by detailed coding schemes that often lead to classifications, which do not deepen our knowledge of the topic. Excessive quantification directs investigations to fallacies that lack theoretical grounds. Within the obsessive exploration of exhaustive analytical matrices, the purpose of many distinctions is not only unclear, but also misleading. This often puts the broader comprehension of the process in jeopardy. Dahlberg (2004) has made a similar argument, when he claims that:

The fundamental problem is that operationalisation requires researchers to focus upon those aspects of the public sphere for which narrowly defined and measurable indicators can be found, thus neglecting other aspects less amenable to quantification. The result is serious loss of meaning (DAHLBERG, 2004, p. 31).

My argument is that, besides the problem of excessive quantification (with its focus on 'measurable indicators'), there is the problem of establishing distinctions that may prove misleading. Many scholars are devoted to demanding classifications that may hinder the interpretation of deliberation as a political process.

This problem becomes apparent, for instance, within the levels of typology regarding justification that are suggested by the DQI. The categorization of inferior, qualified, sophisticated and in depth justification does not reveal much about deliberative processes for several reasons. First of all, judging the completeness of an inference is not as simple as the proponents of the DQI seem to imply, and may vary across different cultures. Secondly, the number of justifications presented does not determine the strength of one's argument. Neither does using quasi-scientific examinations to interpret problems. Complex justifications may even hinder deliberation as such, because they may compromise the general comprehensibility of discourses or embarrass other participants. As observed byDahlberg (2004), "the most prolific posters and positions do not necessarily command the most attention" (DAHLBERG, 2004, p, p. 35). Categorizing levels of justification may nurture some critiques of deliberation that (wrongly) point to the elitist nature of the theory. Thirdly, the coding of the levels of justification neglects the basis of deliberation. The strength of this process should not depend on individual opinions, but on the broader process within which these remarks are inserted. The DQI individualizes processes and transforms deliberation into nothing more than an exchange of utterances. The philosophy of the conscience, strongly contested by Habermas (1987), returns through the backdoor. Rich deliberative processes can be demolished by coders simply because they view each utterance as being unsophisticated. On the contrary, a weak process can be praised for featuring isolated actors and opinions.

Several other criteria proposed by the DQI suffer from similar problems. The criterion of Mutual Respect, for instance, establishes two misleading hierarchies. The first distinguishes implicit respect from explicit respect, using ordinal variables. However, the proponents of the model never explain why explicit demonstrations of respect are preferable to implicit manifestations. The second concerns the respect given to counterarguments. Once again, ordinal variables are used to grade different types of behavior asymmetrically. Applying a value to someone's argument is considered better for deliberation than treating it neutrally. Agreeing with a counterargument is even better, according to the DQI. An approach such as this neglects the agonistic dimension of deliberation. Deliberation requires taking other positions into consideration, but not necessarily agreeing with them. A discussion consisting of numerous agreements may be significantly poorer than one in which arguments are treated neutrally.

The criterion Content of Justifications also leads to unnecessary distinctions. In the first case, it presumes that particular interests and the common good can be easily distinguished and are opposed to each other. The DQI neglects the fact that these dimensions may often go hand-in-hand or be intertwined. In addition, the criterion creates a distinction between two types of common good which seems unwarranted. Why should the notion of the common good be restricted to utilitarian terms and the Rawlsian principle of difference? And why is it important to distinguish types of common good? What does this categorization explain about deliberation?

Lastly, the category Constructive Politics values mediating proposals more than the elaboration of alternative proposals. Consensus appeals receive an even higher grade than mediating proposals. Once again, this type of hierarchy is based on a questionable restrictive conception of deliberation that assumes the middle way is always the best route. Why are alternative proposals not more deliberative than mediating arguments? What type of consensus is implicit in this coding scheme? Are workable agreements (DRYZEK, 2000; ERIKSEN, 2000; JAMES, 2004) included in the coding? Are the authors talking about the often criticized idea of substantive consensus or about meta-consensus (DRYZEK and NIEMEYER, 2006)? Is the non-radical middle way always better in deliberative terms? With this type of hierarchy, the DQI, once again, seems to feed the critiques of deliberative theory with misguided and unclear assumptions.

The revised version of the DQI also creates a new problem, which was not present in the first version of the method. Besides the internal hierarchies within categories, the revised version develops a hierarchy regarding types of deliberation. According to Bächtiger et al. (2009) there would be a more demanding type of process (Type I Deliberation) and a more informal one (Type II Deliberation). Instead of simply differentiating between forms of providing reasons, however, the authors set a very clear normative distinction. Type II Deliberation would "involve a shift away from the idea of purely rational discourse toward a conception of deliberation that incorporates alternative forms of communication (such as story-telling) and embraces self-interested behavior such as bargaining" (BÄCHTIGER et al., 2009, p. 03). Story-telling is not accepted as another way to provide reasons, but is considered a totally different form of communication. It would be simply a more realistic perspective when trying to comprehend real world practices, which are usually below the standards of Type I Deliberation. Throughout their article, the authors explicitly establish different standards for these 'types' of deliberation.

If the DQI establishes some misleading distinctions, it should not be seen as the only method for doing so. Stromer-Galley's (2007) proposal also suffers from this type of problem. To begin with, she sets up a very fragmented division of the units of discourse that does not seem very helpful at a more aggregated level. The distinction of units (which are coded as problem,metatalk, process or social) breaks up the discursive process into unnecessary fragments. These fragments can also be misleading. For instance, the establishment or hindrance of social bonds happens in many ways and not exclusively through what she outlines as 'social'.

However, an even more problematic issue unfolds in her most celebrated criterion,sourcing.Stromer-Galley (2007) advocates that the source of posts should be coded, proposing four main categories: (01) Personal Experience, (02) Briefing Documents, (03) Mass Media or (04) Other Participants. Such distinctions present at least three problems. Firstly, most participants in deliberative processes do not make explicit references to the sources of their opinions. Although some online arenas contain a great number of posts with links to other documents, this should not be seen as a valid rule for any sort of online experiment. Secondly, the categorization proposed by Stromer-Galley (2007) neglects other possible sources participants may frequent, as well as the intertwinement of the sources she discusses. The mass media, for instance, should not be reduced to a channel of information, but sits at the heart of contemporary personal experiences (SILVERSTONE, 2002). Thirdly, and more importantly, there is a problematic assumption that pervades her analysis. This theory implies that personal experiences are somehow poorer than other types of sources. She expected to find more references to the briefing documents, and seemed frustrated by the predominant use of personal experiences.

This type of assumption also manifests in another renowned procedure for the study of online deliberation. Jensen (2003)distinguishes between internal and external justifications, with an implicit preference for the later. Internal justifications, based on personal viewpoints, are somehow considered to be a more superficial and less demanding way to present one's positions. The ability to refer to other sources is valued as an indicator of the qualifications of participants. However, such a view shows how the methods applied may underestimate the capacity of story-telling to present reasons in a publicly comprehensible way. Personal experiences should not be seen as a less informed way to foster one's position. This type of hierarchy devalues personal experiences and neglects the essential grounds of deliberative theory, which is a technique that anticipates varying contributions within a discursive process.

Neglected implications of the concept of a deliberative system

Despite the broad theoretical acceptance of the concept of a deliberative system⁷, most empirical studies addressing online deliberation still neglect its implications. The notion of a deliberative system advances the understanding of deliberation as a broader process, spread throughout time and space. By utilizing this perspective, deliberation may not involve a direct give-and-take of reasons, but may occur through broader discursive clashes. Therefore, to comprehend deliberation, attention must be given to the connections and relationships that exist among several discursive arenas.

Nevertheless, the great majority of methodological procedures still focus either on one arena or, even more problematically, on particular individuals. This type of focus is clear, for instance, in the way several scholars highlight the role of sincerity in understanding deliberation. Proponents of the DQI and Raphäel Kies (2010) note that sincerity is a key component of deliberation. Therefore, they see the inability to measure the sincerity of social actors as a shortcoming in their studies, without realizing that focusing on individuals fosters a restricted view of deliberation. Even LincolnDahlberg (2004), who advocates a broader qualitative approach, states that, "we must not abandon attempts to understand sincerity due to the difficulty of the task" (DAHLBERG, 2004, p. 34). Based on these individualistic premises, such views do not understand deliberation as a public clash of discourses, but as direct form of interaction.

The focus on individuals is also clear in some works grounded on the use of surveys as a method for understanding online deliberation. Questionnaires usually ask individuals if they discuss politics on the web and, if they do, who do they talk to. There is a specific concern about the exposure of individuals to diverse opinions and a fear that subjects who talk to like-minded people may become narrow-minded and anti-democratic. Online deliberation is assumed to happen only when individuals with different opinions communicate with each other.

These types of approaches are in danger of miss-measuring individuals, as John Dryzek (1990) has convincingly discussed. Surveys often assume that beliefs and attitudes are pre-established givens and treat interviewees as research objects, instead of as active political agents who interact with other agents. In addition, surveys frequently adopt an individualized and competitive approach that ignores the criticisms against the philosophy of conscience. Lastly, surveys tend to neglect key developments in deliberative theory that propose a broader understanding of discursive clashes by viewing these debates through the lens of deliberative systems. By this, I do not claim that surveys are useless for deliberative research. Used within mixed method approaches, they can shed light on important topics for deliberative scholars. However, I would recommend extreme care in its use, especially because of the danger fostering an individualized notion of deliberation

Another problematic element in these approaches is the assumption that only contact with opposing perspectives would promote online deliberation (LEV-ON and MANIN, 2009; MUTZ, 2006). Conversations among like-minded individuals are often seen as fostering a form of mobilization, which could hinder deliberation (MUTZ, 2006). Such a view ignores the relevance of conversations among like-minded individuals to increase the chance that some discourses may be expressed publicly, as argued by Mansbridge (1999) and Neblo (2005). If interpreted through systemic lenses, discussions within a group can be essential to deliberation. Different types of discussions, in diverse arenas, at varying moments, offer distinct contributions to deliberative processes.

Another piece of evidence that suggests neglected implications regarding the concept of a deliberative system emerges in the way coding schemes are applied. Mostly, online arenas and initiatives are scrutinized in themselves. Along with the DQI, the frameworks of Stromer-Galley (2007) and Kies (2010) tend to assess interactions within a certain space. Reciprocity is usually restricted to a process of direct interlocution and reason-giving is conceived of being something internal and unique to the specific arena under analysis. The attempt to comprehend these internal relationships often ignores the broader nature of discursive flows. Online deliberation is constrained to a reproduction of face-to-face conversations. The role that information provided by online initiatives plays on a deliberative system is disregarded or even criticized as not fully dialogical. The connections (and disconnections) of arenas, and the discursive routes built online are overlooked. Deliberation is viewed as something to be observed within an initiative or arena, and not across initiatives and arenas.

Specific points in the framework of both Stromer-Galley (2007) and Kies (2010) could be thought of as exceptions in this regard. The former suggests the measurement of sources cited by actors, while the later considers the external impact of arenas in his analyses. These points indicate the relevance of the 'external world' on processes that happen within a given online arena. However, both ideas have limitations. Stromer-Galley's (2007) sourcing, as was already noted, can only assess what is explicitly mentioned and results in missing the idea of uninhibited discursive flows that cannot be properly identified. Kies's (2010) external impact reduces the many possible connections among arenas to one type: influence on the elaboration of political decisions. As a result, neither of the two 'exceptions' is properly equipped to capture the broader idea of deliberative systems. Each idea can grasp some (eventual) connections, but they are not effective when dealing with the idea of structural connections at the grounds of their frameworks.

This is one of the central challenges for current methodological measurements of online deliberation. Understanding the connections, routes and flows among discourses on the web should no longer be thought of as something that can be ignored. If online deliberation is to be understood, these complex processes should be faced properly. Discussions occurring within an online group or forum represent a small fraction of a much more complicated process, that pervades online and offline arenas. The concept of deliberative systems has become essential to obtaining a complete study of deliberation.

The disregard of some specificities of the internet

A third problem with some of the most influential empirical attempts to investigate deliberation on the web is related to a disregard of the nature of online interactions. Some scholars seem so concerned with their attempt to translate the conceptual dimensions of deliberation into empirical categories that they end up missing key aspects of the web.

This type of disregard seems evident in the lack of attention to the different discursive architectures of online arenas. Usually, methodological procedures are conceptualized in a generic way that frequently fails to grasp the particularities of distinct online experiences. The discussion of politics either on a social network community site, a newspaper website, a blog or Facebookgenerates completely different processes. They should not simply be gathered under the umbrella of Type II Deliberation or coded as if they were disembodied discourses. The logics of online discussion vary significantly and methodological procedures have not been able to capture these variations. This is why, according do Dahlberg (2004), several studies of online deliberation make flawed generalizations, not supported by their data.

In addition, interactions that count as a discourse on the web should be amplified if online experiences are to be understood. The role of videos, songs, cartoons, links, images and comments must be conceived of in their specificities and through their intertwinements. It is problematic, for instance, to neglect the centrality of images in Facebook discussions or the role of videos used to respond to other videos on Youtube. However, online deliberation tends to be taken as an asynchronous variation of face-to-face verbal communication. Studies are inclined to focus on forums and communities, measuring the arguments verbally expressed by their members. It is definitely easier to study these interactions; but this procedure may pass over the whole experience of online discussion. One exception here is the recent work of Davies and Chandler (2012), who emphasize the need to comprehend the variety of communicative elements in online interactions and explicitly draw attention to the different modalities discourses may assume.

In this sense, the nature of social ties, the forms of expression, the routes followed by discourses, the regimes of visibility and even the boundaries between public and private are singular in online practices. This is not to say that the internet creates an entirely different world. Nevertheless, there are certain specificities that should be taken into account if web deliberation is to be fully comprehended.

One of these specificities is deeply related to the aspects developed in the previous section. I argue that if deliberation, as such, has much to gain from the idea of deliberative systems, then online experiences cannot be studied without it. The richness of online deliberation lies in the countless dynamic connections that engender new forms of discussion. Either explicitly promoted through linkage, or randomly encouraged through individual practices, the network of networks should not be imagined as a cluster of enclosed arenas. Although it may sound obvious, it is important to emphasize that the idea of a web is essential to the study of this network. However, as many studies focus on the micro-analysis of individual posts within a distinctive arena, the undisciplined discursive flows that surround the specific post are frequently neglected.

An additional specificity is related to the type of engagement that is expected from online deliberators. As opposed to focusing on the discursive process engendered by certain practices and initiatives, studies focus on the energy spent by each participant. These studies often express a feeling of frustration because of a lack of engagement of participants. It is frequently suggested, for instance, that the high levels of one-timers would show the inability of online experiences to foster deliberation. Analogous to this, some scholars seem to expect that users would behave in social networks, online groups and other web arenas in exactly the same way as if they were in conventional meetings. The point I am trying to establish is that most studies of online deliberation seem to lack a sociological understanding of the way in which individuals behave online. Subjects are overburdened with certain expectations that emerge from other interactive structures, a practice which ultimately ignores the dynamics of online experience. In the quest for reciprocal and respectful arguments on the web, many studies simply borrow a pre-established idea of debate. This results in a process that fails to seek out new definitions for public discussion that could better accommodate the idiosyncrasies of the internet.

Concluding remarks

This article has sought to discuss three weaknesses found in prominent methods utilized for the study of online deliberation: (01) the establishment of misleading distinctions; (02) the neglected implications of the concept of a deliberative system; and (03) the disregard of some specificities of the internet. I briefly pointed out that some of the procedures often used for comprehending web discussions have been unable to grasp the nature of online interactions. The focus that these techniques place on micro distinctions has frequently hindered an understanding of the broader picture in which they are inserted.

This does not mean micro-content analysis is, in itself, wrong or misleading. It has been responsible for interesting developments in the areas of both deliberative democracy and internet studies. There are, of course, fruitful findings that help to explain the possibilities of web discussion, thus supporting the work of those responsible for designing online consultations and web forums. Methodological insights also exist that point to new research trends, such as Graham and Witschge's (2003) proposal of re-building argumentative maps or Gerhards and Schäfer's (2010) attempt to study virtual debates through search engines. Therefore, I do not argue that the empirical literature on online deliberation is unproductive.

In my own empirical work on online deliberation, I have attempted to operationalize six criteria (inclusiveness, reason-giving, reciprocity, respect, orientation toward common good and connectivity with other discursive arenas) in ways that combine quantitative and qualitative analyses⁸. It would be beyond the scope of this article to explain how each of these categories was operationalized, but it is important to emphasize how some conceptual moves may lead the analysis in fruitful directions. When discussing reason-giving, for instance, I suggest restricting the quantitative measurement to a variable that simply codes the existence (or inexistence) of justifications, further developing the investigation through a Batesonian-Goffmanian frame analysis, that conceives of frames not as individual strategies but as broader cultural and interactive constructions. This analysis takes into consideration not only words, but also images, memes and links mobilized by posts and comments. Another criterion that deserves attention is reciprocity. My coauthors and I have attempted to distinguish a direct type of reciprocity, usually measured by empirical studies, from a discursive form of reciprocity accessible through frame analysis and more in tune with the systemic approach to deliberation (MENDONÇA, FREITAS and OLIVEIRA, 2014). In addition, my investigations have benefited from the concept ofaffordances, frequently used in technology studies, which paved interesting routes for context-sensitive analyses.

I am not, therefore, skeptical about the possibility of empirically assessing online deliberation and, as a matter of fact, measuring some of its dimensions. I am against excessive micro-quantification, focused on individuals and on arenas (considered as self-enclosed) and not sensitive to the contexts of online interaction. My argument is simply that key weaknesses permeate most of the empirical studies. Such studies would greatly benefit from a more complex view, which does not mean the establishment of numerous detailed categories to capture the minutiae of individual discursive constructions. The core idea of deliberation and the nature of the online experience must be kept in mind. By doing so, the concept of a deliberative system can contribute a great deal because of its emphasis on the reticular character of human interaction. A deliberative system helps to create an understanding of the complexities and specificities of web deliberation, thus generating new routes for empirical studies.

References

• AVRITZER, Leonardo (2009), Participatory Institutions in Democratic Brazil Baltimore: Johns Hopkins University Press. 205pp.
• AVRITZER, Leonardo (2006), New Public Spheres in Brazil: Local Democracy and Deliberative Politics. International Journal of Urban and Regional Research Vol. 30, N° 03, pp. 623–637.
• BÄCHTIGER, André; SHIKANO, Susumu; PEDRINI, Seraina and RYSER Mirjam (2009), Measuring Deliberation 2.0: Standards, Discourse Types, and Sequenzialization. ECPR General Conference, Potsdam. Retrieved from: http://ash.harvard.edu/extension/ash/docs/baechtiger.pdf. Accessed in August 2013.
  » http://ash.harvard.edu/extension/ash/docs/baechtiger.pdf
• BENHABIB, Seyla (1996), Toward a deliberative model of democratic legitimacy. In: Democracy and difference: contesting the boundaries of the political. Edited by BENHABIB, Seyla. Princeton: Princeton University Press. pp. 67-94.
• BENNETT, W. Lance; PICKARD, Victor; IOZZI, David; SCHROEDER, Carl; LAGOS, Taso and CASWELL, C. Evans (2004), Managing the public sphere: journalistic construction of the great globalization debate. Journal of Communication Vol. 54, N° 03, pp. 437-455.
• BLACK, L. W.; BURKHALTER, S.; GASTIL, J. and STROMER-GALLEY, J. (2009), Methods for Analyzing and Measuring Group Deliberation. In:Sourcebook of Political Communication Research: methods, measures, and analytical techniques. Edited by BUCY, E. and HOLBERT, L. New York: Routledge. pp. 323-344.
• BOHMAN, J. (2007), Democracy across borders:from Dêmos to Dêmoi Cambridge: MIT Press. 232pp.
• BOHMAN, J. (2004), Expanding dialogue: the Internet, the public sphere and prospects for transnational democracy. The Sociological Review Vol. 52, N° 01, pp. 131-155.
• BOHMAN, J. (1998), The coming age of deliberative democracy.The Journal of Political Philosophy Vol. 06, N° 04, pp. 400-425.
• BOHMAN, J. (1996), Public Deliberation: pluralism, complexity and democracy. Cambridge: MIT Press. 320pp.
• CHAMBERS, S. (2009), Rhetoric and the Public Sphere: has deliberative democracy abandoned mass democracy? Political Theory Vol. 37, N° 03, pp. 323-350.
• CHAMBERS, S. (2003), Deliberative Democratic Theory. Annual Review of Political Science Vol. 06, pp. 307–326.
• CHARLES, M.; SOKOLOFF, H. and SATULLO, C. (2005), Electoral deliberation and public journalism. In: The Deliberative Democracy Handbook: strategies for effective civic engagement in the twenty-first century. Edited by GASTIL, J. J. and LEVINE, P. San Francisco: Jossey-Bass. pp. 59-67.
• COLEMAN, S. and MOSS, G. (2012), Under construction: the field of online deliberation research. Journal of Information Technology and Politics Vol. 09, N° 01, pp. 01–15.
• COLEMAN, S. and SHANE, P. (2012), Connecting democracy: "online" consultation and the flow of political communication.Cambridge, Mass., London: MIT Press. 423pp.
• CORNWALL, A. and COELHO, V.S.R.P. (2007), Spaces for Change? The politics of citizen participation in new democratic arenas.London: Zed Books. 270pp.
• DAHLBERG, L. (2004), Net-public sphere research: beyond the "first phase". Javnost Vol. 01, pp. 27-44.
• DAHLBERG, L. (2001), The internet and democratic discourse: exploring the prospects of online deliberative forums extending the public sphere. Information, Communication and Society Vol. 04, N° 04, pp. 615–633.
• DAVIES, T. (2009), Introduction. In: Online Deliberation: Design, Research and Practice Edited by DAVIES, T. and GANGADHARAN, S. Chicago: Center for the Study of Language and information. pp. 01-19.
• DAVIES, T. and GANGADHARAN, S. (Eds) (2009), Online Deliberation: Design, Research and Practice Chicago: Center for the Study of Language and Information. 374pp.
• DAVIES, T. and CHANDLER, R. (2012), Online deliberation design: choices, criteria, and evidence. In: Democracy in motion:evaluating the practice and impact of deliberative civic engagement. Edited by NABATCHI, T.; GASTIL, J.; WEIKSNER, G. M. and LEIGHNINGER, M. New York: Oxford University Press. pp. 103-132.
• DELI CARPINI, M. X.; COOK, F. L. and JACOBS, L. (2004), Public deliberation, discursive participation, and citizen engagement: a review of the empirical literature. Annual Review of Political Science Vol. 07, pp. 315–344.
• DRYZEK, John S. (2012), Foundations and Frontiers of Deliberative Governance. Oxford: Oxford University Press. 256p.
• DRYZEK, J.S. (2008), The empirical turn in deliberative democracy.Paper presented at the Theory and practice of deliberative democracy workshop. Australian National University, Canberra. 07-08 Feb 2008.
• DRYZEK, J. S. (2007), Theory, Evidence, and the Tasks of Deliberation. In: Deliberation, participation and democracy: can the people govern? Edited by ROSENBERG, S. New York: Palgrave MacMillan. pp. 237-250.
• DRYZEK, J. S. (2006), Deliberative Global Politics:discourse and democracy in a divided world. Cambridge: Polity Press. 191pp.
• DRYZEK, J. S. (2000), Deliberative democracy and beyond: liberals, critics, contestations. New York: Oxford University Press. 191pp.
• DRYZEK, J. S. (1990), Discursive democracy: politics, policy, and political science. New York/Oakleigh: Cambridge University Press. 249pp.
• DRYZEK, J. S. and NIEMEYER, S. (2006), Reconciling pluralism and consensus as political ideals. American Journal of Political Science Vol. 50, N° 03, pp. 634-649.
• ELSTER, J. (ed). (1998), Deliberative democracy.Cambridge: Cambridge University Press. 296pp.
• ERIKSEN, E. O. (2000), The European Union's democratic deficit: a deliberative perspective. In: Democratic innovation:deliberation, representation and association. Edited by SAWARD, M. Londres: Routledge. pp. 53-65.
• ETTEMA, J. (2007), Journalism as reason-giving: deliberative democracy, institutional accountability, and the News Media's Mission.Political Communication Vol. 24, N° 02, pp. 143-160.
• FISHKIN, J. (2009), When the People Speak:deliberative democracy and public consultation. Oxford: Oxford University Press. 256pp.
• FISHKIN, J. and LASLETT, P. (eds.) (2003), Debating deliberative democracy Malden: Blackwell. 225pp.
• FRESCHI, A. C. and METE, V. (2009), The political meanings of institutional deliberative experiments: findings on the Italian case.Sociologica Vol. 02, Nº 03, pp. 01-55.
• FUNG, A. (2003), Recipes for Public Sphere: eight institutional design choices and their consequences. The Journal of Political Philosophy Vol. 11, N° 03, pp. 338-367.
• FUNG, A. and WRIGHT, E. O. (eds.) (2003), Deepening Democracy. Londres/NY: Verso. 305pp.
• GASTIL, J. and LEVINE, P. (Eds) (2005), The Deliberative Democracy Handbook: strategies for effective civic engagement in the twenty-first century. San Francisco: Jossey-Bass. 336pp.
• GERHARDS, J. and SCHÄFER, M. S. (2010), Is the internet a better public sphere? Comparing old and new media in the USA and Germany. New Media and Society Vol. 12, N° 01, pp. 143–160.
• GIMMLER, A. (2001), Deliberative democracy, the public sphere and the internet. Philosophy and Social Criticism Vol. 27, N° 04, pp. 21-39.
• GOODIN, R. (2008), Innovating Democracy: democratic theory and practice after the deliberative turn. Cambridge: Cambridge University Press. 326pp.
• GRAHAM, T. and WITSCHGE, T. (2003), Search of online deliberation: Towards a new method for examining the quality of online discussions.Communications Vol. 28, N° 02, pp. 173-204.
• GUTMANN, A. and THOMPSON, D. (2004), Why deliberative democracy? Princeton/Oxford: Princeton University Press. 232p.
• GUTMANN, A. and THOMPSON, D. (1996), Democracy and Disagreement. Cambridge/London: The Belknap Press of Harvard University Press. 432p.
• HABERMAS, J. (2006), Political Communication in Media Society: does democracy still enjoy an epistemic Dimension? The impact of normative theory on empirical research. Communication Theory Vol. 16, N° 04, pp. 411-426.
• HABERMAS, J. (2005), Concluding comments on empirical approaches to deliberative politics. Acta Politica, International Journal of Political Science Vo. 40, N° 03, pp. 384-392.
• HABERMAS, J. (1996), Between Facts and Norms: contributions to a discourse theory of law and democracy. Cambridge: MIT Press. 675pp.
• HABERMAS, J. (1987), The Theory of Communicative Action Lifeworld and system: a critique of functionalist reason. Boston: Beacon Press. 457p. Volume 2.
• HAMLETT, P. (2002), Adapting the Internet to Citizen Deliberations: Lessons Learned. In: Proceedings from International Symposium on Technology and Society Raleigh, USA, June 06-08, pp. 213-218.
• HENDRIKS, C. M. (2011), The Politics of Public Deliberation:Citizen engagement and interest advocacy. London: Palgrave Mcmillan. 272pp.
• HENDRIKS, C. M. (2006), Integrated Deliberation: reconciling civil society's dual role in deliberative democracy. Political Studies Vol. 54, N° 03, pp. 486–508.
• JAMES, M. R. (2004), Deliberative democracy and the plural polity Lawrence: University Press of Kansas. 252p.
• JANSSEN, D. and KIES, R. (2005), Online Forums and Deliberative Democracy. Acta Politica Vol. 40, pp. 317–335.
• JENSEN, J. L. (2003), Public spheres on the internet: anarchic or government-sponsored: a comparison. Scandinavian Political Studies Vol. 26, N° 04, pp. 349-374.
• JOSS, S. and DURANT, J. (1995), Public Participation in Science: the role of consensus conferences in Europe. London: Science Museum. 144p.
• KANRA, B. (2009), Islam, democracy, and dialogue in Turkey: deliberating in divided societies. Farnham: Ashgate. 180pp.
• KELLY, K. (2008), Participation and depth of deliberation as components in measuring quality of deliberation. Paperpresented at the Theory and practice of deliberative democracy workshop. Canberra: Australian National University. 07-08 Feb 2008.
• KIES, R. (2010), Promises and limits of web-deliberation. New York: Palgrave Macmillan. 200p.
• LEV-ON, A. and MANIN, B. (2009), Happy accidents: deliberation and online exposure to opposing views. In: Online Deliberation:design, research and practice. Edited by DAVIES, T. and GANGADHARAN, S. Chicago: Center for the Study of Language and information. pp. 105-122.
• LUPIA, A. (2009), Can online deliberation improve politics? Scientific foundations for success. In: Online Deliberation:design, research and practice. Edited by DAVIES, T. and GANGADHARAN, S. Chicago: Center for the Study of Language and information. pp. 59-69.
• MACEDO, S. (ed.) (1999), Deliberative Politics: essays on democracy and disagreement. Nova York/Oxford: Oxford University Press. 289p.
• MAIA, R. (ed.) (2012), Deliberation, the media and political talk New York: Hampton Press. 371pp.
• MANSBRIDGE, J. (1999), Everyday Talk in Deliberative System. In:Deliberative Politics: essays on democracy and disagreement. Edited by MACEDO, S. New York: Oxford University Press. pp. 211-239.
• MANSBRIDGE, J.; BOHMAN, J.; CHAMBERS, S.; ESTLUND, D.; FOLLESDAL, A.; FUNG, A.; LAFONT, C.; MANIN, B. and MARTÍ, J. L. (2010), The place of self-interest and the role of power in deliberative democracy. Journal of Political Philosophy Vol. 18, N° 01, pp. 64-100.
• MARQUES, A. C. S. (2011), Aspectos teórico-metodológicos do processo comunicativo de deliberação online. Revista Brasileira de Ciência Política Vol. 06, pp. 19-40.
• MENDONÇA, R. F. and SARMENTO, R. (forthcoming), Disrespect in online deliberation: inducing factors and democratic potentials. Revista de Ciencia Política
• MENDONÇA, R. F. and AMARAL, Ernesto Friedrich de Lima (2014), Deliberação online em consultas públicas? O caso da assembleia legislativa de Minas Gerais. Revista de Sociologia e Política Vol. 22, Nº 49, p. 177-203.
• MENDONÇA, R. F.; FREITAS, F. V. and OLIVEIRA, W. M. (2014), Reciprocidade discursiva, enquadramento e deliberação: a consulta pública sobre reforma política da ALMG. Análise Social Vol. 211, pp. 244-271.
• MENDONÇA, R. F. (2013a), Teoria Crítica e democracia deliberativa: diálogos instáveis. Opinião Pública Vol. 19, Nº 01, pp. 49-64.
• MENDONÇA, R. F. (2013b), The conditions and dilemmas of deliberative systems. Paper presented at the APSA Annual Meeting, 2013, Chicago. Social Science Research Network. pp. 01-31.
• MENDONÇA, R. F. and PEREIRA, M. A. (2012), Democracia digital e deliberação online: um estudo de caso sobre o VotenaWeb. Revista Latinoamericana de Opinión Pública Vol. 02, pp. 109-158.
• MENDONÇA, R. F. (2011), Reconhecimento e (qual?) deliberação.Opinião Pública Vol. 17, Nº 01, pp. 206-227.
• MENDONÇA, R. F. and SANTOS, D. B. (2009), A cooperação na deliberação pública: um estudo de caso sobre o referendo acerca da proibição da comercialização de armas de fogo no Brasil. Dados Vol. 52, Nº 02, pp. 507-542.
• MIGUEL, L. F. (2014), Deliberacionismo e os limites da crítica: uma resposta. Opinião Pública Vol. 20, pp. 118-131.
• MOUFFE, C. (2005), Por um modelo agonístico de democracia.Revista de Sociologia e Política Nº 25, pp. 11-23.
• MUTZ, D. (2006), Hearing the other side: deliberative versus participatory democracy. Cambridge/New York: Cambridge University Press. 184pp.
• NEBLO, M. (2005), Thinking through Democracy: between theory and practice of deliberative politics. Acta Politica Vol. 40, pp. 169-81.
• OWEN, D. and SMITH, G. (2015), Deliberation, democracy, and the systemic turn. Journal of Political Philosophy Vol. 23, Nº 02, pp. 213–234.
• PARKINSON, J. (2006), Deliberating in the Real World:problems of legitimacy in deliberative democracy. Oxford: Oxford University Press. 209pp.
• PARKINSON, J. and MANSBRIDGE, J. (eds.) (2012), Deliberative Systems Cambridge: Cambridge University Press. 204pp.
• PEDRINI, S. (2012), Deliberative Capacity in the Political and Civic Sphere. Paper presented at the Conference on Deliberative Democracy in Action, Turku/Åbo, Finland. 05-07 June 2012.
• PRICE, V. (2009), Citizens deliberating online: theory and some evidence. In: Online Deliberation: design, research and practice. Edited by DAVIES, Todd and GANGADHARAN, Seeta. Chicago: Center for the Study of Language and information. pp. 37-58.
• RANCIÈRE, Jacques (2001), Ten Theses on Politics. Theory and Event Vol. 05, Nº 03. pp. 1-16.
• ROSENBERG, S. (2007), Deliberation, participation and democracy: can the people govern? New York: Palgrave MacMillan. 352pp.
• SÆBØ, Ø.; ROSE, J. and MOLKA-DANIELSEN, J. (2009), E-Participation: designing and managing political discussion forums. Social Science Computer Review Vol. 28, N° 04, pp. 403-426.
• SAMPAIO, R. C.; MAIA, R. C. M. and MARQUES F. P. J. A. (2011), Participation and deliberation on the internet: a case study on digital participatory budgeting in Belo Horizonte. The Journal of Community Informatics Vol. 07, pp. 01-22.
• SANDERS, L. (1997), Against Deliberation. Political Tinstitutions for citizen participation. Cambridge: Cambridge University Press. 232 pp.heory Vol. 25, Nº 03, pp. 347-376.
• SHANE, P. (2009), Turning GOLD into EPG: lessons from low-tech democratic experimentalism for electronic rulemaking and other ventures in cyberdemocracy. In: Online Deliberation: design, research and practice. Edited by DAVIES, Todd and GANGADHARAN, Seeta. Chicago: Center for the Study of Language and information. pp. 149-162.
• SHAPIRO, Ian (1999), Enough of deliberation: politics is about interest and power. In: Deliberative Politics: essays on democracy and disagreement. Edited by MACEDO, S. New York/Oxford: Oxford University Press. pp. 28-38.
• SILVERSTONE, R. (2002), Por que estudar a mídia?São Paulo: Loyola. 302pp.
• SIMON, W. H. (1999), Three limitations of deliberative democracy: identity politics, bad faith, and indeterminacy. In: Deliberative Politics: essays on democracy and disagreement. Edited by MACEDO, S. New York/Oxford: Oxford University Press. pp 49-57.
• SINTOMER, Y. (2010), O poder ao povo: júris de cidadãos, sorteio e democracia participativa. Belo Horizonte: Editora UFMG. 230p.
• SMITH, G. (2009), Democratic innovations:designing
• SMITH, G. (2005), Beyond the Ballot 57 democratic innovations from around the world. Report prepared for the Power Inquiry, UK. 133pp.
• STEENBERGEN, M. R.; BÄCHTIGER, A.; SPÖRNDLI, M. and STEINER, J. (2003), Measuring deliberation: a discourse quality index. Comparative European Politics Vol. 01, N° 01, pp. 21-48.
• STEINER, J.; BÄCHTIGER, A.; SPÖRNDLI, M. and STEENBERGEN, M. (2004),Deliberative Politics in Action: analysing parliamentary discourse. Cambridge: Cambridge University Press. 206p.
• STROMER-GALLEY, J. (2007), Measuring deliberation's content: a coding scheme. Journal of Public Deliberation Vol. 03, N° 01, pp. 01-35.
• THOMPSON, D. (2008), Deliberative democratic theory and empirical political science. Annual Review of Political Science Vol. 11, pp. 497-520.
• WALES, C.; COTTERILL, S. and SMITH, G. (2010) Do citizens 'deliberate' in on-line discussion forums? Preliminary findings from an internet experiment. Paper presented at the Participatory and Democracy Specialist Group at the Political Studies Association Conference. Edinburgh: Scotland.
• WESSLER, H. (2008), Investigating deliberativeness comparatively.Political Communication Vol. 25, N° 01, pp. 01-22.
• WILHELM, A. G. (2000), Democracy in the digital age New York: Routledge. 184pp.
• WOJCIESZAK, M. and MUTZ, D. C. (2009), Online groups and political discourse: do online discussion spaces facilitate exposure to political disagreement? Journal of Communication Vol. 59, N° 01, pp. 40-56.
• WRIGHT, S. and STREET, J. (2007), Democracy, deliberation and design: the case of online discussion forums, New Media and Society Vol. 09, N° 05, pp. 849-869.
• YOUNG, I. (1996), Communication and the other: beyond deliberative democracy. In: Democracy and difference: contesting the boundaries of the political. Edited by BEBHABIB, S. Princeton: Princeton University Press. pp. 120-135.
• YOUNG, I. (2000), Inclusion and Democracy. Oxford: Oxford University Press. 320pp.
• YOUNG, I. (2003), Activist Challenges to deliberative democracy. In:Debating deliberative democracy Edited by FISHKIN, J. and LASLETT, P. Malden: Blackwell. pp. 102-120.

• 1
  For some examples, see Avritzer (2006; 2009); Cornwall and Coelho (2007); Fung (2003); Fung and Wright (2003); Gastil and Levine (2005); Hendriks (2011); Joss and Durant (1995); Sintomer (2010) and Smith (2005, 2009).
• 2
  For some examples, see Bennett et al. (2004); Charles et al. (2005);Ettema (2007); Maia (2012) and Wessler (2008).
• 3
  See, for instance, Dryzek, (2000); Eslter (1998); Gutmann and Thompson (2004);Macedo (1999); Mansbridge et al. (2010); Mendonça (2011, 2013a); Miguel (2014); Mouffe (2005); Mutz (2006); Owen and Smith (2015);Rancière (2001); Sanders (1997); Simon (1999) and Young (1996, 2003).
• 4
  For some examples, see Bächtiger et al. (2009); Bohman (2004); Coleman and Moss (2012); Davies and Grangadharan (2009); Graham and Witschge (2003); Gimmler (2001); Janssen and Kies (2005); Kies (2010); Marques (2011); Mutz (2006); Pedrini, (2012); Sampaio et al. (2011); Wales et al. (2010); Wilhelm (2000); Wojcieszak and Mutz (2009) and Mendonça and Pereira (2012).
• 5
  For some key contributions, see Benhabib (1996); Bohman (1996, 1998, 2007); Chambers (2003, 2009); Dryzek (1990, 2000, 2006, 2012); Fishkin and Laslett (2003); Gastil and Levine (2005); Gutmann and Thompson (1996, 2004); Habermas (1996, 2005, 2006); Mansbridge (1999); Mansbridge et al. (2010); Parkinson and Mansbridge (2012); Rosenberg (2007); Steiner et al. (2004); Thompson (2008) andYoung (2000).
• 6
  See Bächtiger et al. (2009); Dryzek (2012); Goodin, (2008); Hendriks (2006, 2011); Mansbridge (1999); Parkinson (2006); Parkinson and Mansbridge (2012); Mendonça (2013b).
• 7
  See Parkinson and Mansbridge (2012).
• 8
  Mendonça e Amaral (2014), Mendonça, Freitas and Oliveira (2014), Mendonça and Sarmento (2015).
• (*)
  http://dx.doi.org/10.1590/1981-38212015000300021
• I am thankful to John Dryzek, Wilson Gomes, Rayza Sarmento, Rafael Sampaio and Samuel Barros for their comments on this article. This work was supported by Fapemig (Processes SHA/APQ-00544/11 and CSA-PPM-00211/13), by PRPq, Universidade Federal de Minas Gerais, Call 12/2011, and by CNPq as a Fellowship on Productivity, Process 305117/2014-9. A first version was presented in the International Seminar "Technology and Democracy", held at the Universidade Federal de Minas Gerais, on 30 Oct –1 Nov, 2013.
• Revised by Cabo Verde

Publication Dates

• Publication in this collection
  Dec 2015

History

• Received
  Dec 2014
• Accepted
  July 2015