Assessing some measures of online deliberation

The empirical turn in deliberative democracy has fostered the development of different methodological procedures. Within this literature, studies focusing on the internet have gained increasing attention. The belief that the internet may help solve some of the deliberative deficits of democracies has propelled an interest in the potential benefits and problems of online discourse. This article seeks to discuss some of the methods that have been advocated for the study of online deliberation to point out three of their weaknesses: (01) the establishment of misleading distinctions; (02) the neglect of the implications of the deliberative system; and (03) the disregard of some specificities of the internet.

he empirical turn in deliberative democracy has brought a renewed vitality to this field of research.After a decade of fruitful philosophical development, the 21st century witnessed a growing interest in rethinking conceptual frameworks through empirical inquiry, thus pushing the frontiers of deliberative theory in new directions.Habermas (2005) has endorsed this move, and Kies (2010) attributes this validation to the lack of empirical evidence on the grounds of a highly abstract perspective.
It is therefore not a surprise that Habermas has very recently strongly appreciated and encouraged the efforts accomplished by researchers from around the world to operationalize and test the criteria and presuppositions of the deliberative democratic model in different contexts of discursive interaction (KIES, 2010, p. 34).
This turn has fostered the development of methods applicable to different types of discursive arenas aimed at tackling diverse problems (BLACK et al., 2009;DRYZEK, 2008).Most studies seek to assess either the deliberativeness of specific types of interaction (KELLY, 2008;STEINER et al., 2004;WESSLER, 2008) or the effects of these processes on citizens, decision making and society in the broad sense (DELI CARPINI et al., 2004).There are investigations devoted to tracking preference change (DRYZEK and NIEMEYER, 2006), the exposure to opposite perspectives (LEV-ON and MANIN, 2009;MUTZ, 2006), and processes of social learning provoked by deliberation (KANRA, 2009).There is also significant interest in the design of participatory experiments 1 and the role of the media 2 .Within this profuse literature, studies focusing on the internet have been gaining attention.The belief that the internet may help solve some of the deliberative deficits of democracies has fueled an interest in the potential benefits and problems of online discourse.The aim of this article is to present some of the methods that have been advocated for the study of online deliberation to point out some of their weaknesses.An element of these weaknesses emerges from problems in the empirical literature on deliberative democracy in general, while other issues are specific to texts related to the online phenomena.
This article begins with a very brief review of online deliberation, followed by the presentation of some analytical approaches utilized to study this topic.The following sections provide a discussion of three weaknesses of the previous approaches to online deliberation: (01) the establishment of misleading distinctions, (02) the neglect of the implications of the deliberative system, and (03) the disregard of some specificities of the internet.It must be clear that it is not my aim, in this article, to advocate deliberative democracy against its critics or to deal with the many relevant criticisms raised against this democratic perspective.There is an extensive literature covering this debate 3 .My goal is to foster the advancement of a debate within the deliberative approach, contributing to the development of this literature in its own grounds.

Online deliberation and its measures
Online deliberation is one of the main areas of interest among the most innovative research on deliberative democracy (DAVIES, 2009).Following the excitement evident in studies from the early 1990s that anticipated the emergence of a new public sphere on the internet, and on a more critical perspective, several scholars have tried to understand online practices by examining them through the lens of deliberation 4 .
over the last two decades, becoming one of the most productive areas on political theory (DRYZEK, 2007)5 .
The most recent literature addressing the deliberative approach has highlighted the fact that deliberation should be viewed as a process resulting from the overlap of several arenas and discursive moments6 .These studies have also emphasized that the give-and-take of reasons that serve as the foundation of deliberation occur through a variety of communicative formats, including the presentation of ideas in an emotional way.In addition, deliberation does not require that participants become altruistic beings without particular preferences (MANSBRIDGE et al., 2010;MENDONÇA and SANTOS, 2009).The deliberative process only requires a public clash of discourses that induces reflection in a noncoercive way and promotes a connection between particular experiences and more general principles (DRYZEK, 2006, p. 52).
Online forums may function as arenas that play a role in broader discursive processes, thus nurturing public deliberation (COLEMAN and MOSS, 2012).Far from compromising the benefits of face-to-face group meetings, computer mediated communication may prove especially useful for deliberative work (PRICE, 2009, p. 37).Raphaël Kies (2010) also argues that there is no original contradiction between the internet and deliberation, although some scholars claim that the former can only foster frivolous and empty interactions.Despite his criticism of several approaches to online deliberation, Arthur Lupia (2009) is another scholar who admits its potential: "Online deliberation […] is promising because of its ability to bring people together for the purpose of information exchange without the difficulties caused by physical distances between participants" (LUPIA, 2009, p.60).
In the search for online possibilities regarding deliberation, deliberative democrats have conducted a wide range of investigations.Among the pioneering works in this field are Wilhelm's (2000) investigation about Usenet in the United An increase in the number of studies about online deliberation has led to an emergence of analyses with varying focuses.There are studies about: the design of forums (DAVIES and CHANDLER, 2012;SAEBØ et al., 2009;WRIGHT and STREET, 2007); the deliberativeness of online arenas (HAMLETT, 2002;JANSSEN and KIES, 2005;KIES, 2010;SAMPAIO et al., 2011;STROMER-GALLEY, 2007;WALES et al., 2010); the comparison between online and conventional media spheres (GERHARDS and SCHÄFER, 2010); the role of the internet in promoting contact between opposing perspectives (LEV- ON and MANIN, 2009;MUTZ, 2006;WOJCIESZAK and MUTZ, 2009); the use of online consultations (COLEMAN and SHANE, 2012;DAVIES and CHANDLER, 2012;FISHKIN, 2009;KIES, 2010;SHANE, 2009); and the potential impact resulting from these processes (FRESCHI and METE, 2009).These investigations have offered rich methodological approaches, which vary greatly depending on the type of research problem being addressed.In this article, it would be impossible to provide fair coverage of each of these routes.I will therefore focus on some of the most influential analytical frameworks.However, throughout the discussion I will make quick references to specific aspects of other methodological proposals that will not be featured here.
The first approach I wish to focus on is the Discourse Quality Index (DQI), proposed by Steiner et al. (2004) for the study of parliamentary deliberations.
Following its establishment, this technique was developed further, and has been applied to the comprehension of other discursive spheres (BÄCHTIGER et al., 2009;PEDRINI, 2012).Praised by Habermas (2005) as 'splendid', and increasingly used in empirical studies, the DQI has become the most renowned method for the microanalysis of deliberation, and is one of the most influential approaches for scholars of online deliberation (KIES, 2010).Such influence should not be seen as a misuse of the DQI, because it has been advocated by its proponents: "the DQI can be applied easily and reliably to a wide range of deliberative contexts" (STEENBERGEN et al., 2003, p. 22).
The  Steenbergen et al. (2003, pp. 27-30) After the DQI's initial formulation, the approach has been expanded upon and updated.Bächtiger et al. (2009)  Kies's (2010) model presents some important differences in relation to the two frameworks that were previously mentioned.One important distinction is the consideration of elements that point to the external impact of the forum.The forum is not investigated as a contained environment.A second difference, which is related to the first, addresses the use of surveys and interviews in addition to content analysis.These methods help to promote a more complex picture of processes.
Surveys are also at the heart of methodological procedures adopted by some studies seeking to investigate the exposure of internet users to other perspectives.Wojcieszak and Mutz (2009) used a sample of 1,028 respondents from a large national survey in the United States to investigate if, how and when Americans discuss politics online.The main goal of their survey was to observe the extent to which political talk arising in various types of online groups serve to expose participants to like-minded views as opposed to challenging them via exposure to disagreement (WOJCIESZAK and MUTZ, 2009, p. 43).Participants who (2015) 9 (3) 88 -115 had been engaged in online political discussions were asked whether they usually agreed or disagreed with the other participants' views.This research was developed utilizing some aspects initially addressed by Diana Mutz (2006).She attempted to study real world informal conversations through "several representative national surveys that included information on Americans' networks of political discussion" (MUTZ, 2006, p. 21).The conclusions of this study suggest "cross-cutting exposure is negatively related to participation" (MUTZ, 2006, p. 112).
Although a discussion of the results of these investigations would be valuable, this article will focus on some of the weaknesses of the methods applied.
In doing so, my analysis obviously points to the potential shortcomings in the findings that were reached.Each of the following sections focuses on presenting one of such weakness.

The establishment of misleading distinctions
Micro approaches to online deliberation seem fascinated by detailed coding schemes that often lead to classifications, which do not deepen our knowledge of the topic.Excessive quantification directs investigations to fallacies that lack theoretical grounds.Within the obsessive exploration of exhaustive analytical matrices, the purpose of many distinctions is not only unclear, but also misleading.This often puts the broader comprehension of the process in jeopardy.Dahlberg (2004) has made a similar argument, when he claims that: The fundamental problem is that operationalisation requires researchers to focus upon those aspects of the public sphere for which narrowly defined and measurable indicators can be found, thus neglecting other aspects less amenable to quantification.The result is serious loss of meaning (DAHLBERG, 2004, p. 31).
My argument is that, besides the problem of excessive quantification (with its focus on 'measurable indicators'), there is the problem of establishing distinctions that may prove misleading.Many scholars are devoted to demanding classifications that may hinder the interpretation of deliberation as a political process.
This problem becomes apparent, for instance, within the levels of typology regarding justification that are suggested by the DQI.The categorization of inferior, Several other criteria proposed by the DQI suffer from similar problems.
The criterion of Mutual Respect, for instance, establishes two misleading hierarchies.
The first distinguishes implicit respect from explicit respect, using ordinal variables.
However, the proponents of the model never explain why explicit demonstrations of respect are preferable to implicit manifestations.The second concerns the respect given to counterarguments.Once again, ordinal variables are used to grade different types of behavior asymmetrically.Applying a value to someone's argument is considered better for deliberation than treating it neutrally.Agreeing with a counterargument is even better, according to the DQI.An approach such as this neglects the agonistic dimension of deliberation.Deliberation requires taking other positions into consideration, but not necessarily agreeing with them.A discussion consisting of numerous agreements may be significantly poorer than one in which arguments are treated neutrally.
In the first case, it presumes that particular interests and the common good can be easily distinguished and are opposed to each other.The DQI neglects the fact that these dimensions may often go hand-in-hand or be intertwined.In addition, the criterion creates a distinction between two types of common good which seems unwarranted.Why should the notion of the common good be restricted to utilitarian terms and the Rawlsian principle of difference?And why is it important to distinguish types of common good?What does this categorization explain about deliberation?
Lastly, the category Constructive Politics values mediating proposals more than the elaboration of alternative proposals.Consensus appeals receive an even higher grade than mediating proposals.Once again, this type of hierarchy is based on a questionable restrictive conception of deliberation that assumes the middle way is always the best route.Why are alternative proposals not more deliberative than mediating arguments?What type of consensus is implicit in this coding scheme?Are workable agreements (DRYZEK, 2000;ERIKSEN, 2000;JAMES, 2004) included in the coding?Are the authors talking about the often criticized idea of If the DQI establishes some misleading distinctions, it should not be seen as the only method for doing so.Stromer-Galley's (2007) proposal also suffers from this type of problem.To begin with, she sets up a very fragmented division of the units of discourse that does not seem very helpful at a more aggregated level.The distinction of units (which are coded as problem, metatalk, process or social) breaks up the discursive process into unnecessary fragments.These fragments can also be misleading.For instance, the establishment or hindrance of social bonds happens in many ways and not exclusively through what she outlines as 'social'.
However, an even more problematic issue unfolds in her most celebrated criterion, sourcing.Stromer-Galley ( 2007 discusses.The mass media, for instance, should not be reduced to a channel of information, but sits at the heart of contemporary personal experiences (SILVERSTONE, 2002).Thirdly, and more importantly, there is a problematic assumption that pervades her analysis.This theory implies that personal experiences are somehow poorer than other types of sources.She expected to find more references to the briefing documents, and seemed frustrated by the predominant use of personal experiences.
This type of assumption also manifests in another renowned procedure for the study of online deliberation.Jensen (2003)  to other sources is valued as an indicator of the qualifications of participants.
However, such a view shows how the methods applied may underestimate the capacity of story-telling to present reasons in a publicly comprehensible way.
Personal experiences should not be seen as a less informed way to foster one's position.This type of hierarchy devalues personal experiences and neglects the essential grounds of deliberative theory, which is a technique that anticipates varying contributions within a discursive process.

Neglected implications of the concept of a deliberative system
Despite the broad theoretical acceptance of the concept of a deliberative system 7 , most empirical studies addressing online deliberation still neglect its implications.The notion of a deliberative system advances the understanding of deliberation as a broader process, spread throughout time and space.By utilizing this perspective, deliberation may not involve a direct give-and-take of reasons, but may occur through broader discursive clashes.Therefore, to comprehend deliberation, attention must be given to the connections and relationships that exist among several discursive arenas.
Nevertheless, the great majority of methodological procedures still focus either on one arena or, even more problematically, on particular individuals.This type of focus is clear, for instance, in the way several scholars highlight the role of sincerity in understanding deliberation.Proponents of the DQI and Raphäel Kies (2010) note that sincerity is a key component of deliberation.Therefore, they see the inability to measure the sincerity of social actors as a shortcoming in their studies, without realizing that focusing on individuals fosters a restricted view of deliberation.Even Lincoln Dahlberg (2004), who advocates a broader qualitative approach, states that, "we must not abandon attempts to understand sincerity due to the difficulty of the task" (DAHLBERG, 2004, p. 34).Based on these individualistic premises, such views do not understand deliberation as a public clash of discourses, but as direct form of interaction.
The focus on individuals is also clear in some works grounded on the use of surveys as a method for understanding online deliberation.Questionnaires usually (2015) 9 (3) 88 -115 ask individuals if they discuss politics on the web and, if they do, who do they talk to.There is a specific concern about the exposure of individuals to diverse opinions and a fear that subjects who talk to like-minded people may become narrow-minded and anti-democratic.Online deliberation is assumed to happen only when individuals with different opinions communicate with each other.
These types of approaches are in danger of miss-measuring individuals, as John Dryzek (1990) has convincingly discussed.Surveys often assume that beliefs and attitudes are pre-established givens and treat interviewees as research objects, instead of as active political agents who interact with other agents.In addition, surveys frequently adopt an individualized and competitive approach that ignores the criticisms against the philosophy of conscience.Lastly, surveys tend to neglect key developments in deliberative theory that propose a broader understanding of discursive clashes by viewing these debates through the lens of deliberative systems.By this, I do not claim that surveys are useless for deliberative research.
Used within mixed method approaches, they can shed light on important topics for deliberative scholars.However, I would recommend extreme care in its use, especially because of the danger fostering an individualized notion of deliberation Another problematic element in these approaches is the assumption that only contact with opposing perspectives would promote online deliberation (LEV- ON and MANIN, 2009;MUTZ, 2006).Conversations among like-minded individuals are often seen as fostering a form of mobilization, which could hinder deliberation (MUTZ, 2006).Such a view ignores the relevance of conversations among likeminded individuals to increase the chance that some discourses may be expressed publicly, as argued by Mansbridge (1999) and Neblo (2005).If interpreted through systemic lenses, discussions within a group can be essential to deliberation.
Different types of discussions, in diverse arenas, at varying moments, offer distinct contributions to deliberative processes.
Another piece of evidence that suggests neglected implications regarding the concept of a deliberative system emerges in the way coding schemes are applied.discussion vary significantly and methodological procedures have not been able to capture these variations.This is why, according do Dahlberg (2004), several studies of online deliberation make flawed generalizations, not supported by their data.
In addition, interactions that count as a discourse on the web should be amplified if online experiences are to be understood.The role of videos, songs, cartoons, links, images and comments must be conceived of in their specificities and through their intertwinements.It is problematic, for instance, to neglect the centrality of images in Facebook discussions or the role of videos used to respond to other videos on Youtube.However, online deliberation tends to be taken as an asynchronous variation of face-to-face verbal communication.Studies are inclined to focus on forums and communities, measuring the arguments verbally expressed by their members.It is definitely easier to study these interactions; but this procedure may pass over the whole experience of online discussion.One exception here is the recent work of Davies and Chandler (2012), who emphasize the need to comprehend the variety of communicative elements in online interactions and explicitly draw attention to the different modalities discourses may assume.
In this sense, the nature of social ties, the forms of expression, the routes followed by discourses, the regimes of visibility and even the boundaries between public and private are singular in online practices.This is not to say that the internet creates an entirely different world.Nevertheless, there are certain specificities that should be taken into account if web deliberation is to be fully comprehended.
One of these specificities is deeply related to the aspects developed in the previous section.I argue that if deliberation, as such, has much to gain from the idea randomly encouraged through individual practices, the network of networks should not be imagined as a cluster of enclosed arenas.Although it may sound obvious, it is important to emphasize that the idea of a web is essential to the study of this network.However, as many studies focus on the micro-analysis of individual posts within a distinctive arena, the undisciplined discursive flows that surround the specific post are frequently neglected.
An additional specificity is related to the type of engagement that is expected from online deliberators.As opposed to focusing on the discursive process engendered by certain practices and initiatives, studies focus on the energy spent by each participant.These studies often express a feeling of frustration because of a lack of engagement of participants.It is frequently suggested, for instance, that the high levels of one-timers would show the inability of online experiences to foster deliberation.Analogous to this, some scholars seem to expect that users would behave in social networks, online groups and other web arenas in exactly the same way as if they were in conventional meetings.The point I am trying to establish is that most studies of online deliberation seem to lack a sociological understanding of the way in which individuals behave online.Subjects are overburdened with certain expectations that emerge from other interactive structures, a practice which ultimately ignores the dynamics of online experience.In the quest for reciprocal and respectful arguments on the web, many studies simply borrow a pre-established idea of debate.This results in a process that fails to seek out new definitions for public discussion that could better accommodate the idiosyncrasies of the internet.

Concluding remarks
This article has sought to discuss three weaknesses found in prominent methods utilized for the study of online deliberation: (01) the establishment of misleading distinctions; (02) the neglected implications of the concept of a deliberative system; and (03) the disregard of some specificities of the internet.I briefly pointed out that some of the procedures often used for comprehending web discussions have been unable to grasp the nature of online interactions.The focus Therefore, I do not argue that the empirical literature on online deliberation is unproductive.
In my own empirical work on online deliberation, I have attempted to operationalize six criteria (inclusiveness, reason-giving, reciprocity, respect, orientation toward common good and connectivity with other discursive arenas) in ways that combine quantitative and qualitative analyses8 .It would be beyond the scope of this article to explain how each of these categories was operationalized, but it is important to emphasize how some conceptual moves may lead the analysis in fruitful directions.When discussing reason-giving, for instance, I suggest restricting the quantitative measurement to a variable that simply codes the existence (or inexistence) of justifications, further developing the investigation through a Batesonian-Goffmanian frame analysis, that conceives of frames not as individual strategies but as broader cultural and interactive constructions.This analysis takes into consideration not only words, but also images, memes and links mobilized by posts and comments.Another criterion that deserves attention is reciprocity.My My argument is simply that key weaknesses permeate most of the empirical studies.
Such studies would greatly benefit from a more complex view, which does not mean the establishment of numerous detailed categories to capture the minutiae of individual discursive constructions.The core idea of deliberation and the nature of the online experience must be kept in mind.By doing so, the concept of a deliberative system can contribute a great deal because of its emphasis on the reticular character of human interaction.A deliberative system helps to create an understanding of the complexities and specificities of web deliberation, thus generating new routes for empirical studies.

Revised by Cabo Verde Submitted in December 2014
Accepted in July 2015 's (2003)  comparison of a Usenet group (dk.politik), and a government sponsored forum (Nordpol.dk) in Denmark, andDahlberg's (2001) studies on the renowned experience of Minnesota E-Democracy.The research conducted by Graham and Witschge (2003) is also significant in these early stages of online deliberation research.The investigators focused on a British governmental website (UK Online), which, at the time, had around 20,000 posts.

(
put forth three theoretical reformulations: (01) the consideration of alternative forms of communication (labeled Type 02 (2015) 9 (3) 88 -115Deliberation) besides only contemplating rational discourse; (02) the establishment of standards for considering an interaction as deliberative; (03) the adoption of a sequential approach that does not anticipate encountering all of the features of deliberation throughout the whole process.Such reformulations affect the analytical matrix advanced by these scholars.Equality is measured "by counting the frequency of participation as well as by counting its volume (measured by the number of words)"(BÄCHTIGER et al.,   2009, p. 05).Five levels of justification are considered, with the inclusion of one additional category: sophisticated justification (in depth), which means that "a problem is examined in a quasi-scientific way from various viewpoints"(BÄCHTIGER et al., 2009, p. 05).The variable Respect and Agreement "measures whether speakers degrade (0), treat neutrally (01), value (02), or agree (03) with positions and counterarguments"(BÄCHTIGER et al., 2009, p. 06).A variable identified as Interactivity assesses mutual references between arguments.Concerning Constructive Politics,Bächtiger et al. (2009) establish four categories; in addition to the three factors already suggested in the original version of the DQI, the authors believe consensus appeals should also be considered.To evaluate Type 02 Deliberation, the scholars consider the possibility of deliberative negotiations, arguing that the use of threats and promises "allows to empirically distinguish between 'deliberative' and 'non-deliberative' forms of negotiation"(BÄCHTIGER et al., 2009, p. 08).Lastly, this revised version of the DQI measures the use of story-telling as a source of justification.The idea of looking at the sources of justification was expanded on by Jennifer Stromer-Galley (2007), who developed one of the most influential frameworks for scholars dedicated to the comprehension of online deliberation.Her micro-analytic approach is based on six elements: (01) reasoned opinion expression, (02) references to external sources when articulating opinions, (03) expressions of disagreement and hence exposure to diverse perspectives, (04) equal levels of participation during the deliberation, (05) coherence with regard to the structure and topic of deliberation, and (06) engagement among participants with each other (STROMER-GALLEY, 2007, p. 04).Stromer-Galley (2007) develops a complex coding scheme that begins with dividing units of discourse into four categories that specify the type and aim of that 95 problem (focused on the issue), metatalk (talk about talk), process (talk about the process) or social (talk that fosters or hinders social bonds).The next step involves tracing thoughts within the segmented units."The 'thought' is the unit of analysis for which the deliberations are coded" (STROMER-GALLEY, 2007, p. 22).These thoughts should be understood within turns that may Start a new topic, Respond on topic, Respond to moderator or Continue self.Thoughts that express the problem focused on the arena are coded as expressing an Opinion, an Agreement, a Disagreement, a Fact or a Question.Thoughts representing metatalk are divided into manifestations of Conflict, Consensus, Clarification of one's own position or Clarification of someone else's position.Thoughts regarding process can point to Technical Problems associated with the process, which may include Technical Benefits, Deliberation Process, Deliberation Problems or Deliberation Positive.Lastly, thoughts coded as social can be designated as Salutations, Apologies, Praise or Chitchat.Problem and Metatalk thoughts that are on topic are further coded according to their valence: For; For-but; Against; Against-but or Unsure.In addition, when these thoughts are expanded on, the source of such elaboration is coded as Personal Experience, Briefing Documents, Mass Media or Other Participants.In contrast to the DQI, "the elaboration measure did not categorize the types of reasons offered, the quality of the reasons, nor the accuracy or factual nature of the reasons" (STROMER-GALLEY, 2007, p. 10).Equality among participants "was measured by counting the frequency of participation and by volume-measured by number of words" (STROMER-GALLEY, 2007, p. 11).The measurement of engagement included not only the levels of responsivity, but also the formulation of non-rhetorical questions.Ultimately, the model fostered by Stromer-Galley (2007) has several similarities with the DQI, and thus advances a matrix for a micro-content analysis.A third proposal focusing on this type of micro analysis was recently presented by Raphäel Kies (2010), who gathered elements from different models, including those used by the DQI and by Stromer-Galley (2007).His indicators are presented in Table 02, below: Table 02.Indicators for assessing deliberation according to the Kies (2010) Criteria Indicators Inclusion Evaluation of the ease of access to the online forum, on the basis of connectivity, ICT skills and discursive rules Discursive equality Assessment of discursive concentration and the level of control of the debate Reciprocity Measurement of the proportion of posts that are within a thread and the proportion that start a new thread, in addition to the assessment of the extent to which posts take into consideration opinions previously presented Justification Evaluation of whether the opinions are justified or not and how complex justifications are.It should also be observed he depth of justifications, which is measured by coding the use of internal (based on personal viewpoints) or external (based on facts) justifications Reflexivity Content analysis points to apparent cases of reflexivity.Surveys and interviews help demonstrate more internal processes Empathy Measurement of cases of disrespect and surveys and interviews that ask users about levels of respect Sincerity Assessment of apparent cases of insincerity.Surveys and interviews indicate the participants' perceptions of the intensity of each other's sincerity Plurality Evaluation of sociodemographic profiles of participants and their political involvements External ImpactSigns of extension of the discussion to an external agenda.Participation of political personalities in the forum.Users participate in other discussion spaces.There are concrete outcomes.Source:Kies (2010, pp.56-57).
and in depth justification does not reveal much about deliberative processes for several reasons.First of all, judging the completeness of an inference is not as simple as the proponents of the DQI seem to imply, and may vary across different cultures.Secondly, the number of justifications presented does not determine the strength of one's argument.Neither does using quasi-scientific examinations to interpret problems.Complex justifications may even hinder deliberation as such, because they may compromise the general comprehensibility of discourses or embarrass other participants.As observed byDahlberg (2004), "the most prolific posters and positions do not necessarily command the most attention"(DAHLBERG, 2004, p. 35).Categorizing levels of justification may nurture some critiques of deliberation that (wrongly) point to the elitist nature of the theory.Thirdly, the coding of the levels of justification neglects the basis of deliberation.The strength of this process should not depend on individual opinions, but on the broader process within which these remarks are inserted.The DQI individualizes processes and transforms deliberation into nothing more than an exchange of utterances.The philosophy of the conscience, strongly contested byHabermas (1987), returns through the backdoor.Rich deliberative processes can be demolished by coders simply because they view each utterance as being unsophisticated.On the contrary, a weak process can be praised for featuring isolated actors and opinions.
substantive consensus or about meta-consensus(DRYZEK and NIEMEYER, 2006)?Is the non-radical middle way always better in deliberative terms?With this type of hierarchy, the DQI, once again, seems to feed the critiques of deliberative theory with misguided and unclear assumptions.The revised version of the DQI also creates a new problem, which was not present in the first version of the method.Besides the internal hierarchies within categories, the revised version develops a hierarchy regarding types of deliberation.According toBächtiger et al. (2009) there would be a more demanding type of process (Type I Deliberation) and a more informal one (Type II Deliberation).Instead of simply differentiating between forms of providing reasons, however, the authors set a very clear normative distinction.Type II Deliberation would "involve a shift away from the idea of purely rational discourse toward a conception of deliberation that incorporates alternative forms of communication (such as story-telling) and embraces self-interested behavior such as bargaining"(BÄCHTIGER et al., 2009, p.   03).Story-telling is not accepted as another way to provide reasons, but is considered a totally different form of communication.It would be simply a more realistic perspective when trying to comprehend real world practices, which are usually below the standards of Type I Deliberation.Throughout their article, the authors explicitly establish different standards for these 'types' of deliberation.
) advocates that the source of posts should be coded, proposing four main categories: (01) Personal Experience, (02) Briefing Documents, (03) Mass Media or (04) Other Participants.Such distinctions present at least three problems.Firstly, most participants in deliberative processes do not make explicit references to the sources of their opinions.Although some online arenas contain a great number of posts with links to other documents, this should not be seen as a valid rule for any sort of online experiment.Secondly, the categorization proposed byStromer-Galley (2007) neglects other possible sources participants may frequent, as well as the intertwinement of the sources she

(
distinguishes between internal and external justifications, with an implicit preference for the later.Internal justifications, based on personal viewpoints, are somehow considered to be a more superficial and less demanding way to present one's positions.The ability to refer 101

(
Mostly, online arenas and initiatives are scrutinized in themselves.Along with the DQI, the frameworks ofStromer-Galley (2007) andKies (2010) tend to assess interactions within a certain space.Reciprocity is usually restricted to a process of direct interlocution and reason-giving is conceived of being something internal and 103 specific arena under analysis.The attempt to comprehend these internal relationships often ignores the broader nature of discursive flows.Online deliberation is constrained to a reproduction of face-to-face conversations.The role that information provided by online initiatives plays on a deliberative system is disregarded or even criticized as not fully dialogical.The connections (and disconnections) of arenas, and the discursive routes built online are overlooked.Deliberation is viewed as something to be observed within an initiative or arena, and not across initiatives and arenas.Specific points in the framework of both Stromer-Galley (2007) and Kies (2010) could be thought of as exceptions in this regard.The former suggests the measurement of sources cited by actors, while the later considers the external impact of arenas in his analyses.These points indicate the relevance of the 'external world' on processes that happen within a given online arena.However, both ideas have limitations.Stromer-Galley's (2007) sourcing, as was already noted, can only assess what is explicitly mentioned and results in missing the idea of uninhibited discursive flows that cannot be properly identified.Kies's (2010) external impact reduces the many possible connections among arenas to one type: influence on the elaboration of political decisions.As a result, neither of the two 'exceptions' is properly equipped to capture the broader idea of deliberative systems.Each idea can grasp some (eventual) connections, but they are not effective when dealing with the idea of structural connections at the grounds of their frameworks.This is one of the central challenges for current methodological measurements of online deliberation.Understanding the connections, routes and flows among discourses on the web should no longer be thought of as something that can be ignored.If online deliberation is to be understood, these complex processes should be faced properly.Discussions occurring within an online group or forum represent a small fraction of a much more complicated process, that pervades online and offline arenas.The concept of deliberative systems has become essential to obtaining a complete study of deliberation.The disregard of some specificities of the internetA third problem with some of the most influential empirical attempts to investigate deliberation on the web is related to a disregard of the nature of online (2015) 9 (3) 88 -115 interactions.Some scholars seem so concerned with their attempt to translate the conceptual dimensions of deliberation into empirical categories that they end up missing key aspects of the web.This type of disregard seems evident in the lack of attention to the different discursive architectures of online arenas.Usually, methodological procedures are conceptualized in a generic way that frequently fails to grasp the particularities of distinct online experiences.The discussion of politics either on a social network community site, a newspaper website, a blog or Facebook generates completely different processes.They should not simply be gathered under the umbrella of Type II Deliberation or coded as if they were disembodied discourses.The logics of online , then online experiences cannot be studied without it.The richness of online deliberation lies in the countless dynamic connections that engender new forms of discussion.Either explicitly promoted through linkage, or place on micro distinctions has frequently hindered an understanding of the broader picture in which they are inserted.This does not mean micro-content analysis is, in itself, wrong or misleading.It has been responsible for interesting developments in the areas of both deliberative democracy and internet studies.There are, of course, fruitful findings that help to explain the possibilities of web discussion, thus supporting the work of those responsible for designing online consultations and web forums.Methodological insights also exist that point to new research trends, such as Graham andWitschge's (2003) proposal of re-building argumentative maps or Gerhards and Schäfer's (2010) attempt to study virtual debates through search engines.
coauthors and I have attempted to distinguish a direct type of reciprocity, usually measured by empirical studies, from a discursive form of reciprocity accessible through frame analysis and more in tune with the systemic approach to deliberation(MENDONÇA, FREITAS and OLIVEIRA, 2014).In addition, my investigations have benefited from the concept of affordances, frequently used in technology studies, which paved interesting routes for context-sensitive analyses., therefore, skeptical about the possibility of empirically assessing online deliberation and, as a matter of fact, measuring some of its dimensions.I am against excessive micro-quantification, focused on individuals and on arenas (considered as self-enclosed) and not sensitive to the contexts of online interaction.

Table 01 .
first version of DQI is based on six Habermasian principles: (01) open Categories for assessing deliberation according to DQI