THE QUALITY OF QUALITATIVE RESEARCH : FROM EVALUATION TO ATTAINMENT

The objective of this paper is to emphasize the importance of quality in the research process instead of its valuation afterwards, an issue the literature has given extensive attention to. In addition, it is a reflection on the debate about the quality of qualitative research and presents the assessment of quality as a situated practice. Reflexivity is presented not as a criterion to assess the research quality but as an instrument to achieve it. There are three characteristics of qualitative research that researchers need to pay reflexive attention to. The first is that qualitative studies deal with human experiences; the second that these experiences are subjective; and the third that qualitative knowledge is ideographic and constructed during the study. Beyond these characteristics, issues are signaled that are constantly repeated in the studies and that unknowingly are a threat to their quality are addressed in this paper. DESCRIPTORS: Quality control. Qualitative research. Social validity. LA CALIDAD DE LA INVESTIGACIÓN CUALITATIVA: DE EVALUARLA A LOGRARLA RESUMEN: El objetivo de este artículo es enfatizar la importancia de la calidad en el proceso de investigación y no en su valoración después de ella, algo a lo que la bibliografía se ha dedicado extensamente. Así mismo, reflexiona sobre el debate de calidad de la investigación cualitativa y muestra la valoración de la calidad como una práctica situada. La reflexividad se presenta no como un criterio para evaluar la calidad, sino como un instrumento para lograrla. Tres son las características de la investigación cualitativa a las que los investigadores deben prestar una atención reflexiva. La primera es que los estudios cualitativos versan sobre experiencias humanas; la segunda que estas experiencias son de carácter subjetivo; y la tercera, que el conocimiento cualitativo es ideográfico y construido durante la investigación. Alrededor de estas características, en el artículo se señalan cuestiones que se repiten constantemente en los estudios e inadvertidamente amenazan su calidad. DESCRIPTORES: Control de calidad. Investigación cualitativa. Validez social de la investigación. A QUALIDADE DA INVESTIGAÇÃO QUALITATIVA: DA AVALIAÇÃO À CONCRETIZAÇÃO RESUMO: O objetivo deste artigo é enfatizar a importância da qualidade no processo de investigação e não sua valorização depois de concluído, assunto sobre o qual a bibliografia tem se dedicado largamente. Reflete sobre o debate de qualidade da investigação qualitativa e mostra o valor da qualidade como uma prática em contexto. A reflexividade apresenta-se não como um critério para avaliar a qualidade, mas sim como um instrumento para atingi-la. São três as características da investigação qualitativa a que os investigadores devem prestar uma atenção reflexiva. A primeira é que os estudos qualitativos dizem respeito às experiências humanas; a segunda é que as experiências têm caráter subjetivo; e a terceira, que o conhecimento qualitativo é ideográfico e construído durante a investigação. Para além destas características, assinalam-se questões que se repetem constantemente nos estudos e que, inadvertidamente, ameaçam a sua qualidade. DESCRITORES: Controle de qualidade. Pesquisa qualitativa. Validade social em pesquisa. 883 http://dx.doi.org/10.1590/0104-070720150001150015


INTRODUCTION
"Methods are not procedures to be followed in any standardized way, but rather are created anew in every research project by researchers who hold their work to a standart." 1 Since the end of the past century, the quality of qualitative research has been a theme of interest, debate and, in some cases, dispute.The need to establish the legitimacy of qualitative research in the scientific community and to provide tools for reviewers external to the research process have played a decisive role in the proliferation of articles, texts and the design of evaluation instruments.In addition, the boom in systematic reviews that include qualitative studies and metasyntheses has sharpened the need for agreements on the most appropriate way to value the scientific merit of qualitative studies.
Assessing the quality of a study is not a technical, aseptic exercise, but is contextualized and, in addition, has an educational impact, as the assessment criteria serve as pedagogies of practice. 2t is the educational aspect of quality this article is focused on.I will emphasize the importance of quality in the research process instead of its evaluation afterwards, something the bibliography has paid extensive attention to.4] Instead of writing for future evaluators external to the research process, I write for researchers and mainly beginning researchers, doctoral students and junior researchers.I intend to alert them to issues that interfere in the research quality and I raise themes they should pay reflexive attention to.
In this paper, I link reflexivity with quality.I present reflexivity not as an indicator of quality, but as a tool to achieve it and with an educational potential.I support my arguments on my research and teaching experience, share my own and other researchers' errors.First, I will briefly contextualize the issue of the quality of qualitative research.

THE ISSUE ABOUT THE QUALITY OF QUALITATIVE RESEARCH
The contemporary criteria for qualitative research assessment have been comprehensively developed, despite the lack of a full consensus.One might say that everything starts in 1985, when specific language and criteria are proposed to assess it. 5Then, several criteria and evaluation models were proposed, which opened the door for what has been called the quality debate. 6he issue about the rigor of qualitative research can be overwhelming to young researchers in view of the large number of publications and different positions about it.In a recent literature review, four approaches were identified: 1) those who think that no specific criteria are needed as there is only one kind of research; 2) those who defend specific criteria for qualitative research; 3) those who consider that each qualitative method should contain specific criteria; and 4) those who use an overlapping criterion that comprises the general and particular factors of the qualitative methods. 7The most common position nowadays are researchers who defend specific guidelines for qualitative researcher, 2 but adjusted to the different qualitative methods.
To date, proposals of criteria to assess the merit of qualitative research are endless and in some cases repetitive, with authors indicating different terms for the same thing. 7With his controversy comes the proliferation of technical language.
The core of the quality debate has been the impossibility to assess quality by applying standardized criteria in view of the flexible nature of qualitative research, besides the danger that these criteria will serve as straightjackets. 6Nevertheless, there exists a generalized consensus today that flexible criteria are needed, in view of great disagreements in the bibliography about the most appropriate criterion to assess research quality.What is paradoxical about this undertaking is to underwrite the subjectivity and, at the same time, establish objective criteria that demonstrate the merit of a qualitative study. 6,8Hence, what seems to underline the whole quality issue is the tension about positivist and interpretative positions.
These tensions are shown in proposals for evaluation criteria of qualitative studies as well as in its assessment tools.To give an example, it has been appointed that the evaluation models are based on the rationality of biomedical sciences and that some tools contain soft criteria from quantitative research applied to qualitative research. 2In addition, it has been raised the question about what counts as evidence.It has been pointed out to the fact that, in the health area, evidence is considered as what adjusts to certain positivist standards. 3The consequence of this is that qualitative evidence is discarded.
Where the emphasis of the evaluation is put should also be taken into account, whether it is on the results of a study or it is on the methodological process through which it is developed.In that sense, the emphasis on the assessment of aspects of the research process has been criticized and attributed to positivist positions. 9The importance of critically examining the results of qualitative studies was highlighted in a metasynthesis about women with HIV/AIDS. 10These authors detected an important number of studies whose findings were not qualitative, although the studies figured as such in the methodological sense.In addition, methods like the grounded theory are ruled by criteria that apply to the study results, in which the credibility, originality, resonance and utility of the theory produced are valued. 11The concern with the methodological rigor of qualitative research is a positive legacy. 12he quality of qualitative research should be considered as an ongoing debate 6,13-14 instead of a problem that needs solving. 15In the last 30 years, the debate has led to the improvement and development of its quality. Nowadays, a generalized consensus exists that the quality criteria should act as guidelines and should adjust to each qualitative method.Nevertheless, the difficulty to assess the merit of a qualitative study is acknowledged. 6,8,14Authors of qualitative research texts are increasingly inclined to present some guiding principles, together with a series of questions that guide the assessment of a study's quality. 6,11,17Three recurring principles are present in the bibliography.One is the study's contribution to the research.This contribution refers to the value and relevance of the evidence the study presents.The second criterion is the likelihood or credibility of the research findings.This principle relates to the plausibility or solidity of the arguments.Three is the rigor of the study, which refers to the methodological validity. 6These principles are balanced and respond to the cannons of qualitative research.Not only the reviewers, but whoever engages in qualitative research should be familiar with them.

THE ASSESSMENT OF QUALITY: A CONTEXTUALIZED PRACTICE
To assess the merit of a qualitative study, are available closed tools, like the checklists that contain a large number of items and predetermined score sheets; and open tools with a small number of criteria for the reviewers to use them as guidelines.While, in the first case, the reviewer does not need to be familiar with qualitative research, in the second case, it is absolutely necessary to know it well.
In fact, many of the closed score sheets were elaborated to help people who were not familiar with the qualitative method. 2 Their use has entailed negative consequences though.It has been appointed that the blind application of criteria weakens qualitative research, 4 and that some criteria have encouraged superficial qualitative analysis. 3On the other hand, the closed score sheets tend to present the research as a set of procedures to comply with and can promote the elaboration of reports with quality jargon but void of contents.I have received plenty of research reports that mention terms like: theoretical sampling, saturation or negative case, without evidence that this actually took place in practice.
For those who are starting their qualitative research journey, a review using an open tool that comments on the strengths and weaknesses of their work is undoubtedly more useful than a report in which it is indicated what was complied with and what was not.Much of the research that is learned as a novice researcher comes from the reviewers who attract attention to the weaknesses of the reports.
But assessing a qualitative study is a situated activity.It always implies a professional judgment. 6,11It demands that the evaluator discerns between hardly important errors or omissions to interpretative or fatal ones. 18In addition, evaluators belong to interpretative communities that influence what they read and how they read it; not only a collective, but also a personal and esthetic judgment is made. 18Hence, it should cause no estrangement that, although the same criteria are used by evaluators, different conclusions can be reached.Hence, it is common practice in systematic reviews that the studies be assessed by at least two persons, with a third one to solve discrepancies.The subjective component of the assessment is thus tacitly acknowledged here.
Assessing is a situated act in which not only the reviewer, but also what is reviewed intervenes; that is: if it is a manuscript for publication in a certain journal, if it is a report from a doctoral dissertation or a funding project.In addition, in this assessment is considered how scientific merit is defined in each discipline, that is: what counts as evidence. 3,11To give an example, as qualitative evidence tends to be considered as mere opinion in the health area, the criteria of what is considered as evidence should be expanded to integrate the findings from qualitative studies. 3inally, in the assessment of the research quality there has been a trend to mix up the report itself with the research it represents. 18The report is designed more to persuade the reader of the merits of the study than to reflect it.The emphasis is on the writing, on producing a convincing, evocative text.In the report, linguistic resources are used to persuade the reader about the validity of the study. 18:6.Quality judgments are based on the way the study is presented instead of the actual study. 8,18nderstanding this clearly, helps to receive the criticism of a manuscript.If it is late to correct the errors as they are mistakes committed during the study, it will prepare the researcher for the next one.One learns research practice through mistakes.Therefore, reflexive attention to the quality is needed.An issue that I will discuss next.

ACHIEVING QUALITY: A REFLEXIVE ACT
4] More than 25 years ago, it was alerted that the quality of qualitative research rests on the researcher. 19Other authors 1,4 emphasize this when they affirm that the research is as good as the researcher is, this is not strange as (s)he is the research tool. 20Similarly, quality cannot be separated from a reflexive act as, without the latter, it is difficult to achieve the former.Traditionally, reflexivity has been understood as the process of looking at oneself to critically examine the effect produced in the development of the research. 21In this paper, I will present it as the awareness that the researcher should have about the quality of his/her study.
One might say that we qualitative researchers need to achieve that quality by naturally integrate it into our know-how, that is, that our research actions flow without any need to reflect on their coherence and pertinence.Given the emerging nature of qualitative studies, however, we should also maintain an alert and questioning attitude about their quality.Hence, we are judges and stakeholders here, objects and subjects of our actions at the same time, reflexive beings who in the action of talking and questioning ourselves, we mold our research beings.Therefore, reflexivity is something more than an item on a checklist to verify the rigor of a study, it is key to the achievement of a habitus, 22 of what we might call that of an investigative quality.That is how to achieve the standard referred to in the epigraph of this paper. 1n my experience, epistemological coherence is the key to the quality of qualitative studies and, at the same time, its Achilles heel.The strong presence of positivist research in the health area and the students' socialization in that perspective leads to misunderstandings in my opinion.Qualitative research has three characteristics, deriving from its epistemology, which are a cause of errors, mainly among novice researchers, and which threaten the quality of their studies.The first is that qualitative research deals with human experiences, the second that these experiences are subjective, and the third that qualitative knowledge is ideographic and constructed.These characteristics are well-known but their profound implications tend to go by unnoticed.Making them explicit intends to alert young researchers so as to enable them to correct them before it is too late.Let us examine them.

Qualitative studies focus on human phenomena or experiences instead of populations: making the study proposal
4] That is a key characteristic from which questions about the research theme, sampling, data analysis and the scope of the research derive.Issues that if neglected, invalidate a study or lead to the rejection of qualitative research. 18bove all, it is fundamental for researchers to pay attention to and reflect on what his/her study is about, about how to formulate the research theme and not mix it up with a population group.To give an example, there is a difference between a study on pregnant adolescents or informal caregivers and another about adolescent pregnancy or family care.In the former, the theme is related to people, to a population group; in the latter, to a human phenomenon, i.e. to a theme that affects a group of people.
Qualitative research is about what and not about whom.Therefore, the choice of the participants will be based on their experience instead of some demographic or social variables.They participate in an experience and are therefore selected for the study. 25Introducing variables here and applying stratified sampling, that is: taking a bit of everything, introduces bias and threatens the saturation of the categories.Similarly, choosing the participants based on an experience they do not have experienced themselves and which they witness at most, will provide data that which is not qualitative, no matter how descriptive this data are or how openly it were obtained.Qualitative research is interested in the subjectivity of an experience and data should reflect this.
Participants in a qualitative study do not constitute a sample in the positivist sense, as the goal is not to represent a population or universe; the subjects in a qualitative study are called informants, 26 as they share an experience that provides specific information.Here, the reflexive attention should focus on sampling, seeking different information with a view to a rich description of what is under study 27 and, I repeat, not about who is being studied.If the researcher asks whether the number of participants is sufficient instead of the quality and diversity of the information, (s)he will be asking positivist questions and that will deviate the attention from the study, moving, without even noticing it, from what to whom.Similarly, if one asks about aspects like age and education level without any indication in the data that these topics are relevant, one will be looking for relations among variables, developing a positivist analysis.At this point, one should stop, inspect one's research practice and remember what is being studied and what one wants to know.
The researcher will also focus on how the data analysis is being developed, on the axis of data analysis.If the analysis is more of a recount of people instead of reporting their experiences, this is not bad, as it can be correct during the study, the nature of qualitative research allows this, but one should notice it.The researcher should assess whether the writing of analytic memos and the advances in the results are centered on the participants instead of the description of the research phenomenon.The use of terms in the writing like some, others, most, alert the researcher about a positivist turn, indicating that the analysis is centered on the study informants and not on describing their experience.In this situation, the researcher will find that managing negative cases will be complicated, as these will be about people itself and categories will not saturate.

Qualitative studies take interest in the subjectivity of a human experience: asking
In a qualitative study, there are at least three classes of questions, all different but sharing the objective of capturing the subjectivity of what is being studied.These questions are the research question, the questions asked to participants and the questions asked to the data during analysis.
The research question indicates the experience or phenomenon being studies and, in general, the answer to this question will invite a narration i.e. a description.Thus, the researcher will examine whether the research question addresses a subjective phenomenon, is about something external to the people or is about people themselves.That is of paramount importance, as the research question puts the research in motion and a proper start is needed.The researcher needs to pay attention so as not to raise concealed hypotheses or to relate variables.The term, "if" in a conditional mode, in a research question or objectives, indicates relationships among variables, that the nature of the phenomenon is known beforehand, that a test will be developed.All of these are positivist positions the researcher should detect and avoid.A positivist research question cannot be answered appropriately using a qualitative method, but makes the research incoherent and gives the researcher a headache in the course of the study.
Similarly, here should be clear to the researcher that the place of study is not the object of the study, 28 although the former evidently conditions the latter. 29If the theme under study includes the place of study, the researcher will need to reflect on what makes the place special and clearly indicate its particularities.The place of study is not invariably equal to the research context either, the place of the study is the physical support of the action-interaction. 17,20The context will constitute people's relationships and indicates the conditions under which that action-interaction takes place.If the researcher does not take this into account, it will be difficult to contextualize this study and it is highly possible that its findings will lose explanatory power.One of the most clarifying things in a qualitative research study is precisely to discover that context, as it will give meaning to the experience and will explain the participants' actions-reactions in understandable terms.
Consequently, the questions that are posed to participants in fieldwork will reflect the researchers' interest in capturing their subjective experience.There will be few open, exploratory questions with a descriptive aim.These questions will invite study participants to report on the uniqueness of their history.What is of interest here is to obtain details of the experience instead of generalities, this is a crucial characteristic of data that is qualitative.Researchers will be aware that the qualitative data are obtained through the relationships established with the participants and will pay attention to what is asked and on how it is done.It is fundamental that the researcher critically analyzes the interview guide and his/ her style to obtain data.Nevertheless, it will not be rare for the participants to opine on something external, not experienced, that they report on other people's experiences, that they offer interpretations about facts or people and that they use technical jargon.Therefore, in data analysis, the researcher will consider in the first place whether the information constitutes data and, in the second place, will distinguish between the descriptive and the interpreted data, so as to avoid inconsistent analysis. 30inally, there are the questions the researcher asks the data with a view to generate analysis.These questions are asked to discover what is actually being studied and tuning the researcher with the insiders' point of view.As example, the following questions will be asked to data: What happens in the data?What is the study about?What are participants concerns? 30This type of questions puts the researcher in a position of empathy and openness to discovery.Hence, during the analysis, the researcher will reflect on how (s) he interacts with the data, realize if (s)he mechanically undertakes the procedures and passively await the concepts "to appear", or whether she/he engages in an active and interactive dialogue with data.If software is used to manage the analysis, special caution is due.These programs are very efficient and, if not operated correctly, they promote a mechanical, fragmented and superficial analysis.It is increasingly clearer that findings in a qualitative study are constructed, that they emerge from the interaction between the researchers, the participants and the data. 11The reflexive diary should report these interactions and, if that is the case, re orient the analysis to achieve the quality of the study.

Qualitative knowledge is ideographic and constructed: knowing
The knowledge produced in qualitative studies is ideographic, as they search for generalizations in the case and not in populations. 10herefore, not generalizing one's findings to other groups is not a limitation, as affirmed in some research reports and papers.Qualitative research has its own limitations, many of which derive from the sampling instead of the sample.
The limitations imposed in the fieldwork mold the study and the researcher should be aware of these limitations and acknowledge them in the reflexive diary.The methodological decisions taken during the fieldwork are not so much related to objective motives of rigor, but to negotiations and ethical decisions about what data to obtain and how to do it.Quality of practice demands ethical reflection.
Similarly, qualitative knowledge is constructed in the action of research itself.It is not something that it is found, that it is "there", outside, waiting to be discovered.The data are not collected, although this is the way that is commonly mentioned, but data are obtained or, better, constructed. 11Data triangulation, used frequently as a technique to ensure the validity of a study, can invalidate it when the researcher who uses this technique presupposes the existence of external, objective and verifiable knowledge through multiple sources.In qualitative studies, triangulation does not serve to seek a single reality or an objective fact, but to enrich the data, that is: to add characteristics that turn the description into something saturated.
Knowledge construction is a slow process developed in phases.It is not a whim, but follows the principle of building subjective and saturated knowledge.Qualitative analysis is concurrent with obtaining data, as it will end when catego-ries are theoretically saturated or complete. 11,17,20,30his saturation permits generalizations about the research phenomenon, that is, the ideographic knowledge, and the researcher should distinguish it from information redundancy.In order to have saturation, theoretical sampling should take place, this means in practice, changing the questions the researcher asks to informants.Hence, if no new information is obtained in the fieldwork, the researcher will consider whether this happens because (s)he is always asking the same questions or because different information is sought related to a given category.
Qualitative studies offer new knowledge or new ways of seeing the world.They are a personal construction of the researcher that takes place in interaction with study participants.This construction is a product of who the researcher is, his/ her biography and way of thinking.As research instrument, she/he put all this into play to achieve a good study, but needs to be aware of who (s)he is, how (s)he thinks and does things.In other words, (s)he needs reflexivity for everything to favor the study quality.

CONCLUSION
To ensure the quality in the research process has tended to pass unnoticed in the literature, which has centered on how to assess it instead of how to achieve it.The assessment of qualitative studies has been immersed in positivist logic to the detriment of qualitative research itself.Despite criticisms, the quality debate is needed as, without rigor, qualitative research loses its utility.
In this paper, I have centered my attention on reflexivity.I have presented it not as a quality criterion, but as a tool to achieve it, with a learning potential.I have proposed neither criteria nor new models or schemes, instead I exposed issues that are constantly and inadvertently repeated in some research studies and that threaten their quality.
Qualitative studies are flexible and permit rectifying mistakes and learning from failures.In this lays to a larger extent its educative nature.I have turned reflexivity into the key element for this to happen.Researchers will assess their paradigmatic position, the way in which they constructs the research problem and the way they relate to others in fieldwork and with data.In the dialogue they establish with themselves, they will gradually construct a particular perspective and when they realize the positivist ideas that might slip into this way of thinking, they will rectify those that impede the development of a study of quality.This reflection will tuned them as research instruments and will turn quality into a research act.