An analysis of articles on qualitative studies conducted by doctors published in scientific journals in Brazil between 2004 and 2013

Recebido em: 21/07/2015 Aprovado em: 15/12/2015 Abstract: We analyzed articles on studies carried out by doctors using qualitative methods. A literature search of articles published on the SciELO Brazil database between 2004 and 2013 was performed using the following keywords: qualitative research, interview, focus group, participant observation, content analysis, discourse analysis, social representation, and dialectical hermeneutics. The texts were analyzed using a theoretical framework based on the qualitative research review guidelines (RATS). Articles were grouped into the following categories: consistent, not very consistent, and inconsistent. A total of 135 articles were selected from 28 journals. The majority (64.4%) were considered consistent. The main weaknesses found were: lack of information on methodological path; superficial and descriptive analysis without reference to relevant literature; conclusions that did not advance beyond common sense; decontextualized results, and lack of study limitations. We concluded that the majority of the articles had scientific validity and suggest that journals covering the health field should provide specific norms for articles produced from qualitative studies.


Introduction
Despite the fact that doctors' close contact with patients makes them particularly well placed to carry out qualitative health research, the majority of qualitative health studies are conducted out by non-doctors (TURATO, 2005).Furthermore, the number of medical journals that publish qualitative research is very limited.Qualitative research is understood as research that is concerned with the subjective and relational level of social reality and that deals with history, the universe of meanings, motives, beliefs, values and the attitudes of social actors (MINAYO, 2013).Qualitative research methods are practically unknown to doctors since medical training is generally lacking in human sciences.Furthermore, doctors tend to believe that qualitative research does no lend itself well to the reproducibility and generalizability of results (TAQUETTE; MINAYO;RODRIGUES, 2015).Morse (2006), editor of Qualitative Health Research, suggests that medical researchers believe that qualitative research is very subjective and biased and therefore not scientifically valid.Canesqui (2011, p. 18) reflects on the presence of the social and human sciences in public health and argues that, in view of the lack of attention given by the medical field to the subjective aspects of the reality of people and the social context, "it could be said that medicine continues to be deaf to the human person".In turn, scientific texts on qualitative research tend to use social science jargon that is little-used by doctors (MARSIGLIA, 2013).Social scientists find it difficult to "translate" their language to make it more accessible, which in turn hinders integration across disciplines and hampers the adoption of a more comprehensive view of the phenomenon of illness, life and death by health professionals.More recently, subjects that fall within the spectrum of social and human sciences have begun to be included in medical course curriculums.However, the interest of professors and students generally remains focused on subjects related to the biomedical model of medicine, hampering the construction of a comprehensive view of the human being.
The good news is that the number of publications resulting from qualitative health studies has increased.However, this number has increased at a slower rate than articles concerning quantitative research.Social scientists working in the health sector have repeatedly drawn attention to the lack of publications and the limited theoretical consistency of data interpretation (GOMES; SILVEIRA, 2012).These observations are not limited to national research.In a study of texts produced by the world's most renowned contemporary social sciences authors, Minayo (2012a) observed that there is significant controversy among those who defend empirical studies per se, as an essential component of this type science, and those who emphasize their theoretical and critical role.Many of the former argue that there is a need to for a certain degree of pragmatism in research in order to provide solutions to the concrete problems of society.In a letter to the editor of the Revista de Saúde Pública (Journal of Public Health), Oliveira et al. (2012) promote the discussion about ways of doing science within qualitative research approach.The authors criticize the trivialization of the use of terms without a necessary deepening of conceptual and epistemological understanding, which affects the understanding of the study question and compromises the validity of results.The dissemination of studies with questionable consistency therefore hampers the consolidation of qualitative research as a relevant knowledge producer.
This article assesses the methodological consistency of qualitative health research conducted by doctors and published in indexed scientific journals.Using a theoretical framework based on the qualitative research review guidelines (RATS), the aim of this literature review is to identify the strengths and weaknesses of the articles and thus determine the validity of the knowledge they produce and disseminate.

Methodological path
We conducted a search of articles based on qualitative methods written by doctors and published in journals indexed in the Scientific Electronic Library Online (SciELO) Brazil between 2004 and 2013.SciELO was chosen because it is the most comprehensive collection of scientific journals in Brazil.The publications contained in this data base go through a periodic evaluation by a committee of experienced professionals that represent of the core areas of knowledge and each article goes through a mandatory peer review process.
The following keywords were used for the article search: interview (indepth, semi-structured, open-ended); research/method/qualitative study; focus group; life story; oral history; participant observation; social representations; narrative(s); content analysis; discourse analysis; and dialectical hermeneutics.This search resulted in 1,616 articles.Subsequently, we filtered these articles by title and journal.To increase the likelihood of finding articles written by doctors, we eliminated articles and journals whose titles/subject area clearly addressed non-relevant topics, such as linguistics, economics, politics, etc...For the same reason, we also discarded articles published in nursing and dentistry journals, which traditionally publish work produced by professionals from these areas.We then read the abstracts of the articles that remaining after applying the exclusion criteria.For articles addressing health research, we searched for the curriculums of the first and second authors in the Lattes platform, a major scientific information system maintained by the National Council for Scientific and Technological Development (Conselho Nacional de Desenvolvimento Científico e Tecnológico -CNPq), to determine whether or not the authors were doctors.Furthermore, we selected articles written by doctors suggested by other researchers who are known for conducting qualitative research, as well as articles written by previously identified doctors.Purely conceptual articles not resulting from documentary or field research were also excluded.
The final sample of articles was analyzed based on the qualitative research review guidelines (RATS) -BIOMED, 2014.These guidelines were chosen because their use is recommended to the manuscript reviewers of the Revista de Saúde Pública, one of the most highly-ranked journals in Brazil according to the criteria established by the Coordination for the Improvement of Higher Level Personnel (the Coordenadoria de Aperfeiçoamento de Pessoal de Nível Superior -CAPES), having obtained grade A2 in 2014 in the area Public Health, the principal area addressed by the sample articles.The RATS guidelines are composed of four sets of criteria: (I) R -Relevance of study question; (II) A -Appropriateness of qualitative method; (III) T -Transparency of procedures; and (IV) S -Soundness of interpretive approach.Based on these categories and in an attempt to standardize findings and reduce the subjectivity of the evaluation, we created a score chart with 15 criteria, described in table 1.The articles were then classified in the following three categories according to their score: A -consistent, when they sufficiently met 12 to 15 of the criteria; Bpartially analyzed, only descriptive or not very consistent, when the article did not show methodological transparency or adequate interpretations, only meeting eight to 11 of the criteria; and C -inconsistent or not analyzed, when an article only met seven or less criteria.We are aware that these criteria have certain limitations when it comes to analyzing the complexity and theoretical depth of qualitative research.However, we believe that they make a positive contribution to the analysis of the manuscripts in relation to the items in the RATS guidelines.
The articles were read and reread on two different occasions with a gap of two months to analyze and classify them according to the categories outlined in table 1.We believe that the reanalysis and reclassification of the articles helped to reduce possible biases and disagreements between the two evaluations.

Results
We found 135 articles written by doctors: 82 as authors, and 53 as coauthors.
Other studies have shown that the majority of research on health is quantitative (HOFF; WITT, 2000;MCKIBBON;GADD, 2004;WEINER et al., 2011;YAMAZAKI et al., 2009).Despite the variety of keywords used for the literature search, we are sure that we did not map all of the qualitative studies published by doctors, since the descriptors do not necessarily reveal that the study is qualitative.The articles were written by doctors from 12 different specialist areas: principally public health (38.5%) and psychiatry (34.8%), but also adolescent medicine, family medicine, obstetrics and gynecology, general medicine, pediatrics, homeopathy, indigenous health, geriatrics, dermatology, and hematology.The articles were published in 28 different journals covering two major areas -health sciences (25 articles) and human sciences (3 articles).The following seven journals, in descending order by number of articles, accounted for 75% of the articles: Ciência & Saúde Coletiva, Cadernos de Saúde Pública, Revista de Saúde Pública, Physis, Interface, Revista Brasileira de Educação Médica, and Saúde e Sociedade.Table 2 shows the number of publications by journal and main area.
Of the 28 journals, only five had specific editorial guidelines directed at the publication of qualitative research.Certain journals for example mention that "other forms of original article are accepted when pertinent according the nature of the work".Eight did not contain information relating to the acceptance of qualitative articles, but allowed long articles, thus facilitating the publication of qualitative studies, which generally take up more space.Six journals include norms in their instructions for authors that suggest that qualitative studies will be rejected, for example: "procedures should de explained so that other researchers can repeat the study; statistical procedures should be described".Eight journals only accept short articles, which is effectively an obstacle to qualitative studies.One journal provides ambiguous guidelines, since while informing that studies should present results that can be replicated or generalized, it allows statistical treatment and data categorization.A study by Hoff and Witt (2000) shows that international journals do not specify a standard number of pages for qualitative studies or demonstrate an understanding of the fact that qualitative studies require more space because they are descriptive.

TOTAL
The main findings of the study according to the categories outlined above (A, B, and C) are presented below.We then go on to outline the main weaknesses of the articles based on the four sets of criteria outlined in table 1.

Consistent studies
The majority of the studies (64.4%) were classified as consistent.Despite producing relevant and original knowledge, certain studies did not make this category because they did not obtain an adequate score for other criteria.None of the articles in category A obtained the maximum score (15 points).The criteria that were most absent in the articles were "historical-spatial-social context" and "study limitations".It is important to mention that few articles highlighted study limitations and many of those that did confused "study limitations" with qualitative "method limitations", as the following extracts from certain manuscripts show: "The present study has limitations relating to the generalization of qualitative studies.""As in any other qualitative study, the interpretations made in this article may be considered valid internally within the sample."

Not very consistent or partially-analyzed studies
Category B accounted for 32.6% of the articles.Over half of these articles showed weaknesses in methodological consistency (54.5%) and analysis and interpretation of results (45.4%).The majority also showed weaknesses in transparency of procedures, not explaining for example the theoretical basis of categories or how the material was broken down.The criteria that obtained the lowest scores, in descending order, were: "data recording is described", "data collection/sampling is described", and "ethical aspects".Certain articles did not mention sampling, interview length, who conducted the interviews, and whether they were recorded.With respect to "ethical aspects", the majority of articles mentioned that approval had been obtained from the Research Ethics Committee of the academic institution and signature of a free prior informed consent form.
The articles considered partially analyzed showed weaknesses mainly with respect to the criteria "historical-spatial-social context", "interpretations are clear and supported by the evidence and with reference to existing literature", and "limitations are described".Many did not progress beyond common sense and authors were happy to explain the opinion of the study participants, presenting empirical data as the truth regarding the group in question.Based on the above, it can be inferred that the quality of the texts classified as "inconsistent or partially-analyzed studies" would have been better if there had been an editorial policy that emphasized the importance of qualitative studies and set out specific evaluation guidelines.

Inconsistent studies
Only 3% of the articles were classified in category C. We believe that, in the same way that good qualitative articles may be rejected by reviewers with little experience in the field, badly written articles may also be accepted for the same reason.Articles in this category did not justify the study question, did not mention the theoretical framework, and did not use appropriate research instruments, while in others the author even blames the study participants for the lack of information collected.
Other weaknesses may be highlighted that compromise the articles' validity and scientific soundness: flaws relating to article format and structure, such as describing study objectives and results within the methodology section; conceptual errors, for example stating that discourse analysis was performed when the results sections showed that it was actually content analysis; mentioning inappropriate qualitative methods for the type of study, for example one author mentioned the use of oral history techniques when the most appropriate technique, and in fact the technique that was actually used, was the classic open interview.

Weaknesses found in relation to the RATS guidelines
Relevance of research question justified -few weaknesses were found with respect to this set of criteria.The item "relevance of study question" was not scored in only 10 articles (7.5%), because the research question was not justified or decontextualized.The first criteria, "study question and aim are clearly defined" was absent in 14 articles (10.4%), while the criteria "theoretical framework is consistent with the assumptions" was not scored in only one article.
Appropriateness of qualitative method -three studies (2.2%) did not describe how the qualitative methods were chosen.Furthermore, few authors adequately justified the chosen method.Eighteen articles (13.3%) did not contain any information at all about the research instrument(s).Many of those that did mention the research instrument(s) failed to follow the correct structure by excluding items.The use of qualitative research concepts varied considerably and demonstrated a certain degree of superficiality in the use of terms.In the same article for example, the authors refer to the research instrument as both interview and semi-structured questionnaire.It is also important to highlight that 11 articles (8.1%) concerning empirical studies do not mention inclusion and exclusion criteria.
Transparency of procedures -certain authors mistakenly treated the description of the methodological path as a theoretical treatise about the method used.For example, long definitions of social representation and only a few lines dedicated to the actual methodological path itself in order to understand the research question.This problem was repeated in the references, the majority of which were about the chosen method rather than the research topic.Another inconsistency observed in this item concerns data collection.Very few articles informed who collected the data, which prevents the reader from evaluating the researcher/researched relationship.The articles that described sampling criteria used the term "data saturation" to talk about the number of interviewees without considering the meaning and theoretical and practical application of this concept.
A good description of the study setting and field entry strategy makes results more understandable and affords them more credibility.Only 11 (8.1%) of the studies did not score on this item.Thirty-three (24.4%) articles did not describe how dialogue was conducted in the field, while 46 (34.1%) failed to describe how the information was recorded.On the other hand, one article explains how the method used was changed after data collection had started upon realizing that the instrument was not appropriate to the objectives, thus demonstrating a search for transparency and quality (ESPÍNDOLA;BLAY, 2009).
Ethical aspects were absent in 29 (21.5%)articles about studies involving human beings.In a number of articles, part of the description of the methodology -for example, information about the interview guide, the data saturation criterion, and data analysis strategy -was found in the results section, while others mentioned the use of specific research techniques without providing evidence of its correct use in the results.With respect to studies that used documentary analysis, articles frequently failed to clearly describe the analytical framework.As such, many articles only present a commented overview of what was read.
Soundness of interpretive approach -15 articles (11.1%) do not describe the analytical approach, that is, how the material was organized, classified and analyzed.A total of 105 articles (77.8%) did not consider the historical-spatialsocial context of the information collected in the field, while 117 articles (86.7%) did not consider study limitations in light of aspects such as the advancement of knowledge.In 32 articles (23.7%) interpretations were neither clear nor supported by the evidence and with reference to existing literature.Eleven of the articles were not well written and accessible (8.1%).We found works that did not describe the type of textual analysis and did not develop categories for analysis.Some authors simply present the results and interpret them.One fact that is worth highlighting is that we observed various ways of describing results, which is not necessarily a weakness: certain researchers created categories that resembled the empirical reality, while others used general and abstract categories in their interpretations and others went straight into the discussion of findings without showing that they had gone through a classification process.
With respect to the fields of analysis, certain studies stated that they had adopted a hermeneutic approach, when in reality the study comprised only a thematic analysis.Despite being theoretically separate concepts, thematic analysis and content analysis are treated as synonyms in the majority of studies that adopted these approaches.The term triangulation is also used in different situations: when using various techniques (interview + focus group + observation); when using various methods (qualitative, quantitative, clinical studies); and when professionals from different areas analyze the same question.According to Minayo (2013) and Denzin (1970), the expression "triangulation" is also used polysemously in the literature.
Finally, with respect to results, we observed that in some articles authors give emphasis to statements made by participants that have little relevance to and even diverge from the original assumptions This in itself does not constitute a problem: however, these changes in perspective are not commented, problematized or recognized in the studies' interpretive syntheses.

Discussion
Within the standards of scientific soundness, a qualitative article should comply with the same methodological requirements as any other scientific work: clearly defined study question and aims; grounded in related literature and recognized theories; use of appropriate methodological procedures; explicitly stated classification criteria; findings based on empirical data and with reference to existing relevant scientific literature on the topic; and finally, explicitly described limitations.On the other hand, it is important that each stage of the knowledge cycle adheres to the specificities of qualitative research, whose procedures often differ to those of quantitative studies.However, it is necessary to understand the logic behind both approaches and strive for scientific rigor.
With respect to scientific soundness, one of the main criticisms of these methods is that generalization of findings is not the main purpose of qualitative research in view of its uniqueness and knowledge production aims.Although this may be true, Collingridge and Gantt (2008), from Intermountain Healthcare in Salt Lake City, Utah suggest that it is actually possible to generalize the findings of studies carried out in similar contexts with a certain level of confidence, in the same way as a precedent is set in law based on a generalization established by analyzing the similar characteristics of individual cases.
Another common criticism from those who work with quantitative methods is lack of reliability, given that qualitative research does not aim to be reproducible.In this respect, Collingridge and Gantt (2008) underline that qualitative research does not underestimate the importance of finding similar results in similar settings and that this generally occurs.However, the fact that there are divergences does not mean that the qualitative method is not reliable, since qualitative research does not seek the truth, which, often mistakenly, is treated as a quantification of reality.Qualitative research incessantly seeks to understand and interpret, as faithfully as possible, the internal logic of the study participants and make their "truth" known (MINAYO, 2012b).Differences in interpretation frequently reflect a multifaceted understanding of complex social phenomena.Some authors measure the effectiveness of their research through the surprises they encounter when they go to the field (OLIVEIRA, 2012), always with reference to other studies when they exist.
Our analysis found a higher proportion of consistent articles than that found by other authors who have carried out similar reviews.A systematic review of 58 articles on qualitative studies about family members' perceptions of anorexia and bulimia conducted by Espíndola and Blay (2009) from the Department of Psychiatry at the Paulista School of Medicine found that only nine (less than 20%) were in line with similar criteria to the RATS guidelines with respect to methodological aspects.
With respect to the not very consistent studies, simple interpretations in badly designed studies have been highlighted by various social scientists, including Minayo (2012a), Gomes andSilveira (2012), andCanesqui (2011).The various problems mentioned above could be minimized if the authors received better training that includes the theoretical and philosophical foundations of qualitative research and how to contextualize studies.The development of qualitative health research depends greatly on the skills and competencies of the researcher.Therefore, the polysemy that arises from the superficial use of concepts and reduction of technical methods are considered a crucial problem of qualitative research (BOSI, 2012).Those who do not have a good understanding of the concepts they are working with will have difficulty creating appropriate instruments that ensure intersubjective field work and a comprehensive and interpretative analysis (COHN, 2013;MINAYO, 2013;ONOCKO-CAMPOS, 2011).Bradley, Curry and Devers (2007), from the Department of Epidemiology and Public Health of the School of Medicine at Yale University, suggest that scientific journals play an important role in this respect and agree with the present study's conclusion that clear and open instructions tailored to the specificities of this approach are necessary.
Corroborating the limitations encountered by the present study, a review of qualitative research on health services conducted by Weiner et al. (2011) from the School of Global Public Health at the University of North Carolina, Chapel Hill showed that half of the articles contained little or no details about the key aspects of the method and that only 17% of the articles described how data was collected and analyzed.This lack of clarity contributed towards this type of study being looked down upon.For example, with respect to procedural rigor, very few studies address questions such as the relation between the researcher and study participants, leaving doubts as to the ethical nature of the study and ignoring the implications of this relationship, principally when researchers study their own patients (BRADLEY; CURRY; DEVERS, 2007).
The lack of soundness/completeness of the interpretive approach, responsible for unraveling the "internal logic of the study participants", is another weakness of many of the qualitative articles.However, Yamazaki et al. (2009), professors at the Faculty of Humanities and Sociology at the University of Tokyo, found worse results.At least 41% of the articles they analyzed failed to provide information about the kind of analysis performed.Sofaer (2002), researcher at the School of Public Affairs in New York, poses that data analysis is a major challenge since data obtained in the field is typically suggestive and rarely conclusive.However, the author suggests that the analysis process should be highly deliberate and systematic.Within the same line of thought, Sandelowski and Barroso (2003), from the University of North Carolina, describe a continuum of qualitative findings that involves quality -"no finding", "topical survey", "thematic survey", "conceptual/thematic description", and "interpretive explanation" -and is thus able to differentiate between consistent and inconsistent articles.
It is important to highlight that evidence shows that medical journals are increasingly open to accepting qualitative articles for a number of reasons, two of which are fundamental: the recognition of the relevance of social and psychological factors in the development of diseases; and increasing awareness of their rights among study participants, who increasingly voice their demands in the solution to their problems (MINAYO, 1998). However, Shuval et al. (2011), also from the School of Public Affairs in New York, argue that the proportionate increase in qualitative studies and publications is still very small.In this respect, scientific editors play an essential role.Devers (1999), researcher from the Agency for Health Care Policy and Research in Rockville, highlights that traditional criteria for evaluating qualitative research are rooted in the positivist paradigm and closely associated with statistical and mathematical demands.Devers suggests that adequate guidance for inducing and evaluating qualitative health research would strengthen this field and facilitate the funding and publication of the knowledge produced by these studies.Journals that adopt editorial policies that include qualitative articles publish more and offer a better chance of author improvement.
Therefore, although the number of articles remains small and it is important to ensure that reviewers have a comprehensive conceptual and practical understanding of qualitative research methods, it is commendable that highimpact journals like the Lancet and JAMA are open to qualitative studies (COLLINGRIDGE; GANTT, 2008).

Table 2 .
Number of articles by journal and major area of knowledge