Adoption of protocols to improve quality of medical research

The primary means of dissemination and sharing of scientific research results are publications in specialized journals. The quality and relevance of the investigation are assessed, among others, by the material that was published, which is usually the single public register of the research. Problems in this communication hinder the correct evaluation of a study and limit its effectiveness. Such a situation becomes critical becomes critical in a research field such as the health sciences, since it allows wrong decisions to me made by professionals, and consequently, a real potential of harm to the patients.⁽¹⁾

Medical science requires evidence to identify problems, evaluate the accuracy of the diagnoses or prognoses, compare and assess interventions, describe their adverse or rarest effects, evaluate if an early detection test is really necessary, as well as how to compare intervention costs, among so many other factors. Scientific evidence is produced by means of several experimental approaches and, in general, but not only, by adopting study formats, such as observational studies, randomized studies with intervention, case reports, systematic reviews with meta-analysis, and the opinion of specialists.⁽²⁾

The strength, power, or level of evidence in health depends on how this evidence was generated. One form of representation proposed for this evaluation is called pyramid of evidence (Figure 1). This scheme relates research designs with the types of data generated for which the levels of evidence are proposed. At the peak of the pyramid, there are meta-analyses, followed by systematic reviews and randomized clinical trials. The meta-analyses were conceived, initially, as a tool to incorporate robustness into the evidence generated by the randomized clinical trials, for allowing aggregation and comparison of results from independent studies about the estimation of effect sizes of a given intervention. Thus, they enable the clarification of questions derived from independent research, since they expressively widen the sample of investigated subject through data synthesis. Since they are able to provide precise responses, they are considered the studies with the highest level of evidence.⁽³⁾

Figure 1
A level of evidence pyramid in medical sciences

The base of the naturally will have a large amount of data. Typically, there are the shorter, more restrictive studies with limited resources, that have smaller samples or with less restrictive methodological demands than the randomized clinical trials. These are the generically called observational studies, which can be case-control, cohort, ecological, and interventional. They have various origins, but can be, for example, those derived from graduate dissertations and theses. One the other hand, when taken as a whole, these data comprise an expressive volume of subjects, sites, settings, social, economic and genetic characteristics, among others, which are susceptible to pooling.⁽⁴⁾ By means of specific methods, also started the use of observational studies in meta-analyses for decision-making. With this, studies typically from the base of the pyramid, with a low level of evidence and limited impact, began to provide to provide data for the peak of the pyramid, in systematic reviews and meta-analyses.⁽⁵⁾

Nevertheless, one of the greatest challenges for the use of these data is their large heterogeneity, considering the experimental design, report form, selection of research subjects, inclusion and exclusion criteria, and analytical instruments, or the studied populations. In addition to the particularities inherent to smaller and local studies, it is estimated that 85% of data⁽¹⁾ are wasted due to their low quality, among other reasons.⁽⁵⁾ Therefore, efforts have been made to to increase value to these data and to decrease the information loss, i.e., data that could confer more robustness to the meta-analyses.⁽⁶⁾

In Brazil, scientific production is mostly associated with the research derived from post-graduation programmes. In its majority, there are severe budget constraints, they tend to cover local and regional themes, and therefore, they have low appeal to international - and even to national audiences. This statement is corroborated by the low citation rates of local productions, eloquently exemplified by the fact that 80% of articles published in the field of education in Brazil are not even mentioned by other local projects in the same area.⁽⁷⁾

Whereas in the field of Medicine, Brazil is prolific in terms of absolute volume of scientific production, with 19,636 articles published in 2016, and the 14^th country in a productivity ranking in which 130 countries participate. Nevertheless, during the same year, the number of citations per document in the area of medicine was 2.88. Such a fact bestows on Brasil the 107^th position in the same ranking in number of citations per article published.⁽⁷⁾

In addition to the low rate of citation, and sometimes, of the poor quality, the Brazilian scientific production still suffers with the great concentration at poles. According to data from The São Paulo Research Foundation (FAPESP - Fundação de Amparo à Pesquisa do Estado de São Paulo), more than half the national scientific production comes from the State of São Paulo. In 2017, there were 55,051 publications recorded with researchers of Brazilian institutions, and 42% of them presented with authors from the city of São Paulo. If the State of São Paulo were a country, it would rank 23^rd in scientific production.⁽⁸⁾ Normalizing the world mean of citations per publication, which is an index of visibility and impact of publications, São Paulo (1.06) maintains its position above Brazil (0.88),⁽⁸⁾ and still is outstanding by its international collaboration.⁽⁹⁾

Even with its expressive increase in granting master and doctorate titles during the last decade, there was an increase in the production of articles, but there was no improvement of their quality.⁽¹⁰⁾ This is confirmed by the fact that the number of citations of the Brazilian scientific articles is inferior to that of countries with science-designated budgets lower than those of Brazil.⁽¹¹⁾

Nonetheless, there are efforts for improving Brazilian scientific production. Enterprises such as the Brazilian Reproducibility Initiative seek to verify the reproducibility of biomedical studies in various research centers in Brazil.⁽¹²⁾ It is essential that there be a revision of the subject matters and designs of the studies, without disfavoring issues of national interest, but that allow the reproducibility of the study itself, which cooperates to the increase in quality of Brazilian science. Another demand for the refinement of scientific production is the adoption of recommendations of good scientific practices, and increased transparency in the process, in accordance with the growing recommendations recommendations from the international by the international scientific community.⁽¹²⁾

In this scenario, growing prominence is given to research protocols, which are documents aiming to the standardization and, which are the documents that seek standardization and enhancement of the quality of the most diverse types and modalities of scientific research. The use of a protocol can drive data collection, the manner of describing and reporting the data, and even the way in which to structure the investigation from its conception. The protocols recommend which elements and aspects should be observed and considered in research and in its report.⁽¹³⁾ Besides specifying and considering the different types of research, data reveal an increase in the quality of the articles that follow protocols.⁽¹⁴⁾ Thus, their adoption becomes indispensable for the refinement of national scientific production.

There are several research protocols for different types of studies, with the objective of encompassing demands for the concept of adequate and reproducible experimental study designs. It is important to point out that among the first protocols proposed, some arrived as a demand for studies in the highest levels of the pyramid of evidence. Therefore, taking into account the main types of studies described therein, those that worth mention are the PRISMA,⁽¹⁵⁾ recommended for the performance of systematic reviews, with or without meta-analysis; CONSORT,⁽¹⁶⁾ for randomized clinical trials; STROBE,⁽¹⁷⁾ for cohort and case-control studies; and finally, CARE,⁽¹⁸⁾ for case report studies. Many others are available, including those encompassing specificities of areas and subareas of research.

Seeking dissemination of information, and as a form of incentive for the use of the protocols, the EQUATOR Network (Enhancing the Quality and Transparency of Health Research) was launched in 2008.⁽¹⁾ Currently, EQUATOR is headquartered at the University of Oxford, in the United Kingdom, but also has received financial support from organizations, such as the World Health Organization (WHO), Pan American Health Organization (PAHO), National Health Services (NHS), and National Institutes of Health (NIH), being endorsed and recommended by transparency, ethics, and good practices in scientific production and publication agencies, such as the Committee on Publication Ethics (COPE) and the International Committee of Medical Journal Editors (ICMJE), as well as large editorial conglomerates such as BioMed Central, The Lancet, British Medical Journal, PLoS, among others of high and recognized international reputation.⁽¹⁹⁾ Enhancing the Quality and Transparency of Health Research consists, therefore, of an international collaboration that “aims to improve reliability and value of published literature on health research, promoting transparent and precise reports, and a wider use of robust guidelines in reports.”⁽²⁰⁾

In this way, the primary data obtained in observational studies from the base of the pyramid can be better utilized, considered, and therefore, cited when adopting rigorous methodological practices available and in current use. Despite the protocols not guaranteeing the quality of the research per se (e.g., equipment, supplies, and processes), they reinforce the fact that the data were obtained in a premeditated manner, following the best practices adopted by the international scientific community. When their presentation is standardized, relevant information is not omitted, lost, or diluted throughout the text. Thus, they can be better evaluated by reviewers and readers, and will constitute the base of the pyramid of evidence with greater chances of being cited in other studies, especially international ones.⁽²¹⁾ The data may be limited or regional, but it should be of interest and be useful for international science when, for example, such research is selected to compose a synthesis study of scientific literature (systematic review and meta-analysis). For this, it must be strict. Undoubtedly publishing is necessary, but to have an article citation is an acknowledgment of its relevance.

Thus, the adoption of internationally recognized and standardized protocols can foster the strengthening of scientific production and of the dialog among different national and international researchers, increasing the level of citation and acknowledgment of Brazilian science.

REFERENCES

• ¹
  Simera I, Altman DG, Moher D, Schulz KF, Hoey J. Guidelines for reporting health research: the EQUATOR network's survey of guideline authors. PLoS Med. 2008;5(6):e139.
• ²
  Centre for Evidence-Based Medicine (CEBM). OCEBM Levels of Evidence. Oxford Centre for Evidence-Based Medicine 2011 Levels of Evidence [Internet]. Oxford (UK): CEBM; 2011 [cited 2019 Aug 13]. Available from: http://www.cebm.net/index.aspx?o=5653
  » http://www.cebm.net/index.aspx?o=5653
• ³
  Gurevitch J, Koricheva J, Nakagawa S, Stewart G. Meta-analysis and the science of research synthesis. Nature. 2018;555(7695):175-82. Review.
• ⁴
  Clinical Trial Service Unit & Epidemiological Studies Unit CTSU). Nuffield Department of Population Health. Meta-analyses of observational studies [Internet]. University of Oxford (UK); 2019 [cited 2019 Aug 13]. Available from: https://www.ctsu.ox.ac.uk/research/metaanalyses-of-observational-studies
  » https://www.ctsu.ox.ac.uk/research/metaanalyses-of-observational-studies
• ⁵
  Ioannidis JP. How to make more published research true. PLoS Med. 2014; 11(10):e1001747.
• ⁶
  Macleod MR, Michie S, Roberts I, Dirnagl U, Chalmers I, Ioannidis JP, et al. Biomedical research: increasing value, reducing waste. Lancet. 2014;383(9912):101-4.
• ⁷
  Scimago Journal & Country Rank (SJR). Scimago Journal & Country Rank [Internet]. 2019 [cited 2019 Aug 14]. Available from: https://www.scimagojr.com/countryrank.php?category=3304&order=cd&ord=desc&min=100&min_type=it&year=2016
  » https://www.scimagojr.com/countryrank.php?category=3304&order=cd&ord=desc&min=100&min_type=it&year=2016
• ⁸
  Cross D, Thomson S, Sibclair A. Research in Brazil: A report for CAPES by Clarivate Analytics [Internet]. Clarivate Analytics; 2018 [cited 2019 Aug 14]. Available from: http://www.capes.gov.br/images/stories/download/diversos/17012018-CAPES-InCitesReport-Final.pdf
  » http://www.capes.gov.br/images/stories/download/diversos/17012018-CAPES-InCitesReport-Final.pdf
• ⁹
  Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP). Publicações científicas e colaborações internacionais [Internet]. Rev Pesquisa. 2019 [citado 2019 Ago 14]. Disponível em: https://revistapesquisa.fapesp.br/2019/06/07/publicacoes-cientificas-e-colaboracoes-internacionais/
  » https://revistapesquisa.fapesp.br/2019/06/07/publicacoes-cientificas-e-colaboracoes-internacionais/
• ¹⁰
  Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP). Matrículas e títulos de doutorado concedidos no país [Internet]. Rev Pesquisa. 2018 [citado 2019 Ago 14]. Disponível em: https://revistapesquisa.fapesp.br/2018/01/16/matriculas-e-titulos-de-doutorado-concedidos-no-pais/
  » https://revistapesquisa.fapesp.br/2018/01/16/matriculas-e-titulos-de-doutorado-concedidos-no-pais/
• ¹¹
  Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP). Internacionalização das publicações científicas [Internet]. Rev Pesquisa. 2017 [citado 2019 Ago 14]. Disponível em: https://revistapesquisa.fapesp.br/2017/05/23/internacionalizacao-das-publicacoes-cientificas/
  » https://revistapesquisa.fapesp.br/2017/05/23/internacionalizacao-das-publicacoes-cientificas/
• ¹²
  Andrade RO. Brazilian biomedical science faces reproducibility test. Nature. 2019;569(7756):318-9.
• ¹³
  Altman DG, Simera I, Hoey J, Moher D, Schulz K. EQUATOR: reporting guidelines for health research. Lancet. 2008;371(9619):1149-50.
• ¹⁴
  Pussegoda K, Turner L, Garritty C, Mayhew A, Skidmore B, Stevens A, et al. Systematic review adherence to methodological or reporting quality. Syst Rev. 2017;6(1):131.
• ¹⁵
  Moher D, Liberati A, Tetzlaff J, Altman DG; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097.
• ¹⁶
  Schulz KF, Altman DG, Moher D; CONSORT Group. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c332.
• ¹⁷
  von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP, et al. STROBE Initiative. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: guidelines for reporting observational studies. Int J Surg. 2014;12(12):1495-9.
• ¹⁸
  Gagnier JJ, Kienle G, Altman DG, Moher D, Sox H, Riley D; Care group. The CARE guidelines: consensus-based clinical case reporting guideline development. BMJ Case Rep. 2013;2(5):38-43.
• ¹⁹
  Equator network. Enhancing the QUAlity and Transparency Of health Research. EQUATOR Network: what we do and how we are organised [Internet]. Equator network; 2019 [cited 2019 Aug 14]. Available from: https://www.equator-network.org/about-us/equator-network-what-we-do-and-how-we-are-organised/
  » https://www.equator-network.org/about-us/equator-network-what-we-do-and-how-we-are-organised/
• ²⁰
  Equator network. Enhancing the QUAlity and Transparency Of health Research. About us [Internet]. 2019 [cited 2019 Ago 14]. Available from: http://www.equator-network.org/about-us/
  » http://www.equator-network.org/about-us/
• ²¹
  Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014;383(9913):267-76.

Publication Dates

• Publication in this collection
  20 Dec 2019
• Date of issue
  2020