The importance of analysis in published evidence levels

EDITORIAL

The importance of analysis in published evidence levels

Wanderley Marques Bernardo

Coordinator of the Projeto Diretrizes AMB-CFM and Professor of Evidence-Based Medicine of Faculdade de Medicina de Santos - UNILUS, Santos, SP, Brazil

"Critical analysis of the use of statistical tests in Brazilian publications related to digestive tract surgery"¹ analyzes the strength of evidence and use of statistical tests in two periods, 1987 and 2007, in national gastroenterological surgical publications. It concluded that: an increase in animal studies (from 3.33% to 19.7%) was observed; the distribution of studies according to the strength of the evidence remained intact (randomized assays [less than 3%] and non-randomized, prospective or historical cohorts, and case control studies); and the use of statistical tests increased from 40% to 70%, despite a significant impact in the adequate use having not been observed.

Despite implicit limitations in the evaluation of the two periods, this analysis indicates how scientific information generation in surgical gastroenterology has been among us. This is done with an explicit and reproducible method, in a clear, objective, and understandable way, providing elements for a few reflections, which are exposed below.

Whenever we approach the subject "level or strength of the evidence published", it should be done with at least two points of view: regarding the consequences of the clinical decision and the consequences of the evaluation of the publisher.

The strength of the evidence translates the level of uncertainty, inherent to the inferences of publications regarding the effects on patients by described or executed interventions². Statistical tests should only be useful when the level of uncertainty is small, confirming or not the applicability, with expectations of practical results similar to those of the study. The consequences of publication-based decision making is directly proportional to the level of their uncertainties and, therefore, the lower the strength the greater the risk of an unpredictable result regarding the effects on patients.

Currently, assessing the quality of the publisher and/or author is limited to a series of superficial indicators³, such as the number of quotes obtained along a determined period of time or publication in international journals, regardless of the strength of the evidence of said publications or the strength of evidence of the publications that quote them. Proposals of analysis like that of the work of Orso IRB et al.¹, reach some stakeholders and have a few consequences:

1. On readers: they should be able to identify the level of uncertainty of the published evidence when considering them in their decisions;

2. On peer review: should make explicit the strength of evidence related to publications approved;

3. On quality reviewers: they should review their indicators, giving the appropriate weight to publishers and/or authors depending on the strength of the evidence published;

4. On authors: they should make an effort to produce and spread information with an elevated level of evidence strength, protecting patients of dubious decisions regarding the outcome;

5. On research backers: they should support and stimulate investigators capable of developing practicable projects to answer each category of clinical question;

6. On universities: they should strengthen, through post-graduation programs, initiatives that contemplate the production of relevant, new, strong, and applicable products, stimulating publication in national journals;

7. On publishers: they should occupy their place as opinion makers, contributing appropriately with the health system, since they are responsible for making public consistent, and inconsistent, scientific information.

This study by Orso IRB et al.¹ reproduces several international experiences^4-7, with variations on the distribution of the proportions of strength of the evidence published, which are directly responsible for the need of appropriate statistical tests. The deficiencies of the statistical tests used are not a major problem, since they are the effect, and not the cause, of the weakness of the evidence generated⁸.

This example of quality assessment should be followed and valued, since it strengthens the notion of a Medicine that considers patient experience and values, but also the scientific evidence that permeates this physician-patient relationship.

References

• 1. Nobre MR, Bernardo WM, Jatene FB. Evidence based clinical practice. Part III - Critical appraisal of clinical research. Rev Assoc Med Bras. 2004; 50:221-8.
• 2. Kuroki LM, Allsworth JE, Peipert JF. Methodology and analytic techniques used in clinical research: associations with journal impact factor. Obstet Gynecol. 2009; 114: 877-84.
• 3. Hopewell S, Dutton S, Yu LM, Chan AW, Altman DG. The quality of reports of randomised trials in 2000 and 2006: comparative study of articles indexed in PubMed. BMJ. 2010; 340:c723.
• 4. Chan AW, Altman DG. Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. BMJ. 2005; 330:753.
• 5. Chan AW, Altman DG. Epidemiology and reporting of randomised trials published in PubMed journals. Lancet. 2005; 365:1159-62.
• 6. Dauphinee L, Peipert JF, Phipps M, Weitzen S. Research methodology and analytic techniques used in the Journal Obstetrics & Gynecology. Obstet Gynecol. 2005; 106:808-12.
• 7. Gurusamy KS, Gluud C, Nikolova D, Davidson BR. Assessment of risk of bias in randomized clinical trials in surgery. Br J Surg. 2009; 96:342-9.
• 8. Orso IRB, Pereira JCR, Albuquerque LACD, Cecconello I, Jukemura J. Critical analysis of the use of statistical tests in Brazilian publications related to digestive tract surgery. Rev Assoc Med Bras. 2011; 57(1):35-41.

Publication Dates

• Publication in this collection
  25 Feb 2011
• Date of issue
  Feb 2011