The appraisal of medical scientific evidence

EDITORIAL

The appraisal of medical scientific evidence

Álvaro Nagib Atallah

MD, PhD, MSc, Associate professor of Internal Medicine of Escola Paulista de Medicina, President of the Latinclen Latin American Clinical Epidemiology Network

Any doctor would prefer to base his therapeutic approach on the best existing scientific evidence. We could name it MABBEE - Medical Approach Based on the Best Existing Evidence. The problem is how to define what is good scientific evidence and what is not. We understand that nowadays there is reasonable consensus on accountable methodologies for the critical appraisal of medical literature. The reader may easily find them in the literature of these last two decades (1,2,3,4,5,6), and improve his/her skills through the American College of Physicians Journal Club, which is a supplement to the Annals of Internal Medicine.

However, it is important to point out some features of the model of the studies which generate the evidences to allow the development of "MABBEE medicine". The randomized double-blind control trial with an adequate sample size and quality control is the ideal design to compare placebo to any treatment, or different types of clinical or surgical treatments. Although this seems obvious, it has still not been widely applied to medical practice in this country. Reasoning based on experience, case reports, and retrospective studies are still very common in Medical Journals and meetings.

Another method of obtaining good evidence is the systematic review of medical treatments followed by a statistical summary named meta-analysis. This type of research should include all the published and unpublished controlled clinical trials with good standards. The typical odds ratio - the final result - usually represents the best evidence regarding that specific treatment until the moment the meta-analysis was properly completed. However the consistency of the results depends on the clinical trials found in the literature. Publication bias, which is the tendency by some journal editors not to publish clinical trials with negative results, affects not only the authors submitting the papers but also the systematic review preparation, and finally the doctors interested in the best evidence-based medicine, which, as a consequence, affects the patients.

Prospective cohort studies are much better than retrospective studies, because they allow for stronger conclusions. A prospective study performed in developing countries has better chances of earning confidence.

Diagnostic tests must be assessed regarding their utility. That means the adoption of an excellent gold standard, definition of cut-oft points, calculation of sensitivity, specificity, and predictive values, and describing confidence intervals.

Comparisons with a traditional diagnostic test (Receiver Operating Characteristics - ROC) can also be made. For more details the reader may be referred to (2) and (6).

In other words, good clinical research depends on the use of clinical epidemiology and the adherence to very well-designed studies, which take into account the prevention of bias and the application of appropriate statistical methods.

When reading a scientific paper try to identify the following points:

1. Was there a clear and simple research question?

2. Was the study design appropriate to answer the question?

3. Were the inclusion and exclusion criteria of studied patients clear?

4. Were there acceptable definitions for the diagnoses and the outcomes?

5. Were the sample size and the statistical power calculated?

6. Was there good quality control of, for example when analyzing clinical trial: frequency of drop-outs, adherence to the treatment and to the randomization process, co-intervention, and contamination prevention?

7. Were the statistical analyses appropriate to answer the more relevant aspects of the research question?

8. Were the conclusions based exclusively on the study results?

We come back again to the critical appraisal of medical literature.

So, in order to practice MABBEE Medicine, or Evidence-Based Medicine, all doctors — and not only Medical Journal editors — will need to be familiar with the essentials of clinical epidemiology in the future.

• 1. CASTELO, A; SESSO, R; ATALLAH, A. Epidemiologia Clínica: Uma ciência básica para o clínico J Pneumol. 15(2): 89-98, 1988.
• ²
  Dep. of Clinical Epidemiology and Biostatistic MacMaster University Can Med Assoc J. How to read clinical journals. 124:555-558, 703-710, 869-879, 985-990, 1156-1162, 1981.
• 3. BENNETT RJ, SACKETT DL, HAYNES RB, NEUFELD UR, TUGWELL P, ROBERTS R. A controlled trial of teaching critical appraisal of the medical literature to medical students. JAMA 257:2451-2454 1987.
• 4. FLETCHER RH, FLETCHER SW. Clinical research in general medical journals: a 30-year prospective N Engl J Med. 301:180-183, 1979
• 5. POCOCK SJ HUGHES MD, LEE RJ. Statistical problems in the reporting of Clinical Trials. N Engl J Med. 317:426-432, 1987.
• 6. SACKETT, D HAYNES RB; TUGWELL P. Clinical Epidemiology. A basic science for Clinical Medicine. Little, Brown and Company ed. Boston. 1985.

Publication Dates

• Publication in this collection
  29 May 2009
• Date of issue
  June 1995