Observational studies: why are they so important?

Estudos observacionais: por que são tão importantes?

Randomized clinical trials (RCTs) are known to be the gold standard for medical research, and this study design is preferred for investigating the efficacy of new interventions. Many authors and editors believe that the RCT design always surpasses other research designs. However, this is not an unquestionable truth.

The main advantage of RCTs is that they provide better control over possible bias through randomization and blinding. In other words, the great strength of an RCT is its high internal validity. On the other hand, rigid design control could reduce the ability to generalize the results.¹ Another issue relating to RCTs is the fact that recruitment, randomization and blinding are not always possible because of technical issues (e.g. surgical procedures) or ethical issues (e.g. in a hypothetical trial to prove the association between smoking and lung cancer, it would be unethical to randomly assign a group to smoke).

But what should be done when RCTs are unfeasible or unethical? The answer could be a well-designed observational study. This term describes a range of study designs that includes cross-sectional, case-control, prospective and retrospective cohort studies. The main characteristic of observational studies is that any intervention is not determined by the protocol, but by clinical practice.²

Cross-sectional studies are used to estimate prevalence. They are relatively quick and cheap, and can be used to study multiple outcomes. However, they do not differentiate between cause and effect or within the sequence of events. They are useful for identifying associations that can then be more rigorously studied using a cohort study or randomized controlled study. The most important problem with this type of study is in differentiating between cause and effect from a simple association.³

Case-control studies are considered to be simple to organize and useful for hypothesis generation. They retrospectively compare two groups in order to identify predictors of an outcome. They allow calculation of odds ratios. The disadvantages are that they can only evaluate one outcome and that the presence of bias is usually high and difficult to assess. The major difficulty could lie in determining an adequate control group.³

Cohort studies are excellent for estimating the incidence and natural history of a condition. They may be prospective or retrospective and sometimes two cohorts are compared. They analyze predictors (risk factors) that enable relative risk calculation. Since they measure events in temporal sequence, they can distinguish causes from effects. Retrospective cohorts are considered to be cheaper and quicker, but may have fragile results, particularly if the database is inadequate. Prospective cohorts are more accurate and, with a good protocol (with adequate sample size and follow-up) can have results that are as reliable as those of RCTs.³

The main problem in observational studies is the presence of confounders and selection bias (which are prevented in RCTs through randomization and blinding). A confounder can be defined as any factor that is related not only to the intervention (e.g. treatment) but also to the outcome and could affect both.⁴ One good example is age: in a study on the relationship between smoking (exposure) and lung cancer (outcome), age could be implicated as a factor that would increase the incidence of the outcome. Thus, if one of the groups (smokers or non-smokers) has an older population, the increase in lung cancer could be influenced by age (as a confounder), and not by the exposure studied.

However, advanced statistical tools may enable good and reliable control over many confounders. Some tools like propensity scores and sensitivity analysis, when correctly performed, could drastically reduce the bias caused by the lack of randomization.⁵

Some authors have studied the results from RCTs, compared with similar observational studies. Concato et al. published an evaluation of meta-analyses that compared outcomes between RCTs and observational studies and reached the following conclusion: "The results from well-designed observational studies (with either a cohort or a case-control design) do not systematically overestimate the magnitude of the effects of treatment, as compared with those in randomized, controlled trials on the same topic".⁶ In other words, if the observational study has good methodological quality, the results are quite similar.

Many investigators have pointed out that the main strength of observational studies is their greater proximity to "real life situations", since RCTs have stricter inclusion criteria and rigid protocols that may not reflect clinical practice. By definition, observational studies have greater heterogeneity of medical interventions and patient populations that are closer to clinical practice.² Other advantages of observational studies are that they are usually cheaper than RCTs and can be used to investigate rare outcomes and to detect unusual side effects, and that some designs are easily and quickly performed.

Observational studies also are important for creating new hypotheses, proving the external validity of RCTs already performed, establishing the sample size for an RCT and evaluating which patient subsets really benefit from each alternative intervention of effective alternative therapies.¹ In this way, it can be said that observational studies can be complementary to RCTs.

Although the evidence level of observational studies appears to be lower than that of RCTs, it is clear that this kind of investigation is crucial for elucidating many scientific questions. Not only authors but also editors around the world are giving more attention to these studies. The take-home message is that the study question and the quality of the methodology applied to answer it are much more important than the study design. In this context, observational studies may be the best way to answer the many medical questions in situations in which the classical RCT approach does not apply.

REFERENCES

• ¹
  Hannan EL. Randomized clinical trials and observational studies: guidelines for assessing respective strengths and limitations. JACC Cardiovasc Interv. 2008;1(3):211-7.
• ²
  Yang W, Zilov A, Soewondo P, et al. Observational studies: going beyond the boundaries of randomized controlled trials. Diabetes Res Clin Pract. 2010;88 Suppl 1:S3-9.
• ³
  Mann CJ. Observational research methods. Research design II: cohort, cross sectional, and case-control studies. Emerg Med J. 2003;20(1):54-60.
• ⁴
  McNamee R. Confounding and confounders. Occup Environ Med. 2003;60(3):227-34; quiz 164, 234.
• ⁵
  Joffe MM, Rosenbaum PR. Invited commentary: propensity scores. Am J Epidemiol. 1999;150(4):327-33.
• ⁶
  Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med. 2000;342(25):1887-92.

Publication Dates

• Publication in this collection
  2014

History

• Received
  23 Oct 2013
• Accepted
  13 Nov 2013