Assessing the risk of bias in randomized controlled trials in the field of dentistry indexed in the Lilacs (Literatura Latino-Americana e do Caribe em Ciências da Saúde) database

Avaliação do risco de viés de ensaios controlados randomizados de odontologia indexados na base de dados Lilacs (Literatura Latino-Americana e do Caribe em Ciências da Saúde)

ORIGINAL ARTICLE

Assessing the risk of bias in randomized controlled trials in the field of dentistry indexed in the Lilacs (Literatura Latino-Americana e do Caribe em Ciências da Saúde) database

Avaliação do risco de viés de ensaios controlados randomizados de odontologia indexados na base de dados Lilacs (Literatura Latino-Americana e do Caribe em Ciências da Saúde)

Christiane Alves Ferreira^I; Carlos Alfredo Salles Loureiro^II; Humberto Saconato^III; Álvaro Nagib Atallah^IV

^IMSc. Doctoral student in the field of health sciences at Universidade Federal de São Paulo - Escola Paulista de Medicina (Unifesp-EPM), São Paulo, Brazil

^IISpecialist and doctoral student in the field of health sciences at Universidade Federal de São Paulo - Escola Paulista de Medicina (Unifesp-EPM), São Paulo, Brazil

^IIIMD, PhD. Adjunct Professor at Universidade Federal do Rio Grande Norte (UFRN), Natal, Rio Grande do Norte, Brazil

^IVMD, PhD. Titular Professor and Director of the Brazilian Cochrane Center, Universidade Federal de São Paulo - Escola Paulista de Medicina (Unifesp-EPM), São Paulo, Brazil

^{Address for correspondence}

ABSTRACT

CONTEXT AND OBJECTIVE: Well-conducted randomized controlled trials (RCTs) represent the highest level of evidence when the research question relates to the effect of therapeutic or preventive interventions. However, the degree of control over bias between RCTs presents great variability between studies. For this reason, with the increasing interest in and production of systematic reviews and meta-analyses, it has been necessary to develop methodology supported by empirical evidence, so as to encourage and enhance the production of valid RCTs with low risk of bias. The aim here was to conduct a methodological analysis within the field of dentistry, regarding the risk of bias in open-access RCTs available in the Lilacs (Literatura Latino-Americana e do Caribe em Ciências da Saúde) database.

DESIGN AND SETTING: This was a methodology study conducted at Universidade Federal de São Paulo (Unifesp) that assessed the risk of bias in RCTs, using the following dimensions: allocation sequence generation, allocation concealment, blinding, and data on incomplete outcomes.

RESULTS: Out of the 4,503 articles classified, only 10 studies (0.22%) were considered to be true RCTs and, of these, only a single study was classified as presenting low risk of bias. The items that the authors of these RCTs most frequently controlled for were blinding and data on incomplete outcomes.

CONCLUSION: The effective presence of bias seriously weakened the reliability of the results from the dental studies evaluated, such that they would be of little use for clinicians and administrators as support for decision-making processes.

Key-words: Randomized controlled trial [publication type]. Selection bias. Double-blind method. Evidence-based dentistry. Empirical research

RESUMO

CONTEXTO E OBJETIVO: Ensaios controlados randomizados (ECRs) bem conduzidos representam o mais alto nível de evidência quando a pergunta de pesquisa é sobre o efeito de intervenções terapêuticas ou preventivas. No entanto, o grau de controle de viés entre os ECRs apresenta grande variabilidade entre estudos. Por esta razão, com o aumento do interesse e produção das revisões sistemáticas e metanálises, foi necessário desenvolver metodologia suportada por evidência empírica, para incentivar e valorizar a produção de ECRs válidos e com baixo risco de viés. O objetivo deste trabalho foi realizar uma análise metodológica da área de odontologia quanto ao risco de viés de ECRs de acesso aberto, disponibilizados no banco de dados do Lilacs (Literatura Latino-Americana e do Caribe em Ciências da Saúde).

TIPO DE ESTUDO E LOCAL: Trata-se de um estudo sobre metodologia conduzido na Universidade Federal de São Paulo (Unifesp) que avaliou o risco de viés dos ECRs, utilizando as seguintes dimensões: geração da sequência de alocação, sigilo da alocação, cegamento e dados sobre desfechos incompletos.

RESULTADOS: Dos 4.503 artigos classificados somente 10 (0,22%) estudos foram considerados verdadeiros ECR e, destes, somente um estudo foi classificado como sendo de baixo risco de viés. Os itens mais frequentemente controlados pelos autores dos ECR foram cegamento e dados sobre desfechos incompletos.

CONCLUSÃO: A presença efetiva de viés enfraqueceu seriamente a confiança nos resultados dos estudos de odontologia avaliados, sendo pouco úteis para clínicos e gestores como suporte a processos de decisão.

Palavras-chave: Ensaio clínico controlado aleatório. Viés de seleção. Método duplo-cego. Odontologia baseada em evidências. Pesquisa empírica

INTRODUCTION

The history of the efforts to create unbiased comparison groups for preventive or therapeutic interventions goes back more than three centuries.¹ Randomized clinical trials (RCTs), in the form in which they are known today, were introduced within medical science by Bradford Hill at the end of the 1940s. According to the Glossary of Terms in the Cochrane Collaboration, RCTs are a type of prospective study in which two or more interventions introduced by the investigator are compared. Of these, one is a control group (in which non-intervention, placebo or a standard intervention is used). RCTs use participant allocation sequences that are random.²

Well-conducted RCTs represent the individual highest level of evidence when the research question relates to the effect of therapeutic or preventive interventions. However, the degree of control over bias between RCTs presents great variability between studies. For this reason, with the increasing interest in and production of systematic reviews and meta-analyses, it has been necessary to develop validated tools that might help to evaluate the risk of bias in RCTs. The aim of such tools is to ensure that the clinical question is answered correctly, i.e. with an effect size that is as close as possible to the truth. RCTs with a high risk of bias may overestimate or underestimate the magnitude of interventions, and the misleading nature of such results may compromise clinical decision-making.³

OBJECTIVE

The objective of the present study was to conduct a methodological analysis within the field of dentistry, regarding the risk of bias among open-access RCTs that are available in the Lilacs (Literatura Latino-Americana e do Caribe em Ciências da Saúde) database, which is a regional database covering literature from Latin America and the Caribbean.

METHODS

The studies were retrieved through a hand search in open-access dental journals that are indexed in the Lilacs database and available on the internet (www.bireme.br). For this, a time period of six years was established (2002, 2003, 2004, 2005, 2006 and 2007).

An initial search in Lilacs identified 93 journals within the fields of dentistry or orthodontics. Of these, 74 were current journals, 40 were open-access, seven were available on subscription and the remainder was unavailable in Lilacs. These 40 journals were selected and a hand search was conducted in these, in order to classify the study designs and identify the RCTs (Figure 1). Studies were eligible for inclusion regardless of the participants' ages, clinical conditions, contexts, genders or ethnicity. Any type of intervention carried out on human beings was eligible. Any type of outcome was eligible, and RCTs were not excluded as a function of the relevance or irrelevance of the outcomes. The main criterion for a more detailed assessment was the presence or indication of randomization or random allocation as terms in the title or abstract of study.

Empirical evidence supports at least six dimensions as potential sources of risk of bias in RCTs: randomized allocation sequence,⁴ allocation sequence concealment,⁴ blinding,⁵ evaluation of incomplete outcome data,⁶ selective outcome reporting⁷ and other biases. Adequate procedures for these items are important for ensuring internal validity for RCTs.⁸ The Cochrane Handbook for Systematic Reviews of Interventions recommends a tool that independently assesses these dimensions of RCTs, taking them as isolated domains without attributing weights or different ranks to them. For each of these dimensions, three response options were offered: yes (low risk of bias; score 2), when this dimension was correctly implemented and reported; no (high risk of bias; score 0), when this dimension was not implemented or not reported, or if an invalid method was reported; or unclear risk (unclear risk of bias for one or more domains; score 1), when the implementation method or the report left doubts.²

The classification (Higgins and Green: ² a classification used to summarize the risk of bias according to domains in each study and between studies) made it possible to rank each of the studies individually and to rate the validity of publications for a set of studies, as in the scheme below:

• Low risk of bias: unlikely to be enough biased to seriously change the results; within studies: low risk of bias for all domains; set of studies: most of the information comes from studies with a low risk of bias;

• Unclear risk of bias: plausible bias that raises some doubt about the results; within studies: unclear risk of bias for one or more domains; set of studies: most of the information comes from studies with a low risk of bias;

• High risk of bias: plausible bias that seriously weakens the reliability of the results; within studies: high risk of bias for one or more key domains; set of studies: the proportion of the information coming from studies with a high risk of bias is enough to affect the interpretation of the results.

To assess the risk of bias in studies considered to be potential RCTs, two independent reviewers (CAF and CAL) rated each study in order to ensure that these trials could be reproduced.⁹ Disagreements were discussed until reaching a consensus. The inter-rater reliability between the reviewers was measured using Cohen's kappa statistics. The reliability was considered outstanding: kappa = 0.824.

The statistical analyses were performed using the Statistical Package for the Social Sciences (SPSS) software, version 12.0 for Windows. Multiple logistic meta-regression was used to determine the percentage variance in the dependent variable "statistical significance" of the studies caused by the dimensions of bias. This was taken to be a yes/no category that could be explained by the following independent variables: generation of allocation, allocation sequence concealment, blinding and incomplete outcome data. Only the main outcome of the RCTs was considered, in determining the significance of the study. The method used consisted of free entry of variables and removal of variables that did not fit with the model. The significance level used in the tests was 5% (alpha = 0.05), i.e. tests were taken to be statistically significant with p values less than 5% (P < 0.05).

This study was submitted to the Research Ethics Committee of Universidade Federal de São Paulo - Escola Paulista de Medicina (Unifesp-EPM), under the number 1891/06, and was approved.

RESULTS

Table 1 presents the frequencies and percentages of studies classified in the categories of RCT and other study designs, the frequency and percentage of unclassified studies, the absolute frequency, the annual percentage and the total number of studies evaluated.

The journals evaluated contributed 4,879 articles. Among these, 4,581 could be classified regarding the study design, while no classification was possible for 298 (6.11%). Among the classified studies, 78 (1.73%) were initially classified as potential RCTs, while 4,503 were classified as other types of design. Over the study period, the data indicated a trend of growth in the total number of studies and in the number of RCTs.

Table 2 presents the studies, evaluated according to author, year of publication, classification of risk of bias in the dimensions, classification of the risk of bias in each study, the statistical significance of the main outcome assessed and the frequency of dimensions with a low risk of bias in each study.^10-87

Among the 78 studies evaluated, 77 (98.72%) were classified as presenting a high risk of bias for all dimensions assessed; 6 (7.69%) for three dimensions; nine (11.54%) for two dimensions; and 27 (34.62%) for only one dimension. A single study (1.28%)⁵⁸ was classified as presenting low risk of bias for all of the dimensions. In summary, 77 (98.72%) of the articles were classified as studies presenting a high risk of bias and only one study (1.28%) was classified as presenting a low risk of bias. None of the studies was classified as presenting an unclear risk of bias, since the combination of one unclear dimension and other dimensions with a low risk of bias did not occur.

With regard to statistical significance, 28 studies (35.90%) presented an effect size with a significant difference in favor of the intervention, while there was no significant difference in 40 studies (51.28%) and nine studies (11.54%) were not formally assessed, statistically. All of the studies that controlled for three or more dimensions presented non-significant results.

Figure 2 presents the risk classification graphically, between high, unclear and low risk, for the four dimensions evaluated in the 78 open-access RCTs evaluated.

Among the 78 studies evaluated, the dimension for which the risk of bias was most correctly controlled for was blinding, in which 26 studies (33.33%) presented a low risk of bias. This was followed by controlling for incomplete outcome data, in which 23 studies (29.49%) were classified as presenting a low risk of bias. Ten studies (12.82%) presented a low risk of bias in relation to the method for generating the allocation, seven studies (8.97%) in relation to allocation sequence concealment. Thirty-six studies (46.15%) among the sample of studies evaluated did not control for the risk of bias relating to any of the dimensions under examination.

Table 3 presents the results from the logistic meta-regression for evaluating the association between the risk of bias in the domains assessed and the statistical significance of the main outcomes of the studies.

The logistic meta-regression indicated that among the four domains, only the allocation sequence generation and the blinding were associated with the statistical significance of the outcomes. Studies that used correct sequences for generating the allocation presented a chance of presenting significant results that was 2.20 times greater, but the association was not significant (P = 0.07). Studies that used blinding reduced the chance of presenting significant results by a factor of 2.28 times.

DISCUSSION

One of the limitations of this study was that it was restricted to evaluating the journals containing open-access studies that are indexed in Lilacs. These only represent part of the scientific production of a country or region. Some articles within the field of dentistry are published in journals with restricted access or in medical journals, and there is a growing tendency towards publication in dentistry in international journals that are not indexed in Lilacs.⁸⁸ On the other hand, the present investigation used a hand search, which is considered to be the gold standard: it is more sensitive and retrieves between 92% and 100% of the RCTs. Traditional electronic searches may retrieve 42% of the RCTs, while searches using highly sensitive strategies (HSS) that do not exist for Lilacs may retrieve up to 80% of the RCTs.⁸⁹ In the present study, the hand search retrieved 4,879 open-access studies. The quantity of open-access studies within dentistry went up from 2.44% in 2002 to 5.18% in 2007, in relation to the total number of health studies, independent of the subject.

The reason for the use of regional databases is the ability to complement the large international databases to adequately answer clinical questions and provide contextual evidence to support decision-making processes to meet the needs of local or regional populations and help solving the global information poverty.⁹⁰ Clark and Castro⁹¹ analyzed the Lilacs database and found that in 71% of the cases, a search in this database could be useful for the authors of systematic reviews. The data from the present study indicated that, if evaluated systematically and rigorously, only 78 (1.70%) out of the 4,581 studies were potential RCTs and 10 (0.02%) were true RCTs, because only these 10 studies used correct allocation sequence generation.

The main role of correct random sequence generation for the allocation is to ensure that the groups are balanced, in relation to both known and unknown factors, thus equalizing the initial prognosis before introducing the experimental intervention. For this dimension, among the 78 RCTs evaluated, 64 (82.05%) did not report on it or used an incorrect method, four studies (5.13%) were classified as unclear and only 10 studies (12.82%) used a method that was truly randomized for generating the allocation sequence for the groups. Among these ten randomized studies, five of them used simple or unrestricted randomization, among which four studies^24,39,44,58 used random number tables and one study⁴⁸ used a computer-generated random sequence. The other five authors used restricted sequences for generating the allocation: Botelho et al.¹⁷ used randomization in blocks, from a computer-generated list; Proano de Casalino and Guillen Galarza,⁶⁵ de Assis et al.³² and Peruzzo et al.⁶³ tossed a coin; and Mora Loya et al.⁵⁵ used a random number table, but in a restricted manner. Many authors reported that their study was randomized but did not report the method used and many mentioned randomization only in the title or abstract but did not report it in the text of the article. Other studies reported the randomization as synonymous with allocation to groups through the judgment of an experienced researcher.

Since 1995, there has been good empirical evidence regarding the effects from non-randomized allocation sequences. Schulz et al.⁵ found that, on average, studies with inadequate allocation sequences exaggerated the odds ratios by 41% and by 30% for unclearly concealed trials (after adjustment for other aspects of quality), in comparison with the studies that used adequate sequence allocation. The results suggested that if the allocation is not randomized, even in studies with correct allocation sequence concealment, it is possible for the investigators and patients to decipher the allocation of the participants to some degree. Recently Kunz et al.⁹² confirmed empirically that not using randomized generation of sequence allocation, or using it inadequately, could result in overestimates of the interventions. However, this bias may occur in either direction: in favor of or against the intervention. In the present review, correct use of sequence generation for the allocation doubled the chance that the study would furnish significant results, although the meta-regression did not produce a significant result. This was probably because of type II error due to the small sample of articles and the type of outcome used (statistical significance), which requires a major difference in effect size for it to be greater than what might be expected by chance.

The first meta-epidemiological study to provide empirical evidence regarding the effect from allocation concealment was published in 1995.⁴ This evidence was confirmed subsequently by other studies.^93,94 These studies together involved evaluations on 1,615 primary studies. More recently, Wood et al.⁹⁵ evaluated 102 meta-analyses and 804 studies and concluded that lack of allocation sequence concealment produced a mean increase in the estimates of 17%. This deviation did not occur or was smaller in studies that measured objective outcomes (9%) and was greater in subjective studies (31%). The importance of allocation sequence concealment is that its absence or incorrect use may result in bias in selecting participants based on prognostic factors, even in the presence of a correct method for generating the allocation. For example, when the result from the random draw is known, patients allocated to a given type of intervention may be considered inadequate and be eliminated from the study. Randomization associated with allocation sequence concealment may avoid selection bias, but when used alone, it may be insufficient to avoid this type of bias. In the present evaluation, 67 studies (85.90%) presented high risk of bias, four (5.13%) studies were unclear and only seven studies (8.97%) reported a correct method for the concealment of allocation. Among the latter, only two studies^58,63 used complete control over selection bias, through concomitantly using randomization and allocation sequence concealment. Lima et al.,⁵⁰ Ozaki et al.,⁵⁸ Salgado et al.,⁷³ Bortolluzy et al.,¹⁶ Costa et al.,²⁸ Moraes et al.⁵⁶ and Peruzzo et al.⁶³ used central randomization by third parties, or sequentially numbered envelopes or packages containing medications or the test and control substances. Guimarães et al.⁴⁷ used opaque sealed envelopes. Allocation sequence concealment was the least frequently observed dimension among the articles evaluated, and was the most frequent source of confusion with blinding.

Lack of blinding may lead to performance bias, thus signifying a systematic difference between the groups, with regard to the care received, because of intentional exposure to different co-interventions other than the experimental intervention. It may also induce detection bias, which results in systematic differences between groups regarding the way in which outcomes are measured or reported.² While there is empirical evidence that generation and allocation sequence concealment are more homogenous regarding modification of the effect size of interventions, double blinding produces bias of smaller magnitude and the deviations are more heterogenous. Colditz et al.⁹⁶ evaluated 113 RCTs and found that if correct blinding was used, the effects obtained from the interventions were more modest. Empirical studies have indicated that incorrect blinding increases the size of the effect from interventions by an average of 9%.⁹⁷ The impact from lack of blinding varies according to the type of outcome. Objective and very marked outcomes such as mortality are less influenced by absence of blinding and sometimes may not need this type of control. Subjective studies are much more susceptible to this type of bias.⁹⁵ In the present study, blinding was the bias control dimension most frequently seen among the studies evaluated. Out of the 78 studies, 26 (33.33%) used a correct method of blinding, even if not all of them were double blind. One study⁴⁶ reported a triple blind method, while nine studies reported a double-blind method: de Assis et al.,³² Bortolluzy et al.,¹⁶ Botelho et al.,¹⁷ Calvo et al.,¹⁹ Faraco et al.,³⁶ Guimarães et al.,⁴⁷ Kogawa et al.,⁴⁸ Rivera et al.⁶⁸ and Vieira et al.⁸⁶ Fifteen studies reported single blinding of the healthcare providers or outcome assessors.^{10,23,25,27,28,30,31,33,37,39,44,46,50,58} In another 14 studies, the authors reported blinding but did not describe the method, or blinding was impossible. The studies by Gispert Abreu et al.⁴¹ and Gispert Abreu et al.⁴² were examples of the impossibility of blinding, because there was no report of any alternative treatment or placebo for the controls.

In the present study, it was found that 23 studies (29.49%) were classified as presenting low risk of bias for the dimension of incomplete outcome data, while 55 studies (70.51%) presented an unclear or high risk of bias. Some studies stated that the number of participants at the end of the study was identical to the number that underwent the initial examination. For some of these studies, this coincidence did not seem plausible, which suggests that participants who started the study but did not complete it were simply disregarded, and that only the number of participants who completed the study was considered for the analysis. In such cases, the loss of participants was erroneously reported as zero. Among the studies that evaluated or reported missing data, none of them used intention-to-treat analysis.

The empirical evidence for evaluating the effect of incomplete outcome data on the magnitude of the effects from interventions was assessed by Tierney et al.⁹⁸ They evaluated 14 meta-analyses on individual patients' data and found that incomplete outcome data had a real effect on the outcomes and that the bias was greater in the treatment group than in the control group. In 2009, Nüesch et al.⁹⁹ evaluated 14 meta-analyses with subjective outcomes that included 167 trials and 41,170 participants. The results indicated that trials with incomplete outcome data tended to present results that were between 25% and 50% more favorable to the intervention.

Taking into account the set of 4,581 studies, only one study (0.22%) was classified as presenting low risk of bias for the four dimensions assessed and 77 potential RCTs (98.72%) were classified as presenting high risk of bias.

The level of bias control in the set of studies evaluated in Lilacs indicates that this is a body of evidence with less validity than that found in other databases such as Medline. When evaluating 3,631 studies on dental prostheses from journals indexed in Medline, Dumbrigue et al.¹⁰⁰ found that 1.71% were RCTs and 16% had a low risk of bias. Montenegro et al.¹⁰¹ found that in periodontics, RCTs were frequently of poor quality. The sequence generation could only be assessed in 17% of the RCTs and the allocation was only concealed in 7% of the RCTs evaluated. Blinding was implemented slightly more frequently, in 55% of the RCTs. The dimension of incomplete outcome data could not be assessed in 44% of the RCTs. Sjögren and Halling et al.¹⁰² evaluated 200 medical and dental RCTs and found that, in relation to generation of randomization, although the RCTs mentioned the term "randomization", only one third of them presented a true method for generating randomization. Inadequate methods were reported in 6% of the RCTs and two thirds of the RCTs did not present any randomization method. Regarding allocation sequence concealment, these same authors found that only 13% of the dental RCTs and 9% of the medical RCTs reported adequate methods. With regard to blinding, they found that 28% of the dental RCTs and 4% of the medical RCTs implemented double blinding, but that the method was only adequate in 18% of the dental RCTs and 2% of the medical RCTs. Regarding incomplete outcome data, 35% of the dental RCTs and 44% of the medical RCTs reported this item, thus suggesting that the risk of bias due to incomplete data was low. In an assessment on RCTs relating to implant dentistry published over 10-year period (1991 to 2000), Dumbrigue et al.¹⁰⁰ found 43 possible RCTs. The randomization method was explicit in 51% of the RCTs, but only 12% incorporated blinding in assessing the outcome. Looking at overall quality scores, only 2% of the RCTs adequately reported on control of bias in the three areas examined, while 56% were deficient in one area and 42% were deficient in two areas. Reporting of randomization procedures and blinding of outcomes for most of the implant RCTs was inadequate. Subject retention and documentation of subject withdrawals were adequately reported. Although the number of RCT's in dentistry with low risk of bias is not encouraging, the Lilacs database presents a worse situation than do other databases.

The distribution of the RCTs according to the importance that their authors gave to the issue of bias control, as reflected in the different degrees to which this was implemented, did not correspond to the potential for changing the effect size. This suggests that the same problem that exists in the gap between the best available clinical evidence and its adoption in practice¹⁰³ is also an evident gap between the empirical evidence in meta-epidemiological studies and the choice of dimensions used in the studies. Guidelines and better dissemination of evidence relating to methodology are needed in order to reduce this gap.

The results from the present study indicated that the proportion of information coming from studies with a high risk of bias that were published in open access journals indexed in Lilacs was sufficient to seriously affect the interpretation of the results. The effective presence of bias seriously weakened the reliability of the results from the studies evaluated, thus indicating that there is practically no trustworthy evidence that clinicians and administrators can use as support for decision-making processes in the field of dentistry. In other words, the internal validity of most the RCTs evaluated was very poor. Future studies will need to investigate whether this is a question relating to the database examined (Lilacs) or to accessibility, given that all of the RCTs evaluated were available with open-access; or whether there is a situation of extreme insufficiency of quantity of and control for the risk of bias in RCTs within the field of clinical dental research in Latin America and the Caribbean region.

CONCLUSION

Out of the 4,879 articles classified, only 10 studies (0.22%) were considered to be true RCTs and, of these, only a single study was classified as presenting low risk of bias.

All of the studies that controlled for three or more dimensions of potential sources of risk of bias presented non-significant results. The item that the authors of these RCTs most frequently controlled for was blinding. There is an evident gap between the empirical evidence produced by meta-epidemiological studies and the importance and frequency of use of bias control in studies on dentistry.

The effective presence of bias seriously weakened the reliability of the results from the dental open-access studies evaluated in the Lilacs database, such that they were of little use in dentistry for elaborating guidelines or systematic reviews as support for decision-making processes.

Date of first submission: May 31, 2010

Last received: December 15, 2010

Accepted: January 6, 2011

Sources of funding: None.

Conflict of interest: The distinguished Professor Dr. Álvaro Nagib Atallah is an author of this manuscript and substantially contributed towards producing it, and he is also the editor of the São Paulo Medical Journal.

Universidade Federal de São Paulo — Escola Paulista de Medicina (Unifesp-EPM), São Paulo, Brazil

• 1. Chalmers I. Comparing like with like: some historical milestones in the evolution of methods to create unbiased comparison groups in therapeutic experiments. Int J Epidemiol. 2001;30(5):1156-64.
• 2. Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.0.1 [updated September 2008]. The Cochrane Collaboration; 2008. Available from: http://www.cochrane.org/training/cochrane-handbook Accessed in 2010 (Dec 6).
• 3. Sackett DL. Why randomized controlled trials fail but needn't: 1. Failure to gain "coal-face" commitment and to use the uncertainty principle. CMAJ. 2000;162(9):1311-4.
• 4. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;273(5):408-12.
• 5. Day SJ, Altman DG. Statistics notes: blinding in clinical trials and other studies. BMJ. 2000;321(7259):504.
• 6. May GS, DeMets DL, Friedman LM, Furberg C, Passamani E. The randomized clinical trial: bias in analysis. Circulation. 1981;64(4):669-73.
• 7. Chan AW, Altman DG. Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. BMJ. 2005;330(7494):753.
• 8. Moher D, Pham B, Jones A, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet. 1998;352(9128):609-13.
• 9. Landis JR, Koch GG. An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics. 1977;33(2):363-74.
• 10. Aguiar AA, Saliba NA. Toothbrushing with vegetable oil: a clinical and laboratorial analysis. Braz Oral Res. 2004;18(2):168-73.
• 11. Alves RV, Machion L Casati MZ, Sallum EA, Sallum AW. Alterações de profundidades de sondagem após raspagem com diferentes instrumentos. Acompanhamento de 6 semanas [Changes in probing depth measurements following scaling and root planing with different instruments. A 6-week follow-up]. Odontol Clín Cient. 2003;2(3):187-90.
• 12. Amaral MT, Guedes-Pinto AC, Chevitarese O. Effects of a glass-ionomer cement on the remineralization of occlusal caries: an in situ study. Braz Oral Res. 2006;20(2):91-6.
• 13. Andrade AKP, Micheli G, Pannuti C, et al. Redução bacteriana em lesões de furca grau II após raspagem associada ou não à aplicação do laser de Nd: YAG [Bacterial reduction in class II furcation after root debridement with or without Nd: YAG laser irradiation]. RPG Rev Pos Grad. 2005;12(3):308-15.
• 14. Bellón Leyva S, Calzadilla Mesa XM. Efectividad del uso del propóleo en el tratamiento de la estomatitis aftosa [Effectiveness of the use of propolis in the treatment of aphthous stomatitis]. Rev Cuba Estomatol. 2007;44(3).
• 15. García Bermúdez-Cuesta MV, Díaz Romero RM, Littman Sevilla J, et al. Efectos de la musicoterapia sobre la ansiedad generada durante la atención dental, en las mujeres embarazadas en el Servicio de Estomatología del Instituto Nacional de Perinatología. Rev ADM. 2004;LXI(2):59-64. Available from: http://www.medigraphic.com/pdfs/adm/od-2004/od042d.pdf Accessed in 2010 (Dec 7).
• 16. Bortoluzzi MC, Manfro R, Grandi CF, Restelato JMF. Ensaio clínico randomizado comparativo do nível de dor pós-operatória da cirurgia dentária tratados com naproxeno ou celecoxibe: estudo piloto [Randomized clinical trial comparing the postoperative pain level of dental surgery treated with naproxen and celecoxib: a pilot study]. RGO (Porto Alegre). 2007;55(4):343-7.
• 17. Botelho MA, Bezerra Filho JG, Correa LL, et al. Effect of a novel essential oil mouthrinse without alcohol on gingivitis: a double-blind randomized controlled trial. J Appl Oral Sci. 2007;15(3):175-80.
• 18. Braga JKS, Cavalcante PR, Cavalcante AA, Ribeiro CCC. Retenção oral do flúor em crianças com diferentes experiências da doença após escovação com um dentifrício de baixa concentração [Fluoride oral retention in children with different caries experience after tooth brushing with a low F dentifrice]. Rev Fac Odontol Porto Alegre. 2006;47(3):29-32.
• 19. Calvo AM, Sakai VT, Modena KCS, et al. Comparação da eficácia de etoricoxib e ibuprofeno no controle da dor e do trismo após exodontia de terceiros molares inferiores [Comparison of the efficacy of etoricoxib and ibuprofen in pain and trismus control after lower third molar removal]. Rev Odontol Univ Cid São Paulo (Online). 2006; 18(1)29-36.
• 20. Camps Mullines I, Corona Carpio MH, Medina Magluen C, Bruzón Varona C, Ibáñez Castillo MC. Eficacia de la crema de aloe al 25 por ciento en la estomatitis subprotésica grado II [Efficacy of 25 por ciento aloe cream in treating second degree denture stomatitis]. Rev Cuba Estomatol. 2007;44(3).
• 21. Cardoso VES, Buzalaf MAR. Concentração de flúor na saliva após o uso de dentifrício fluoretado, precedido ou não por um bochecho com cálcio [Salivary fluoride concentration after the use of a fluoridated dentifrice, preceded or not by a calcium rinse]. Ciênc Odontol Bras. 2004;7(1):38-42.
• 22. Castillo Castillo A, Pérez Borrego A, Guntiñas Zamora MV. Uso del Inmunoferón en el tratamiento de la estomatitis aftosa recurrente en niños [Use of Immunoferon in the treatment of aphthous recurrent stomatitis in children]. Rev Cuba Estomatol.2006;43(3).
• 23. Chambrone D, Pasin IM, Conde MC, et al. Effect of enamel matrix proteins on the treatment of intrabony defects: a split-mouth randomized controlled trial study. Braz Oral Res. 2007;21(3):241-6.
• 24. Colombo AP, Gonçalves C, Rodrigues RMJ, et al. Microbiological evaluation of adjunctive systemic and local tetracycline administration combined with scaling and root planing in the treatment of chronic periodontitis. Braz J Oral Sci. 2003;2(7):370-7. Available from: http://libdigi.unicamp.br/document/?view=8087 Accessed in 2010 (Dec 7).
• 25. Conti PCR, Miranda JES, Conti ACCF, Pegoraro LF, Araújo CRP. Partial time use of anterior repositioning splints in the management of TMJ pain and dysfunction: a one-year controlled study. J Appl Oral Sci. 2005;13(4):345-50.
• 26. Corona Carpio MH, González Rodríguez W, García Milanês M, Martínez Ramos MR, Núñez-Antúnez L. Eficacia del extracto de Vimang en la estomatitis subprótesica Grado I [Efficacy of Vimang extract in Grade I Subprosthetic Stomatitis]. Rev Cuba Estomatol. 2007;44(4).
• 27. Cortés Vargas A, Juárez López LA, Ramírez González G. Evaluación clínica de óxido nitroso-midazolam-acetaminofén para modificar la conducta en pacientes odontopediátricos [Clinical evaluation of nitrous oxide-midazolam-acetaminophen to modify the behavior in pediatric dental patients]. Rev ADM. 2002;59(2):45-9.
• 28. Costa LRRS, Costa PSS, Lima ARA. A randomized double-blinded trial of chloral hydrate with or without hydroxyzine versus placebo for pediatric dental sedation. Braz Dent J. 2007;18(4):334-40.
• 29. Cruz Chu CM, Díaz-Pizán ME. La música como medio alternativo en la modificación de la conducta del niño durante el tratamiento dental [Music as an alternative method to modify children's behaviour during dental treatment]. Rev Estomatol Hered. 2005;15(1):46-9.
• 30. Cury JA, Marques AS, Tabchoury CPM, Del Bel Cury AA. Composition of dental plaque formed in the presence of sucrose and after its interruption. Braz Dent J. 2003;14(3):147-52.
• 31. Damante CA, Greghi SLA, Sant'Ana ACP, Passanezi E. Avaliação clínica dos efeitos do laser em baixa intensidade (GaAlAs) na cicatrização de gengivoplastia em humanos [Clinical evaluation of the effects of low-intensity laser (GaAIAs) on wound healing after gingivoplasty in humans]. J Appl Oral Sci. 2004;12(2):133-6.
• 32. de Assis Cde A, Antoniazzi RP, Zanatta FB, Rösing CK. Efficacy of Gluma Desensitizer on dentin hypersensitivity in periodontally treated patients. Braz Oral Res. 2006;20(3):252-6.
• 33. Derceli JR, Pires JR, Tardivo TA, et al. Comparative study of the clinical and anti-microbial efficacy of tongue cleaners. Braz J Oral Sci. 2007;6(22):1407-10.
• 34. Dourado AT, Caldas Junior AF, Alves DF, Falcão CA. Bacteriemia during endodontic treatment in relation to the technique of biomechanical preparation: randomized clinical trial [Bacteriemia durante o tratamento endodôntico em função da técnica de preparo biomecânico: ensaio clínico randomizado]. J Appl Oral Sci. 2005;13(4):334-9.
• 35. Elias F, Pinzan A, Bastos JRM. Influência do complexo flúor-xilitol no controle da placa dentária e do sangramento gengival em pacientes herbiátricos com aparelho ortodôntico fixo [Influence of fluorine-xylitol complex in the dental plaque and gingival bleeding control in herbiatric patients with fixed orthodontic brace]. Rev Dent Press Ortodon Ortopedi Facial. 2006;11(5):42-56.
• 36. Faraco FN, Armonia PL, Simone JL, Tortamano N. Assessment of cardiovascular parameters during dental procedures under the effect of benzodiazepines: a double blind study. Braz Dent J. 2003;14(3):215-9.
• 37. Frazao P, Sammarone M, Halk Villa S. Effect of disclosing agents in oral hygiene. Cienc Odontol. 2004;1(1):52-9.
• 38. García López E, Roche Martínez A, Blanco Ruiz AO, Rodríguez García LO. La ozonoterapia en el tratamiento de la estomatitis subprótesis [Ozone therapy in the treatment of subprosthesis stomatitis]. Rev Cuba Estomatol. 2003;40(2).
• 39. García Arocha R, González Blanco O, Solorzano AL, et al. Efecto de la bencidamina en el tratamiento de la inflamación postodontectomía del tercer molar [Effect of the benzidamine in the inflamation postodontectomic of the third molar]. Arch Venez Farmacol Ter. 2004;23(2):143-9.
• 40. Gentile LC, Greghi SLA. Clinical evaluation of dentin hypersensitivity treatment with the low intensity Gallium-Aluminum-Arsenide laser - AsGaAl. J Appl Oral Sci. 2004;12(4):267-72.
• 41. Gispert Abreu E, Herrera Nordet M, Felipe Lazo I. Control indirecto del grado de infección por Estreptococos mutans en la primera infancia [Indirect control of the level of Streptococci mutans infection in early childhood]. Rev Cuba Estomatol. 2004;41(2).
• 42. Gispert Abreu E, Herrera Nordet M, Felipe Lazo I. Prevención indirecta de caries en la temprana infancia: área Aballí: II Parte [Indirect prevention of dental caries in early childhood: Aballí Area: II Part]. Rev Cuba Estomatol. 2006; 43(4).
• 43. Gómez AM, Valencia MA, Marín DA, et al. Influencia del rofecoxib en los niveles de pg-e2 e il-1B, el movimiento dentario y el dolor durante la retracción de caninos [Influence of Rofecoxib on pg-e2 and il-1ß on tooth movement and pain during canine retraction]. CES Odontol. 2003;16(2):15-25.
• 44. Gonçalves C, Rodrigues RMJ, Feres-Filho EJ, Colombo AP. Clinical effects of systemic and topical tetracycline therapy on chronic periodontal disease. Braz J Oral Sci. 2004;3(8):384-9.
• 45. González Rodríguez WC, Corona Carpio MH, Martínez Ramos MR, García Milanés M, Núñez Antúnez L. Pulpotomías de molares temporales en pulpas muertas con tintura de propóleos al 10 por ciento [Pulpotomies of dead pulps in temporal molars using 10 per cent propolis tinction]. Rev Cuba Estomatol. 2007;44(3).
• 46. Grisi DC, Salvador SL, Marcantonio RA. Efficacy of Carisolv as an adjunctive therapy to scaling and root planing on subgingival calculus removal. Braz Dent J. 2006;17(3):213-8.
• 47. Guimaraes ARD, Peres MA, Vieira RS, et al. Self-perception of side effects by adolescents in a chlorhexidine-fluoride-based preventive oral health program. J Appl Oral Sci. 2006;14(4):291-6.
• 48. Kogawa EM, Kato MT, Santos CN, Conti PCR. Evaluation of the efficacy of low-level laser therapy (LLLT) and the microelectric neurostimulation (MENS) in the treatment of myogenic temporomandibular disorders: a randomized clinical trial. J Appl Oral Sci. 2005;13(3):280-5.
• 49. Laureano Filho JR, Vasconcellos RJH, Luz ACM, Carvalho DBSS, Gonçalves RB. Análise comparativa de duas prilocaínas disponíveis no mercado - estudo piloto [Analyzes comparative of two available prilocaínes in the market pilot study]. Revista de Cirurgia e Traumatologia Buco-Maxilo-Facial. 2003;3(3):27-34. Available from: http://www.revistacirurgiabmf.com/2003/v3n3/V3N3-artigo%204.pdf Accessed in 2010 (Dec 10).
• 50. Lima ARA, Costa LRSR, Costa PSS. A randomized, controlled, crossover trial of oral midazolam and hydroxyzine for pediatric dental sedation. Pesqui Odontol Bras. 2003;17(3):206-11.
• 51. Llerena GC, Meza DM, Muñoz VM, Tobón D, Murillo VE. Efectos del peróxido de carbamida al 10 por ciento en el órgano dentino pulpar de premolares sanos utilizado por un periodo de 8 y 15 días [Effects of carbamide peroxide to 10% in the dentine pulpar organ used in healthy premolars for a period of 8 and 15 days]. CES Odontol. 2006;19(1):19-24.
• 52. Martínez Ramos MR, Corona Carpio MH, González Rodríguez W, García Milanês M. Eficacia de la terapia floral de Bach aplicada en niños de primer grado con hábito de succión digital [Efficacy of Bach's flower therapy in first-grade schoolchildren who have the habit of finger sucking]. Rev Cuba Estomatol. 2007;44(3).
• 53. Medina Vega LD, Martín Pino J, Portal Fernández W, et al. Utilización de la moxibustion en el tratamiento de la celulitis facial odontogena. Acta Odontológica Venezolana. 2006;44(1):38-41. Available from: http://www.actaodontologica.com/ediciones/2006/1/moxibustion_celulitis_facial_odontogena.asp Accessed in 2010 (Dec 10).
• 54. Miura KK, Ito IY, Enoki C, Elias AM, Matsumoto MAN. Anticariogenic effect of fluoride-releasing elastomers in orthodontic patients. Braz Oral Res. 2007;21(3):228-33.
• 55. Mora Loya TA, Trujillo Fandiño JJ, Mora Sierra JS. Eficacia y seguridad de la aplicación de subgalato y subsalicilato de bismuto como agentes hemostáticos después de la extracción quirúrgica de terceros molars [Efficacy and safety of applying bismuth subgallate and subsalicilate as hemostatic agents after surgical extraction of third molars]. Rev ADM. 2003;60(3):90-4.
• 56. Moraes SM, Pessan JP, Ramires I, Buzalaf MAR. Fluoride intake from regular and low fluoride dentifrices by 2-3-year-old children: influence of the dentifrice flavor. Braz Oral Res. 2007;21(3):234-40.
• 57. Orellana A, Salazar E. Evaluación clínica de la terapia con dexametasona e ibuprofeno en la cirugía de los terceros molares retenidos [A clinical evaluation of dexamethasone and ibuprofen therapy in impacted third molar surgery]. Acta Odontol Venez. 2007;45(1):54-60.
• 58. Ozaki F, Pannuti CM, Imbronito AV, et al. Efficacy of a herbal toothpaste on patients with established gingivitis: a randomized controlled trial. Braz Oral Res. 2006;20(2):172-7.
• 59. Pannuti CM, Lotufo RFM, Cai S, et al. Effect of a 0.5 percent chlorhexidine gel on dental plaque superinfecting microorganisms in mentally handicapped patients. Pesqui Odontol Bras. 2003;17(3):228-33.
• 60. Pannuti CM, Mattos JP, Ranoya PN, et al. Clinical effect of a herbal dentifrice on the control of plaque and gingivitis: a double-blind study. Pesqui Odontol Bras. 2003;17(4):314-8.
• 61. Peña Ruiz T, Delgado Ramos A, González Lorenzo A. Láser helio-neón combinado con clorhexidina al 0,2 por ciento. Efectos clínicos y microbiológicos en el tratamiento de la gingivitis crónica [Helium-neon laser combined with 0.2 per cent chlorhexidine. Clinical and microbiological effects in the treatment of chronic gingivitis]. Rev Cuba Estomatol. 2007;44(3).
• 62. Pereira LJ, Gavião MB, Engelen L, Van der Bilt A. Mastication and swallowing: influence of fluid addition to foods. J Appl Oral Sci. 2007;15(1):55-60.
• 63. Peruzzo DC, Jandiroba PF, Nogueira Filho Gda R. Use of 0.1% chlorine dioxide to inhibit the formation of morning volatile sulphur compounds (VSC). Braz Oral Res. 2007;21(1):70-4.
• 64. Pinheiro SL, Bengtson AL, Imparato JCP, Oda M. Dentina infectada selada com cimento antibacteriano [Infected dentin sealed with antibacterial cement]. RGO (Porto Alegre). 2004;52(5):359-62.
• 65. Proaño de Casalino D, Guillen Galarza MF. Comparación de las técnicas anestésicas de bloqueo mandibular troncular convencional directa y Gow-Gates en exodoncia de molares mandibulares [Comparison of conventional direct anaesthetic techniques for inferior alveolar nerve block and Gow-Gates in mandibular molar surgery]. Rev Estomatol Hered. 2005;15(1):30-5.
• 66. Querido SMR, Cortelli SC, Araújo MWB, Cortelli JR. Clinical and microbial evaluation of dental scaling associated with subgingival minocycline in chronic periodontitis subjects. Braz Oral Res. 2004;18(2):110-5.
• 67. do Rego MA, Koga-Ito CY, Jorge AO. Effects of oral environment stabilization procedures on counts of Candida spp. in children. Pesqui Odontol Bras 2003;17(4):332-6.
• 68. Rivera S, Yevenes I, Reyes J, Norero H, Monardes V. Efecto de colutorio-gel de clorhexidina sobre el crecimiento de placa en 24 horas [Effect of chlorhexidine mouthrinses-gel on de novo plaque formation in 24 hours]. Rev Odonto Ciênc. 2006;21(54):358-63.
• 69. Rojano Santillán A, Pizano Damasco MA, Banderas Tarabay JA. Dosis efectiva de midazolam para sedación consciente en estomatología pediátrica [Effective midazolam dosage for conscious sedation in pediatric dentistry]. Rev ADM. 2004;61(4):130-6.
• 70. Rojas de Morales TR, Contreras JR, Rivera L, et al. Comportamiento clínico de dos cementos de óxido de cinc eugenol y su relación con el pH del medio bucal: evaluación a los doce meses [Clinical performance of two zinc-oxide eugenol cements and their relationship with the pH of oral medium: a twelve-month evaluation]. Acta Odontol Venez. 2002;40(3):9-16.
• 71. Roscher T, Rösing CK, Gjermo P, Aass AM. Effect of instruction and motivation in the use of electric and manual toothbrushes in periodontal patients. A comparative study. Braz Oral Res. 2004;18(4):296-300.
• 72. Salazar de Plaza E, Perrone M, Escalona L, Plaza Y. Eficacia analgésica del nimesulide en el dolor periodontal postoperatorio [Analgesic efficacy of nimesulide in postoperative periodontal pain]. Acta Odontol Venez. 2002;40(1).:21-5.
• 73. Salgado ADY, Maia JL, Pereira SLS, Lemos TLG, Mota OML. Antiplaque and antigingivitis effects of a gel containing Punica granatum Linn extract: a double-blind clinical study in humans. J Appl Oral Sci. 2006;14(3):162-6.
• 74. Samra Quintero PA, Bernardoni Socorro C, Morón Borjas A, et al. Changes in blood pressure in children undergoing psychological treatment before dental procedures. Acta Odontol Latinoam. 2006;19(1):9-12.
• 75. Shintome LK, Umetsubo LS, Nagayassu MP, et al. Avaliação clínica da laserterapia no tratamento da hipersensibilidade dentinária [Clinical evaluation of laser therapy on dentin hypersensitivity treatment]. Ciênc Odontol Bras. 2007;10(1):26-33.
• 76. Silva MAGS, Mendonça EF, Rodrigues KFM, Bariani C, Silva GBL. O papel do laser de baixa intensidade e da clorexidina 0,12% no tratamento da mucosite bucal em crianças leucêmicas submetidas à quimioterapia [The utilization of the low energy laser and chlorexidine 0.12% in the management of oral mucositis due to chemotherapy in children with leukemia]. ROBRAC. 2006;15(39):24-31.
• 77. Soares JA, Pires Júnior DR. Influence of sodium hypochlorite-based irrigants on the susceptibility of intracanal microbiota to biomechanical preparation. Braz Dent J. 2006;17(4):310-6.
• 78. Tavares MG, Machado AP, Motta BG, et al. Electro-acupuncture efficacy on pain control after mandibular third molar surgery. Braz Dent J. 2007;18(2):158-62.
• 79. Torres SR, Peixoto CB, Caldas DM, et al. A prospective randomized trial to reduce oral Candida spp. colonization in patients with hyposalivation. Braz Oral Res. 2007;21(2):182-7.
• 80. Ubeda LT, Cardoso VE, Buzalaf MA. Fluoride uptake in dental enamel after using fluoridated dentifrice, preceded or not by a CaCl2 solution rinse. J Appl Oral Sci. 2005;13(4):418-23.
• 81. Valle García AW, Vilchis Torres T, Arróniz Padilla S. Efectividad clínica de un dentífrico con triclosán y citrato de zinc [Clinical effectiveness of a dentifrice with triclosan and zinc citrate]. Rev ADM. 2002;59(5):166-71.
• 82. Vasconcelos BCE, Porto GG, Nogueira RVB. Controle da dor após exodontias com a utilização de bupivacaína ou lidocaína: estudo piloto [Pain control after dental extractions using bupivacaine or lidocaine: pilot study]. Ciênc Odontol Bras. 2005;8(1):28-38.
• 83. Vasconcelos BCE, Lago CAP, Lago LM, Silva MBL. Avaliação da sintomatologia dolorosa nas anestesias infiltrativas no palato quando utilizado a xilocaína a 5% e/ou placebo (vaselina) como anestésico tópico: estudo piloto [The evaluation of the painful symptomatology in infiltrativas anesthesias in the palate when used the xilocaine 5% and/or placebo (petroleum jelly) as topical anesthetic: pilot study]. Rev Odonto Ciênc. 2006;21(51):82-6.
• 84. Vergani SA, Silva EB, Vinholis AH, Marcantonio RAC. Systemic use of metronidazole in the treatment of chronic periodontitis: a pilot study using clinical, microbiological, and enzymatic evaluation. Braz Oral Res. 2004;18(2):121-7.
• 85. Vieira ALF, Zanella NLM, Bresciani E, et al. Evaluation of glass ionomer sealants placed according to the ART approach in a community with high caries experience: 1-year follow-up. J Appl Oral Sci. 2006;14(4):270-5.
• 86. Virga C, Aguzzi A, Hubert S, Aramburú G, Ricco V. Estudio de eficacia clínica de dos formulaciones de anestesia tópica [A study on the effectiveness of two topical anesthesia formulations]. Acta Odontol Venez. 2007;45(3):375-9.
• 87. Zanata RL, Navarro MFL, Pereira JC, et al. Effect of caries preventive measures directed to expectant mothers on caries experience in their children. Braz Dent J. 2003;14(2):75-81.
• 88. Gil-Montoya JA, Navarrete-Cortes J, Pulgar R, Santa S, Moya-Anegón F. World dental research production: an ISI database approach (1999-2003). Eur J Oral Sci. 2006;114(2):102-8.
• 89. Hopewell S, Clarke M, Lefebvre C, Scherer R. Handsearching versus electronic searching to identify reports of randomized trials. Cochrane Database Syst Rev. 2007;(2):MR000001.
• 90. Horton R. The less acceptable face of bias. Lancet. 2000;356(9234):959-60.
• 91. Clark OA, Castro AA. Searching the Literatura Latino Americana e do Caribe em Ciências da Saúde (LILACS) database improves systematic reviews. Int J Epidemiol. 2002;31(1):112-4.
• 92. Kunz R, Vist G, Oxman AD. Randomisation to protect against selection bias in healthcare trials. Cochrane Database Syst Rev. 2007;(2):MR000012.
• 93. Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med. 2001;135(11):982-9.
• 94. Egger M, Juni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol Assess. 2003;7(1):1-76.
• 95. Wood L, Egger M, Gluud LL, et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ. 2008;336(7644):601-5.
• 96. Colditz GA, Miller JN, Mosteller F. How study design affects outcomes in comparisons of therapy. I: Medical. Stat Med. 1989;8(4):441-54.
• 97. Pildal J, Hróbjartsson A, Jørgensen KJ, et al. Impact of allocation concealment on conclusions drawn from meta-analyses of randomized trials. Int J Epidemiol. 2007;36(4):847-57.
• 98. Tierney JF, Stewart LA. Investigating patient exclusion bias in meta-analysis. Int J Epidemiol. 2005;34(1):79-87.
• 99. Nüesch E, Trelle S, Reichenbach S, et al. The effects of excluding patients from the analysis in randomised controlled trials: meta-epidemiological study. BMJ. 2009;339:b3244.
• 100. Dumbrigue HB, Jones JS, Esquivel JF. Control of bias in randomized controlled trials published in prosthodontic journals. J Prosthet Dent. 2001;86(6):592-6.
• 101. Montenegro R, Needleman I, Moles D, Tonetti M. Quality of RCTs in periodontology--a systematic review. J Dent Res. 2002;81(12):866-70.
• 102. Sjögren P, Halling A. Randomised controlled trials and publication trends in periodontal research during 1980-2000. J Clin Periodontol. 2002;29(12):1112-7.
• 103. Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC. A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction. JAMA. 1992;268(2):240-8.

Publication Dates

• Publication in this collection
  13 May 2011
• Date of issue
  Mar 2011

History

• Reviewed
  15 Dec 2010
• Received
  31 Mar 2010
• Accepted
  06 Jan 2011