Acessibilidade / Reportar erro

Negative results in scientific research: ethical aspects

Abstract

Null, negative or unexpected results are possible occurrences for researchers around the world. Not publishing such results is a waste of resources (time, money, and effort). The objective of this study was to evaluate, through a questionnaire, what medical students and physicians thought about the publication of unexpected or negative results in research and to discuss the ethical aspects of the matter. The questions were answered by 40 students and 30 physicians from a private medical school. It is concluded that the publication of negative or unexpected results is still insufficiently discussed and accepted, perpetuating the belief that publishing such results may harm the researchers’ reputation. Almost all participants believe it is important to publish these kind of results, but only about 60% of them would publish such results. It is therefore important and necessary to broaden the discussion on this subject in medical schools to create a new academic mindset.

Publications; Editorial policies; Publications bias; Reproducibility of results; Peer review

Resumo

Resultados nulos, negativos ou inesperados são ocorrências possíveis para pesquisadores em todo o mundo. Não publicar tais resultados representa desperdício de recursos (de tempo, dinheiro e esforços). O objetivo deste estudo foi avaliar, mediante aplicação de questionário, o que estudantes de medicina e médicos pensam sobre a publicação de resultados inesperados ou negativos em pesquisa e discutir os aspectos éticos da questão. As questões foram respondidas por 40 alunos e 30 médicos de uma faculdade privada de medicina. Conclui-se que ainda é pouco discutida e aceita a publicação de resultados negativos ou inesperados, persistindo a crença que publicar tais resultados pode prejudicar a reputação dos pesquisadores. Quase todos os participantes acreditam ser importante a publicação desse tipo de resultados, mas apenas cerca de 60% os publicariam. Torna-se, então, importante e necessário ampliar a discussão sobre o assunto nas escolas médicas para se criar nova mentalidade acadêmica.

Publicações; Políticas editoriais; Viés de publicação; Reprodutibilidade dos testes; Revisão por pares

Resumen

Los resultados nulos, negativos o inesperados son acontecimientos posibles para los investigadores en todo el mundo. No publicar tales resultados representa un desperdicio de recursos (de tiempo, dinero y esfuerzos). El objetivo de este estudio fue evaluar, a través de la aplicación de un cuestionario, qué piensan los estudiantes de medicina y los médicos sobre la publicación de resultados inesperados o negativos en la investigación y discutir los aspectos éticos de la cuestión. 40 alumnos y 30 médicos de una facultad privada de medicina respondieron al cuestionario. Se concluye que aún es poco discutida y aceptada la publicación de resultados negativos o inesperados, persistiendo la creencia de que publicar tales resultados puede perjudicar la reputación de los investigadores. Casi todos los participantes creen que es importante la publicación de este tipo de resultados, pero solo el 60% de ellos los publicaría. Se hace importante y necesario, entonces, ampliar la discusión sobre este tema en las facultades médicas para crear una nueva mentalidad académica.

Publicaciones; Políticas editoriales; Sesgo de publicación; Reproducibilidad de resultados; Revisión por expertos

The academic community is constantly under pressure to obtain funding for their research and, after finalizing them, to publish the results obtained. This process improves scientific knowledge and brings benefits to the researchers that, with this, get greater credibility, better academic positions and guarantee the continuity of resources for their projects 11. Fanelli D. Negative results are disappearing from most disciplines and countries. Scientometrics. 2012;90(3):891-904.

2. Meis L, Velloso A, Lannes D, Carmo MS, Meis C. The growing competition in Brazilian science: rites of passage, stress and burnout. Braz J Med Biol Res [Internet]. 2003 [acesso 10 maio 2018]; 36(9):1135-41. Disponível: http://ref.scielo.org/9mkvgr
http://ref.scielo.org/9mkvgr...

3. Angell M. Publish or perish: a proposal. Ann Intern Med. 1986;104(2):261-2.

4. Lawrence PA. The politics of publication: authors, reviewers and editors must act to protect the quality of research. Nature [Internet]. 2003 [acesso 10 maio 2018];422(6929):259-61. Disponível: https://bit.ly/2Iucka9
https://bit.ly/2Iucka9...
-55. Monteiro R, Jatene FB, Goldenberg S, Población DA, Pellizzon RF. Critérios de autoria em trabalhos científicos: um assunto polêmico e delicado. Rev Bras Cir Cardiovasc [Internet]. 2004 [acesso 10 maio 2018];19(4):III-VIII. Disponível: http://ref.scielo.org/wmwmmm
http://ref.scielo.org/wmwmmm...
.

The publish or perish policy induces the association of scientific productivity with academic success and threatens researchers, and can also distort knowledge, particularly when the results obtained are not what was expected. Still, it is necessary to always publish, looking for high impact periodicals 33. Angell M. Publish or perish: a proposal. Ann Intern Med. 1986;104(2):261-2.,44. Lawrence PA. The politics of publication: authors, reviewers and editors must act to protect the quality of research. Nature [Internet]. 2003 [acesso 10 maio 2018];422(6929):259-61. Disponível: https://bit.ly/2Iucka9
https://bit.ly/2Iucka9...
.

Competition should not prevent the scientific community from publishing any results, since producing and disseminating knowledge is the ultimate and desired goal of science. Researchers must be accountable to society, to the academic or charitable service to which they belong, and to the sponsors of the studies. In addition to these inherent aspects of academic responsibility, the publication of negative results can accelerate studies on a given topic, avoiding efforts to be undertaken when it is already known that the expected result will not be obtained.

The publication of unexpected or negative results entails extra difficulties for the researcher. Many emphasize, explore, and discuss positive results, failing to document negative results that are then considered insignificant 66. Parasuraman S. Prospective on publishing negative results. J Pharm Negat Results [Internet]. 2015 2015 [acesso 10 maio 2018];6(1):1. Disponível: https://bit.ly/2IcJtni
https://bit.ly/2IcJtni...

7. Prasad V, Vandross A, Toomey C, Cheung M, Rho J, Quinn S et al. A decade of reversal: an analysis of 146 contradicted medical practices. Mayo Clin Proc [Internet]. 2013 [acesso 10 maio 2018];88(8):790-8. Disponível: https://mayocl.in/2jOLpb7
https://mayocl.in/2jOLpb7...
-88. Oberhofer AL, Lennon RP. A call for greater power in an era of publishing negative results. Acta Med Acad [Internet]. 2014 [acesso 10 maio 2018];43(2):172-3. Disponível: https://bit.ly/2rB18xV
https://bit.ly/2rB18xV...
. However, publishing these results is important because observing the mistakes already made always brings some learning and saves time and resources; and evaluating only selected results can lead us to the wrong conclusions 99. Prasad V, Cifu A, Ioannidis JP. Reversals of established medical practices: evidence to abandon ship. Jama. 2012;307(1):37-8.,1010. Teixeira da Silva JA. Negative results: negative perceptions limit their potential for increasing reproducibility. J Negat Results Biomed [Internet]. 2015 [acesso 10 maio 2018];14:12. Disponível: https://bit.ly/2I6TUwz
https://bit.ly/2I6TUwz...
.

Failure to publish unexpected or negative results is of concern to scientific knowledge 1111. Knight J. Negative results: null and void. Nature. 2003;422(6932):554-5. and may have several reasons including, among others, the fear that this may suggest inadequate design of the study, inducing unfavorable evaluation of research by journal editors when compared to others with positive results or to discredit the researcher. Considering the need to better discuss this issue and clarify possible ethical doubts, the objective of this study was to evaluate what medical students and physicians think about the publication of unexpected or negative results and discuss the ethical aspects involved in the issue.

Methods

The project was approved by the institutional ethics committee according to the norms for research involving human beings of the Conselho Nacional de Saúde - CNS (National Health Council): Resolution CNS 466/2012. All study participants signed a free and informed consent form.

This is an exploratory cross-cut study conducted through questionnaires with open and closed questions to evaluate the knowledge about the habit of reading scientific articles, previous participation in clinical research and writing of scientific articles, the importance of publishing negative results and ethical aspects of the dissemination of such results. A sample of convenience represented by 70 subjects selected at a private medical school (São Leopoldo Mandic School of Medicine, Campinas, São Paulo) was divided into two groups: Group 1 with 40 medical students (10 in each series from 1 to 4) and Group 2 with 30 physicians (of different specialties, teachers or not).

Data collected from the questionnaire applied to the study sample were tabulated in the Excel program and analyzed statistically with the GrapPad Prism software (version 6.0, La Jolla, CA, USA). Response rates in the groups were calculated as a percentage using a 95% confidence interval, and Fisher’s exact test was used to compare the groups (teachers and students), considering a significance level of 5%.

Results

Physicians had a mean of 20.7 years since graduation, had an average age of 45.8 (9.4) years and 29 of them had teaching activities. Students had a mean of 23.3 (5.1) years. All respondents, except one student, reported reading scientific articles (in the medical field) frequently, with 76.6% of physicians and 46.1% of students reporting reading more than five articles / month. All physicians and 72% of students reported reading articles in languages other than Portuguese.

It was also stated by 86.7% of physicians and 55% of students (p* = 0.084) that they were participating in research projects at the time of the interview; 66.7% of the physicians interviewed reported having participated in research projects as students. Among the 27 (out of 30) physicians (90%) who reported having read articles containing negative or unexpected results, 18 (66.6%) remembered the subject, but among the 17 (42.5%) students who reported having read such articles, only 7 (41.2%) remembered the theme presented.

Twenty-eight of the 30 physicians and all 40 students believe it is important to publish negative research results, but only 20 physicians and 26 students would publish such results in their reports. 90% of physicians and 87.5% of students believe that research subjects should be informed of negative results obtained, and 80% of physicians and 25% of students reported knowing researchers who posted negative research results. All students and 29 of the 30 physicians believe it is important to publish this type of result, but only two physicians and one student know journals for that purpose.

The answers given by students and physicians to questions about publication of research results and their importance, including ethical questions, are found in Table 1.

Table 1
Answers of physicians and medical students to the questions about negative research results.

Discussion

Although erroneously recognized as being indicative of unsuccessful research, negative, null or unexpected results are frequent and may be useful in scheduling new research, saving time and resources 77. Prasad V, Vandross A, Toomey C, Cheung M, Rho J, Quinn S et al. A decade of reversal: an analysis of 146 contradicted medical practices. Mayo Clin Proc [Internet]. 2013 [acesso 10 maio 2018];88(8):790-8. Disponível: https://mayocl.in/2jOLpb7
https://mayocl.in/2jOLpb7...
,99. Prasad V, Cifu A, Ioannidis JP. Reversals of established medical practices: evidence to abandon ship. Jama. 2012;307(1):37-8.. Publishing these results, however, often requires more effort than publishing research with positive results. According to Teixeira da Silva 1010. Teixeira da Silva JA. Negative results: negative perceptions limit their potential for increasing reproducibility. J Negat Results Biomed [Internet]. 2015 [acesso 10 maio 2018];14:12. Disponível: https://bit.ly/2I6TUwz
https://bit.ly/2I6TUwz...
, among the 13 journals originally created to disseminate unexpected or negative results, only five were active in 2015 and are still active to date.

The first such publication to be reported is from 1997, the Journal of negative observations in genetic oncology, which has been discontinued 1010. Teixeira da Silva JA. Negative results: negative perceptions limit their potential for increasing reproducibility. J Negat Results Biomed [Internet]. 2015 [acesso 10 maio 2018];14:12. Disponível: https://bit.ly/2I6TUwz
https://bit.ly/2I6TUwz...
. Another journal aimed at publishing negative results was released in 2002 - Journal of negative results in biomedicine - with the premise that both negative and positive results can improve future research and decision making.

However, in a 2011 study, Fanelli 11. Fanelli D. Negative results are disappearing from most disciplines and countries. Scientometrics. 2012;90(3):891-904. evaluated the impact of what was supposed to be the result of competition for funding and the quest for citing among authors, observing that of the 4,600 articles analyzed between 1990 and 2007 there was a 22% increase in reports of positive results, with more significant growth in Asian countries compared to Europe and the United States. Although it was not possible to explain this pattern, it suggested a decrease in the pioneering and/or objectivity of the research.

The São Leopoldo Mandic School of Medicine had only four years of operation at the time of this study (2016), but it prioritizes the stimulation of participation in research projects, which could be evidenced by the information obtained from 86.7% of physicians and 55% of the students were participating in research at the time.

The high rate of reading of scientific articles by most interviewees and even the reading of published research with negative or unexpected results (90% of the physicians and 42.5% of the students interviewed) may be influenced by the voluntary nature of participating in the research, since respondents tended to be those who have already done research and wish to collaborate with other researchers responding to the study. However, even reading medical literature frequently, more than 90% of respondents were unaware of the existence of journals specifically intended for such publications.

The United States Clinical Trials 1212. U.S. National Institutes of Health. ClinicalTrials.gov is a registry and results database of publicly and privately supported clinical studies of human participants conducted around the world [Internet]. c2015 [acesso 10 abr 2017]. Disponível: https://clinicaltrials.gov/
https://clinicaltrials.gov/...
initiative, created in 1977 as a joint effort of the Food and Drug Administration and the US Department of Health and Human Services (HSS), through the National Institutes of Health (NIH) to establish a platform for recording information on clinical trials conducted by public initiative (research institutes and government agencies) and the private sector (pharmaceutical companies) in order to obtain up-to-date and reliable clinical research data. In 2004, the World Health Organization (WHO) and the International Committee of Medical Journal Editors (ICMJE) established the “Clinical Trials Register Platform” 1313. World Health Organization. International Clinical Trials Registry Platform (ICTRP) [Internet]. Geneva: WHO, c2015 [acesso 10 abr 2017]. Disponível: http://bit.ly/2HDbDY2
http://bit.ly/2HDbDY2...
to register clinical trials and ensure the publication of all results, negative or positive.

However, the international agreement in which world-wide journals began to publish only articles with prior registration on these platforms in 2005 was not enough to avoid deviations in the publication of results 1313. World Health Organization. International Clinical Trials Registry Platform (ICTRP) [Internet]. Geneva: WHO, c2015 [acesso 10 abr 2017]. Disponível: http://bit.ly/2HDbDY2
http://bit.ly/2HDbDY2...
. In the population we studied, all interviewees, except one student, recognized that negative results are not devoid of value and should be published even if they are unfavorable. Yet a third of physicians and 15% of students believe that scientific journals prefer to publish positive results and that this may be influenced by economic interests in funded research (13.3% of physicians and 2.5% of students).

Concern about the impact on academia with the publication of negative results was evident when we found that 4 of the 30 physicians and 12 of the 40 students stated that the reputation of the researcher could be hampered by this. Even believing that the publication of negative results saves resources, and that this has been growing in recent years (43.3% of physicians and 47.5% of students), the belief is that, in practice, almost no negative results are reported or (43.3% of physicians and 40 of the students).

Despite the obligation to divulge results, some authors mention that the results of research published after the creation of the Registry of Clinical Research have different objectives from those originally stated at the time of the research in the registry 1313. World Health Organization. International Clinical Trials Registry Platform (ICTRP) [Internet]. Geneva: WHO, c2015 [acesso 10 abr 2017]. Disponível: http://bit.ly/2HDbDY2
http://bit.ly/2HDbDY2...
,1414. Rasmussen N, Lee K, Bero L. Association of trial registration with the results and conclusions of published trials of new oncology drugs. Trials [Internet]. 2009 [acesso 10 maio 2018];10:116. Disponível: https://bit.ly/2rAJMkP
https://bit.ly/2rAJMkP...
. On the other hand, Kaplan and Irvin 1515. Kaplan RM, Irvin VL. Likelihood of null effects of large NHLBI clinical trials has increased over time. PLoS One [Internet]. 2015 [acesso 10 maio 2018];10(8):e0132382. Disponível: https://bit.ly/1J15XWG
https://bit.ly/1J15XWG...
, in 2015, identified a significant increase in null results in cardiology research with the registry.

In the pediatric area, a retrospective study conducted between 2008 and 2010 to evaluate clinical trials with pediatric patients revealed that discontinuation and non-publication were frequent, with thousands of children being exposed to interventions that did not result in useful information: from 559 trials analyzed, 19 % were discontinued early and of the 455 completed, 136 were not published, representing 69,165 pediatric patients 1616. Ross JS, Mulvey GK, Hines EM, Nissen SE, Krumholz HM. Trial publication after registration in ClinicalTrials.gov: a cross-sectional analysis. PLoS Med [Internet]. 2009 [acesso 10 maio 2018]; 6(9):e1000144. Disponível: https://bit.ly/2I6qWND
https://bit.ly/2I6qWND...
.

Submission of research results to publication in journals implies peer review when careful examination of the manuscript should be undertaken by specialists who assist the editor of the journal in deciding whether or not to accept the article for publication. This review may suffer from evaluation bias, favoring the acceptance of studies with positive results. This higher probability of publication of studies with favorable, positive or statistically significant results, when compared to studies of similar quality with negative results or that were not able to show statistically significant differences, has been called a “positive outcome bias” 1717. Pica N, Bourgeois F. Discontinuation and nonpublication of randomized clinical trials conducted in children. Pediatrics [Internet]. 2016 [acesso 10 maio 2018];138(3):e20160223. Disponível: https://bit.ly/2rAbEWi
https://bit.ly/2rAbEWi...
,1818. Dickersin K, Min YI, Meinert CL. Factors influencing publication of research results: follow-up of applications submitted to two institutional review boards. Jama. 1992;267(3):374-8..

An interesting and illuminating study by Emerson and colleagues 1919. Emerson GB, Warme WJ, Wolf FM, Heckman JD, Brand RA, Leopold SS. Testing for the presence of positive-outcome bias in peer review: a randomized controlled trial. Arch Intern Med [Internet]. 2010 [acesso 10 maio 2018];170(21):1934-9. Disponível: https://bit.ly/2rygXGj
https://bit.ly/2rygXGj...
investigated the influence of presenting zero or positive results on acceptance for publication in peer-reviewed journals: two “intentionally fabricated” manuscripts (one with the positive end result, confirming the main hypothesis and another with null results) were sent to two different groups of reviewers of two orthopedic journals. The positive results of the manuscript resulted in a higher approval rate (97.3 versus 80%, p <0.001) and less indication of methodological errors (0.41 versus 0.85, p <0.001). Other studies, however, did not observe an influence of the presentation of negative results in acceptance for publication among 246 abstracts submitted for peer review in international journals 2020. Van Lent M, IntHout J, Out HJ. Peer review comments on drug trials submitted to medical journals differ depending on sponsorship, results and acceptance: a retrospective cohort study. BMJ Open [Internet]. 2015 [acesso 10 maio 2015];5(9):e007961. Disponível: https://bit.ly/2KSE46P
https://bit.ly/2KSE46P...
.

Journal editors should consider providing specific guidance for reviewers to evaluate manuscripts describing negative or zero results in order to minimize the effects of “positive outcome bias” by encouraging authors to submit their studies (provided they are of good quality) even if they have negative or no results (19).

There are several obstacles to the publication of these studies: the authors themselves do not have much incentive to report the data; if funding is available, sponsors tend not to have an interest in the publication of the data, and scientific journals do not usually accept such manuscripts 2121. Anderson ML, Chiswell K, Peterson ED, Tasneem A, Topping J, Califf RM. Compliance with results reporting at ClinicalTrials.gov. N Engl J Med. 2015;372(11):1031-9.. Publishing results of clinical trials independent of their outcome is a way of contributing to scientific medical knowledge, but also an ethical and legal obligation researcher and research funders 2121. Anderson ML, Chiswell K, Peterson ED, Tasneem A, Topping J, Califf RM. Compliance with results reporting at ClinicalTrials.gov. N Engl J Med. 2015;372(11):1031-9.,2222. Sandercock P. Negative results: why do they need to be published? Int J Stroke. 2012;7(1):32-3..

The ethical issue is very important: if individuals gave their consent to participate in a clinical or observational study, they did so because they believed that the expected results would be beneficial and useful to science or other people; they were exposed to the risk and inconvenience of participating in the study for good deeds and that authors should make the data obtained public for the benefit of all 2222. Sandercock P. Negative results: why do they need to be published? Int J Stroke. 2012;7(1):32-3..

An international initiative called “All Trials” attempts to stimulate the publication of all clinical trials, regardless of results and has the support of scientific journals to do so, taking into account that information resulting from tests with negative results can be lost, resulting in practice of poor quality medicine, wrong decisions in the treatment of diseases and unnecessary repetition of studies 2323. All Trials. All trials registered/all results reported [Internet]. c2017 [acesso 18 jan 2018]. Disponível: http://bit.ly/1h9PUXL
http://bit.ly/1h9PUXL...
.

Successful publication of all research results requires cultural change in academia. Discussing and clarifying these issues with medical students can inspire a new generation of researchers who will value all the scientific outcomes achieved, enabling more transparency, reduced efforts and new methods of scientific research.

Final considerations

Our exploratory study used a convenience sample with only 70 participants, which is a limitation and does not allow us to state that the results obtained are valid for other groups of physicians and medical students. In this sample, subjects interviewed have the perception that journals prefer to publish studies with positive results: 66% of physicians and 85% of students believe that it is easier to publish such results.

Despite mentioning the habit of scientific reading, the publication of negative or unexpected results in scientific research is still little discussed and accepted in the study population. Failure to publish these types of outcomes may perpetuate erroneous conclusions from the selection of results. Broadening the discussion on this subject is important in every academic environment, but especially in medical schools, to clarify, contribute to scientific knowledge and reinforce ethical concepts.

Referências

  • 1
    Fanelli D. Negative results are disappearing from most disciplines and countries. Scientometrics. 2012;90(3):891-904.
  • 2
    Meis L, Velloso A, Lannes D, Carmo MS, Meis C. The growing competition in Brazilian science: rites of passage, stress and burnout. Braz J Med Biol Res [Internet]. 2003 [acesso 10 maio 2018]; 36(9):1135-41. Disponível: http://ref.scielo.org/9mkvgr
    » http://ref.scielo.org/9mkvgr
  • 3
    Angell M. Publish or perish: a proposal. Ann Intern Med. 1986;104(2):261-2.
  • 4
    Lawrence PA. The politics of publication: authors, reviewers and editors must act to protect the quality of research. Nature [Internet]. 2003 [acesso 10 maio 2018];422(6929):259-61. Disponível: https://bit.ly/2Iucka9
    » https://bit.ly/2Iucka9
  • 5
    Monteiro R, Jatene FB, Goldenberg S, Población DA, Pellizzon RF. Critérios de autoria em trabalhos científicos: um assunto polêmico e delicado. Rev Bras Cir Cardiovasc [Internet]. 2004 [acesso 10 maio 2018];19(4):III-VIII. Disponível: http://ref.scielo.org/wmwmmm
    » http://ref.scielo.org/wmwmmm
  • 6
    Parasuraman S. Prospective on publishing negative results. J Pharm Negat Results [Internet]. 2015 2015 [acesso 10 maio 2018];6(1):1. Disponível: https://bit.ly/2IcJtni
    » https://bit.ly/2IcJtni
  • 7
    Prasad V, Vandross A, Toomey C, Cheung M, Rho J, Quinn S et al. A decade of reversal: an analysis of 146 contradicted medical practices. Mayo Clin Proc [Internet]. 2013 [acesso 10 maio 2018];88(8):790-8. Disponível: https://mayocl.in/2jOLpb7
    » https://mayocl.in/2jOLpb7
  • 8
    Oberhofer AL, Lennon RP. A call for greater power in an era of publishing negative results. Acta Med Acad [Internet]. 2014 [acesso 10 maio 2018];43(2):172-3. Disponível: https://bit.ly/2rB18xV
    » https://bit.ly/2rB18xV
  • 9
    Prasad V, Cifu A, Ioannidis JP. Reversals of established medical practices: evidence to abandon ship. Jama. 2012;307(1):37-8.
  • 10
    Teixeira da Silva JA. Negative results: negative perceptions limit their potential for increasing reproducibility. J Negat Results Biomed [Internet]. 2015 [acesso 10 maio 2018];14:12. Disponível: https://bit.ly/2I6TUwz
    » https://bit.ly/2I6TUwz
  • 11
    Knight J. Negative results: null and void. Nature. 2003;422(6932):554-5.
  • 12
    U.S. National Institutes of Health. ClinicalTrials.gov is a registry and results database of publicly and privately supported clinical studies of human participants conducted around the world [Internet]. c2015 [acesso 10 abr 2017]. Disponível: https://clinicaltrials.gov/
    » https://clinicaltrials.gov/
  • 13
    World Health Organization. International Clinical Trials Registry Platform (ICTRP) [Internet]. Geneva: WHO, c2015 [acesso 10 abr 2017]. Disponível: http://bit.ly/2HDbDY2
    » http://bit.ly/2HDbDY2
  • 14
    Rasmussen N, Lee K, Bero L. Association of trial registration with the results and conclusions of published trials of new oncology drugs. Trials [Internet]. 2009 [acesso 10 maio 2018];10:116. Disponível: https://bit.ly/2rAJMkP
    » https://bit.ly/2rAJMkP
  • 15
    Kaplan RM, Irvin VL. Likelihood of null effects of large NHLBI clinical trials has increased over time. PLoS One [Internet]. 2015 [acesso 10 maio 2018];10(8):e0132382. Disponível: https://bit.ly/1J15XWG
    » https://bit.ly/1J15XWG
  • 16
    Ross JS, Mulvey GK, Hines EM, Nissen SE, Krumholz HM. Trial publication after registration in ClinicalTrials.gov: a cross-sectional analysis. PLoS Med [Internet]. 2009 [acesso 10 maio 2018]; 6(9):e1000144. Disponível: https://bit.ly/2I6qWND
    » https://bit.ly/2I6qWND
  • 17
    Pica N, Bourgeois F. Discontinuation and nonpublication of randomized clinical trials conducted in children. Pediatrics [Internet]. 2016 [acesso 10 maio 2018];138(3):e20160223. Disponível: https://bit.ly/2rAbEWi
    » https://bit.ly/2rAbEWi
  • 18
    Dickersin K, Min YI, Meinert CL. Factors influencing publication of research results: follow-up of applications submitted to two institutional review boards. Jama. 1992;267(3):374-8.
  • 19
    Emerson GB, Warme WJ, Wolf FM, Heckman JD, Brand RA, Leopold SS. Testing for the presence of positive-outcome bias in peer review: a randomized controlled trial. Arch Intern Med [Internet]. 2010 [acesso 10 maio 2018];170(21):1934-9. Disponível: https://bit.ly/2rygXGj
    » https://bit.ly/2rygXGj
  • 20
    Van Lent M, IntHout J, Out HJ. Peer review comments on drug trials submitted to medical journals differ depending on sponsorship, results and acceptance: a retrospective cohort study. BMJ Open [Internet]. 2015 [acesso 10 maio 2015];5(9):e007961. Disponível: https://bit.ly/2KSE46P
    » https://bit.ly/2KSE46P
  • 21
    Anderson ML, Chiswell K, Peterson ED, Tasneem A, Topping J, Califf RM. Compliance with results reporting at ClinicalTrials.gov. N Engl J Med. 2015;372(11):1031-9.
  • 22
    Sandercock P. Negative results: why do they need to be published? Int J Stroke. 2012;7(1):32-3.
  • 23
    All Trials. All trials registered/all results reported [Internet]. c2017 [acesso 18 jan 2018]. Disponível: http://bit.ly/1h9PUXL
    » http://bit.ly/1h9PUXL

Publication Dates

  • Publication in this collection
    Apr-Jun 2018

History

  • Received
    1 June 2017
  • Reviewed
    23 Dec 2017
  • Accepted
    29 Jan 2018
Conselho Federal de Medicina SGAS 915, lote 72, CEP 70390-150, Tel.: (55 61) 3445-5932, Fax: (55 61) 3346-7384 - Brasília - DF - Brazil
E-mail: bioetica@portalmedico.org.br