Invisible science: publication of negative research results

A ciência invisível: a publicação dos resultados negativos de pesquisa

Abstract

An important part of scientific research activities yield negative results – non-confirmatory and null data, inconclusive experiments, unexpected data. These results permeate the entire research cycle and constitute an important part of the full scientific knowledge flow generation. However, despite the acknowledgment that it is the non-confirmatory findings that result in the rejection of consolidated hypotheses that drive the progress of science, most of these investigation routes are not documented. Growing competition for resources, tenure, and impact publications induces researchers to produce “positive” results that are more likely to be published, interfering with the principles of science reproducibility and self-correction and in the scientific communication cycle. This study aims to review negative results incorporation in the traditional academic publication cycle. It also seeks to identify and systematize the main barriers that prevent researchers from publishing negative results. This exploratory study is based methodologically on the scarce literature on the subject. It confirms the initial assumption that few scientific journals accept, edit special issues or are dedicated to the publication of negative results.

Keywords
Negative results; Negative results journals.; Scientific communication

Resumo

Uma considerável parcela das atividades da pesquisa científica é constituída de resultados negativos – dados não confirmatórios e nulos, experimentos não conclusivos, dados inesperados –, os quais permeiam todo o ciclo da pesquisa e constituem uma parte importante da integridade dos fluxos de geração de conhecimento. Porém, apesar do reconhecimento de que são os achados não confirmatórios, que resultam na rejeição de hipóteses consolidadas, os responsáveis por impulsionar o progresso da ciência, grande parte desses percursos investigativos não são documentados. A crescente competição por recursos, por posições e por publicações de impacto induz os pesquisadores a produzir resultados “positivos”, cuja possibilidade de serem publicados seja, provocando uma interferência nos princípios de reprodutibilidade e de autocorreção da ciência e no ciclo de comunicação científica. O presente estudo objetiva analisar a incorporação dos resultados negativos no ciclo de publicação acadêmica tradicional, além de identificar e sistematizar as principais barreiras que impedem o pesquisador de publicá-los. Este estudo exploratório tem como base metodológica a escassa literatura sobre o tema e constata a suposição inicial de que são poucos os periódicos científicos que aceitam, editam números especiais ou são dedicados à publicação de resultados negativos.

Palavras-chave
Resultados negativos; Periódicos de resultados negativos; Comunicação científica

Introduction

In the 19^th century, physicists Albert Michelson and Edward Morley developed a series of experiments to detect the relative movement of matter through “luminiferous ether” – a theoretical medium through which electromagnetic waves would propagate. Although the experiments clearly contradict the existence of the stationary ether theory, the scientific community initially neglected these non-confirmatory results. It was only when Michelson and Morley (1987) published their results in 1887 in the American Journal of Science, under the title “On the relative motion of the Earth and the luminiferous ether”, that the prevalent theory was scrutinized (Anderson; Olsen; Sprott, 2013). This article addresses what would be known as the most famous non-confirmatory experiment.

A few years later Albert Einstein developed his special theory of motivated relativity, in part due to the lack of evidence of the luminiferous ether existence as published by Michelson and Morley (Einstein; Infield, 1936). This fact is widely recognized as the starting point for the “second Scientific Revolution” and contributes to Michelson being awarded the 1907 Nobel Prize in Physics. A rare compliment for a “negative” result, as emphasized by van Hilten (2015).

Science is, by nature, a collaborative enterprise that follows a dialectical trajectory of errors and successes, which processes and interlocutions, based on the scientific method, are gradually converging towards new knowledge and discoveries. This dialogue is more evident in the continuous cycle of confrontations and ruptures that have historically taken place between the prevailing paradigms and the inevitability of new ideas. When we position ourselves at an epistemologically more comprehensive point of observation, the importance of negative results for the progress of science is sharpened: it is the negative findings that result in the rejection of hypotheses that drive science, and that are recorded in its annals as a catalyst for the scientific revolution and also as a compliment to the refutation. The question that this investigation meant to answer is the following: what is the space available for communication vehicles that focus negative results in the cycle of traditional scientific communication?

The scientific context induced by the bias of positive results has a clear impact on scientific communication flows, more specifically in academic publications, which only in a few cases accept negative results, under the argument of the low impact of these reports. This condition, in turn, influences the evaluation and reward parameters established by the institutions that promote research, which disregard contributions to science from reports of experiments that do not fit the “positive” model of science and, consequently, cause a gap in the academic memory of research institutions and science as a whole.

However, as identified in the present study, there is a tenuous prospect of change in the scientific scenario, saturated as it is by the bias of positive results. Despite the often opposing circumstances and the debates and controversies surrounding these issues, it is observed that recognition emerges in some scientific communities that the publication of negative results can contribute greatly to the progress of science. In this direction, there are repositories aimed at recording these results, scientific awards granted to non-confirmatory research and, above all, scientific journals that accept, edit special issues or are totally dedicated to articles that report negative findings; there are also some studies on the impact of these issues on open science and scientific communication as in Smaldino et al. (2019).

Understanding the importance of this invisible part of research activities for the progress of science, this article aims to present the current scenario of recognition and rejection of these results and to review their incorporation in the cycle of scientific communication, more specifically, in the cycle of traditional academic publications, identifying journals focused on negative results, mapping the disciplinary areas where they originate from and quickly reviewing observations and studies on acceptance, impact factor and citation level of articles published by these journals; an attempt is also made to identify and systematize the main barriers that prevent researchers from publishing their negative results. This exploratory route is methodologically based on the scarce literature – a finding expected in advance –, on the subject.

It is important to advise that the study is part of a broader project called “Invisible Science”, which aims to investigate the causes of the lack of formal records and sharing of part of the products generated by scientific research, especially publications and data. At first, the individual, organizational and technological obstacles that interfere with the lack of records and sharing of research data generated in the long-tail of science were identified. The studies found that a considerable part of the data generated by failed experiments and that, for different reasons, did not confirm the hypotheses, were not published. This is the reason behind the main motivation for the performance of the present study.

The importance of negative results for science and the praise of the Refuted Theory

In the daily laboratories routine, the negative results – considered in this study, in their entire spectrum, as non-confirmatory and null results, inconclusive experiments, unexpected data, etc. –, permeate the entire research cycle, and constitute an important part of the integrality of scientific research flows. As the Scientific Negative Results Journal points out in its home page, scientific discoveries are like a floating iceberg, exhibiting about 10% of published findings supported by 90% of unpublished negative results, giving the dimension of the huge loss of information (Negative Results, [2017]). This intrinsic condition of scientific experiments was experienced by the Brazilian researcher Ligia da Veiga Pereira, who works on the setting of a library of the Brazilian genetic profile, when she reaffirms that “the development of LaNCE – National Laboratory of Embryonic Stem Cells, cost years of negative results because starting cell culture was very difficult” (Pereira, 2019, p.85).

Observing the generation of flows of knowledge, it appears that scientific projects rarely result in perfect discoveries of elements about the universe, nature and society. This is because, as Anderson, Olsen and Sprott (2013) point out, research is often based on methods with actual limitations, imperfect experimental models and hypotheses based on uncertain assumptions, which constantly challenge the self-correcting power of science and the principles of reproducibility of scientific experiments. However, even considering these situations of uncertainty and negativity trailed by science, in many cases, the procedures, flows and methodologies are strictly correct, but lead to dead ends, whose pathways need to be documented, evaluated, published and shared so that other investigators do not follow the same trail or in order for their methodologies to be improved. “A lot of research is ultimately not useful or does not lead to positive results [...]. This is the way science goes. Obtaining negative results is not disparaging” (Zago, 2011, p.3), and it can mean a relevant contribution to science and to save lives. “For example, a study that finds that a drug has no effect on an infection can be an important and actionable information” (Oransky; Marcus, 2016, p.1), indicating that, in many cases, negative studies are more reliable than positive results.

“Null, negative or unexpected results are potential occurrences for researchers around the world”, confirm Guimarães et al. (2018, p.245). Nevertheless, the triumphalistic perspective of science, which focuses in a privileged way on the final research products – the invention, the article, the patent, the thesis –, throws into obscurity all that is not successful and that is inconclusive. As a consequence, it leaves an important part of the twisted trails of scientists unrecorded, and often creates a conflict with integrity, objectivity and reproducibility of the research process, forced by the positive bias of science or the interest of any of its players, such as scientific editors and financial supporters (Fanelli, 2010; Fanelli, 2012). Therefore, the negative results of thousands of experiments, many of them of great importance for the progress of science, will never come to light and will remain forever buried in the investigators laboratories records, drawers and pen-drives, where they will, inevitably, vanish with time.

Historically, the example of the rejection of the light ether that “inspired” Einstein to formulate the theory of relativity illustrates well one of the intrinsic conditions of science: the antagonism between the established and the new, revealing unknown horizons and forcing science to reflect critically on the validity of current thinking. This is because “The aim of science is not to produce indisputable truths, but to discuss them. Not even the natural and exact sciences produce indisputable truths. [...] The objective is to produce standardized public discussions”, summarizes the philosopher of science Latour (2017, p.3) in an interview with a Brazilian newspaper journalist. The “arguability” of the arguments is part of the scientific work. That goes for astronomers, biologists or chemists, adds Latour.

The philosopher Popper celebrates the importance and, at the same time, bases this question in his book “Conjectures and refutations: the growth of scientific knowledge”:

Refutation has often been regarded as establishing a scientist’s failure, or at least of his theory. It should be stressed that this is an inductivist error. Every refutation should be regarded as a great success by the scientist who refuted the theory, but also of the scientist who created the refuted theory and who, in the first instance, suggested, if only indirectly, the refuting experiment

(Popper, 1962, p.243).

Popper (1962) concludes highlighting the need for records of this entire cycle that is configured as the moto-continuous of scientific knowledge generation:

Even if a new theory should meet an early death, it must not be forgotten; rather, its beauty should be remembered, and history should register our gratitude to it - for bequeathing to us new and, perhaps, still unexplained experimental facts and, with them, new problems; and for the services it has thus rendered to the progress of science during its successful but short life

(Popper, 1962, p.243).

The most noble aspect of science, as noted by Matosin et al. (2014), is its supposed transparency in presenting all the sides of a scientific finding. In theory, scientific principles are always subject to reconsideration, and there are times when new evidence refutes old hypotheses and impacts current thinking, no matter how long-lasting it may be. This transition process between current thought paradigms and the new conceptions fascinated the physicist and philosopher Thomas Kuhn.

For Kuhn (1970), the moment the body of evidence that supports the competing paradigm appears to overcome the evidence that supports the dominant paradigm, scientists will more easily change their loyalty to the old paradigm, and then a new ideal will be installed. However, it is not an easy process, since the innate condition of the human being to maintain the status quo and to overcome old beliefs has an extraordinary weight in this continuous cycle of scientific discoveries.

Getting close to the objectives of this study, we understand that both philosophers, Popper and Kuhn, celebrate paradigmatic disruptions as essential inflection points for the progress of science, which are triggered by the “debatability” of arguments of science, as explained by Latour (2017). They make it clear, however, that the challenge of opposing consolidated scientific views through disruptive results, imposes the immense need to break the cognitive, behavioral, economic and social barriers, which are intrinsic to the organicity of the web of science.

These barriers interfere with the cycles of scientific communication and the reproducibility of the experiments because non-confirmatory or negative results are very often associated with methodologically flawed or poorly designed studies, which can have a negative impact on the researcher’s career. In this connection, many researchers develop the attitude that continuing and publishing negative results is a waste of resources and time, despite the understanding that not doing so is unethical (Hendrix, [2016]). “Therefore, negative results are an inconvenient truth, and ignoring the inconsistent results [with current paradigms] would only be human”, summarize Matosin et al. (2014, p.171).

Therefore, negative results are a valuable component of scientific literature, since they compel the research community to critically evaluate and validate current thinking and, fundamentally, lead us towards an integral science (Matosin et al., 2014). However, as will be seen below, researchers and scientific editors attitude in the face of unexpected results triggers an invisibility process that spreads to the entire scientific communication infrastructure.

The investigator facing his negative results

There is a time when investigators – after trials and more trials, checks and confirmations –, find that their experiment did not disapprove of the null hypothesis and did not confirm their new theory. From there, possibly, they will close their laboratory notebooks and move to new experiments that can generate results that are more likely to be confirmatory. With such decision, they will make a significant part of their research routes invisible and avoid any narrative on the experiences and new knowledge relevant to their community.

This silence illustrates the common fact that the results of thousands of non-confirmatory experiments will forever remain in obscurity. Even with the assumption that the scientific method is absolute and that the experiment was well designed and controlled, its results, though, may never be reported. The literature provides evidence of the fact that negative results are three times less likely to be published in scientific journals when compared to results considered positive (Dickersin et al., 1987; Fanelli, 2010; Fanelli, 2012; Hendrix, [2016]; Timbie, 2019). Furthermore, the small fraction of academic articles that report results that fail to support the hypotheses being tested, if they are published, are less likely to be quoted (Fanelli, 2010).

Even understanding that these types of results occur daily in the world of science and that they are part of the life of laboratories, there is an increasing pressure on scientists to select investigative paths that lead to high-impact knowledge. The growing competition for funding, academic positions and prestige, combined with the increasing use of bibliometric parameters to evaluate careers (for example, number of publications and the impact factor of journals where they are published), put pressure on scientists to continually produce “publishable” results (Anderson; Olsen; Sprott, 2013). Competition is encouraged in scientifically advanced countries, as it is considered that it increases researchers’ efficiency and productivity (Fanelli, 2010); however “when competition is pervasive, its effects can jeopardize the progress, efficiency and integrity of science”, highlight Anderson, Olsen and Sprott (2013, p.1).

The “publish or perish” mantra, assumed by most scientific communities, induces the development of a culture that associates productivity with academic success and creates, as a consequence, a threat that lures over the researchers, influencing their investigative pathways. This condition can also “distort knowledge, particularly when the results obtained are not those expected”, as pointed out by Guimarães et al. (2018, p.246), thus producing spurious results, skewed towards statistically significant results that are accepted by editorial codes. Faced with a negative result, therefore, a scientist may be tempted not to waste time publishing it – behavior known as the “file drawer effect”; or make it positive, by reformulating the initial hypotheses, selecting the results to be published, reformulating data or analyses, among other strategies. These procedures are expressed by the English acronym HARKing (Hypothesizing After the Results are Known) (Fanelli, 2010).

This fact is greatly related to the more general phenomenon known by the term “publication bias”, which denotes the trend observed in scientific publishing, that positive evidence is more likely to be published than negative evidence. However, it is necessary to pay attention to the fact that when slanted positive results are incorporated, this behavior strongly affects the integrity of the generation of knowledge and contributes to the exacerbation of the so-called “reproducibility crisis” of scientific experiments. The extent of this crisis can be partially assessed by a survey carried out by the journal Nature and reported by Monya Baker. Through a brief online questionnaire on reproducibility, the journal interviewed 1576 investigators from different areas. The review of the results yielded quite eloquent numbers: “More than 70% of the researchers tried and failed to reproduce the experiments of other researchers, and more than half failed to reproduce their own experiments” (Baker, 2016, p.1).

Hendrix ([2016]) concludes that the positive results bias is a special form of publication bias. It is a more elegant term to express that the probability of authors sending, or editors accepting, positive results is greater than the probability of submitting and accepting negative or inconclusive results. Still according to Hendrix ([2016], online), “In summary: publishing only positive results and conceal negative results produces a distorted view of reality and results in an unnecessary repetition of experiments already carried out [...]”.

This imbalance condition often leaves aside inconclusive results or those that challenge our most sedimented assumptions. The scale of values assumed by the academic community and its interlocutors, such as research development agencies, explicitly supports results considered positive. Along this line, the researchers themselves strive to obtain these results and are quick to publish them, leaving failed attempts in the dark.

How can this culture be changed to value negative results? Asks van Hilten (2015). However, as it will be seen below, publication of negative data has already, for some time, been a critical problem on the agenda of scientific communication processes.

Publication of negative results: revealing the dark matter of science

There are several consequences arising from keeping negative results in the dark that, at first, represent a waste of resources in terms of time, money and intellectual effort. The most perceptible argument is that sharing the failed experiments and their results can prevent other researchers from other research groups from going through dead ends “that will lead to equally negative results (and that probably will not be published either)”, as Piropo observes (2014, p.1); if another laboratory wishes to try a similar experiment, its team has the opportunity to incorporate changes based on previous attempts, enhancing the collaborative and open nature of science. As Pfeffer and Olsen (2002) point out in the Journal of Negative Results in BioMedicine, publishing well-documented flaws can reveal fundamental distortions and obstacles in commonly used methods, drugs or reagents, such as antibodies or cell strains, ultimately leading to improved design research and clinical decisions. However, there are political, moral and ethical developments that should be considered when reviewing positive publication bias. For example, skepticism about the power of self-correction, social relevance and science integrity, and the unnecessary repetition of experiments that use animal models, are on the critical agenda. A more complete and holistic view of science can also be included in the moral scope, which goes beyond the dramatic effects expected by the editors, who according to Mudrak (2019, p.4) “the current trend focused on spectacular results can lead to false conclusions”.

We must remember that the scientific editors’ disregard of reports of non-confirmatory research is not a new phenomenon, as warned by van Hilten (2015). In this scenario, however, the insecure and indecisive attitudes of the authors to disclose their negative results are incorporated. In an investigation published in 1987, the area of clinical trials was already showing that statistically significant results in this field are three times more likely to be published than those that corroborate null hypotheses. However, perhaps, surprisingly, researchers concluded that, “instead of being the result of editorial decisions, this occurred because scientists were not able to write and submit papers describing their negative outcomes” (van Hilten, 2015, p.4). Many authors choose not to proceed with non-confirmatory findings, because they generate little scientific interest and few citations. As a consequence, the number of reports on non-significant data has been progressively declining (Fanelli, 2012; Matosin et al., 2014). Furthermore, there is no exaggeration in stating that academic publications that accept negative results are scarce, despite the tens of thousands circulating academic journals (Mudrak, 2019). This includes repositories aimed at publishing other research products, especially research data.

However, it is also noticeable that a latent awareness against the positive bias has been gradually triggering movements and actions through which part of the scientific community tries to reverse the pessimism that involves the publication of negative results (Matosin et al., 2014). In this direction, several academic journals were created especially to publish negative results; conventional journals open spaces to publish them or publish special issues about these findings; new ways of offering access to negative data are also emerging, for example the PubPeer (2017); they are also, increasingly, being represented in wide-ranging journals such as Plos One, which developed a collection called “The Missing Pieces: A Collection of Negative”, Null and Inconclusive Results that makes references to the “many null results filed indefinitely and, finally excluded from scientific records” (Plos One, 2015, online). This collection underscores inconclusive studies, null findings or flawed replications of other published research.

Chart 1 summarizes the survey carried out in July 2019, through Google search. The terms searched were: “journals that publish negative results” and “journal of negative results”. The chart was also complemented by the identification of examples of journals that accept negative results cited by the authors of the surveyed literature. The objective was to confirm the perception – registered by the few authors who addressed the topic –, that there are few dedicated journals or journals that accept negative results; and that these journals are concentrated in a few areas and have a low impact factor.

Chart 1
Journals that publish negative results (2000-2019).

Study area		Journal title		URL		Impact factor (2018)
Biology		Journal of Insect Science		https://academic.oup.com/jinsectscience		1.446
		Journal of Negative Results – Ecology and Evolutionary Biology		http://www.jnr-eeb.org/index.php/jnr		Not found
		Scientific Journal Negative Results		https://www.negative-results.org/		Not found
		The All Results Journals: Biology		http://arjournals.com/index.php/Biol		Not found
Biomedicine		Disease Models & Mechanisms		https://dmm.biologists.org/		4.028
		European Journal of Negative Results in Biomedicine		http://mcmed.us/journal/ejnrb		Not found
		Journal of Negative Results in BioMedicine		https://www.biomedcentral.com/search/		Not found
Natural and Clinical Sciences		Scientific Reports		https://www.nature.com/srep/		4.525
Social Sciences		Journal of Spurious Correlations		https://www.jstor.org/		Not found
Ecology		New Negatives in Plant Science.		https://www.journals.elsevier.com/new-negatives-in-plant-science		Not found
Pharmacy		Journal of Pharmaceutical Negative Results		http://www.pnrjournal.com/		Not found
Physics		The All Results Journals: Physics		http://arjournals.com/index.php/Phys		Not found
Artificial Intelligence		Journal of Interesting Negative Results in Natural Language Processing and Machine Learning		http://jinr.site.uottawa.ca/		Not found
Mathematics		Rejecta Mathematica		http://rejecta.github.io/mathematica/		Not found
Study area		Journal title		URL		Impact factor (2018)
Medicine		Journal of Negative Observations in Genetic Oncology (discontinued)		http://www.path.jhu.edu/NOGO		Not found
		Journal of Negative Results in Speech and Audio Sciences		https://www.worldcat.org/title/journal-of-negative-results-in-speech-and-audio-sciences-jnrsas/oclc/53164584		Not found
		The American Journal of Gastroenterology (There was a special edition with articles with negative results in 2016)		https://journals.lww.com/ajg/pages/default.aspx		Not found
Nanotechnology		The All Results Journals: Nanotechnology		http://arjournals.com/index.php/Nano		Not found
Nutrition and Health		Journal of Negative and no Positive Results		https://revistas.proeditio.com/jonnpr/index		Not found
Psychology		BMC Psychology		https://bmcpsychology.biomedcentral.com/		Not found
		Journal of Articles in Support of the Null Hypothesis		https://www.jasnh.com/		Not found
Chemistry		ACS Omega		https://pubs.acs.org/journal/acsodf		2.584
		The All Results Journals: Chemistry		http://arjournals.com/index.php/Chem		Not found
Multidisciplinary		F1000Research		https://f1000research.com/about		Not found
		International Journal of Negative & Null Results		https://www.journalnetwork.org/journals/international-journal-of-negative-and-null-results		Not found
		International Journal of Negative Results		https://openaccesspub.org/journal/ijnr		Not found
		Journal of Unsolved Questions		http://junq.info/		Not found
		Journal of Contradicting Results in Science		https://www.magzter.com/IN/EManuscript/Journal-of-Contradicting-Results-in-Science/Science/		Not found
		PeerJ		https://peerj.com/		2.353
		Plos One		https://journals.plos.org/plosone/		2.776

• Source: Prepared by the authors (2020).

In a first analysis of the survey outcome, journals can be classified according to their negative results acceptance policies in two categories: journals that “do not accept” negative results and those that “accept” negative results; among the journals that accept, we have: those that accept episodically; those that publish special issues about negative data and, finally, those that are dedicated to such results. There are also some platforms that are dedicated or that accept negative data, such as the F1000Research Open for Science.

The interim survey summarized in Chart 1 corroborates what the scarce literature on the topic reveals: in a universe of thousand of scientific journals, only a minimal portion is open to the publication of negative results. The survey found only 30 journals. There are many and different reasons for this to occur, mainly related to the belief, on the part of the editors, that these results are rarely mentioned and that researchers have no personal and institutional interest in disseminating these data, as discussed by several authors. Enhancing this aspect, Mudrak (2019) notes that journals that accept negative results are not heavily used and have low levels of citation for these studies. As an immediate consequence, this performance is reflected in the cautious behavior of other journals when ordering similar articles.

Another point observed in the survey is that there is a clear concentration of these few journals in the areas of life sciences and health: Biology and, especially, Medicine, Biomedicine, Pharmacy, where clinical trials are prevalent and for which there is a clamor for publishing all of them. In this connection, the international All Trials campaign, launched by Sense About Science in 2013, calls for “all past and present clinical trials to be registered and their complete methods and results to be reported” (van Hilten, 2015, p.4). The negative results in these areas are critical in ethical, economic and social terms; editors’ disregard for them and their opacity has, in some cases, disastrous consequences, as reported by Piropo (2014).

In addition to the scarcity of journals, the literature reports other reasons that lead investigators to neglect research results that are considered negative. Based on the analysis of the authors who dealt with the subject, the main reasons that lead researchers not to publish the negative results of their research were identified and systematized in Chart 2 (Fanelli, 2012; Matosin et al., 2014; Hendrix, [2016]; Guimarães et al., 2018). They cover a wide spectrum that begins with the actual idealization of science and its relationship with the valorization of finished scientific endeavors; they go through the researchers’ individual beliefs, attitudes and cognitive processes that reject their negative results; they involve the professional and academic impact of publishing negative results; the organizational and funding barriers that disqualify these results; the rigorous review of research flows and methodologies that have produced negative results and the more in-depth peer review; and, finally, the precariousness of the editorial infrastructures geared to negative results.

Chart 2
Reasons for not publishing negative results (2000-2019).

Barriers		Detailing
Contextual/scientific		• Positive bias of scientific research.
		• Prejudice against non-confirmatory research.
		• Pressure to produce “successful” research.
Individual, behavioral and cognitive		• Exposure to failure.
		• Academic vanity.
		• Perception that it is a waste of time and resources to report negative results.
		• Little personal interest in publicizing.
		• Uncertainty about results.
Professionals		• Negative impact on career and promotions.
		• Failure to recognize funding and reward schemes from funding agencies.
Organizational		• Little incentive to publish negative results by the research institution, funders and private partners.
Academic		• Passes the idea of failure in the research flow/design/methodology.
		• Perception that the formulation of the hypothesis is fragile and the team does not fully master the phenomenon studied.
		• Little scientific rigor.
		• Questioning from colleagues.
		• Academic competition for prestige, position and funding.
Editorials		• Stricter peer review.
		• Few journals that accept negative results (only some disciplines publish journals of this nature).
		• Low impact of journals that accept.
		• Low level of citation of negative results.

• Source: Prepared by the authors (2020).

Conclusion

The present study confirms the initial assumption that few scientific journals accept, edit special issues or are dedicated to the publication of negative results. Furthermore, this list of journals has relatively low impact factors – when identified –, and the journals focus on specific areas, such as Biomedicine, indicating that the publication of negative results is not a universally accepted practice in science, thus revealing the unfavorable position of negative results in the broad cycle of scientific communication.

Publication is the most important factor for academic success. In this framework, scientific editors are in a position to support a cultural change in the positive bias of scientific journals. However, for negative results to be inserted in the cycle of scientific communication, it is also necessary to change the behavior pattern of the researchers themselves, who are reluctant to publish their non-confirmatory results, as well as the reward and funding schemes that do not give the deserved support to studies which results are considered negative.

Another issue identified by the study is that the literature is scarce and poorly formalized (there are few journal articles, for example), portraying, perhaps, a possible neglect regarding this problem, on the part of Information Science and of the disciplines most affected by this imbalance. Although, at this time, the survey in an open access database indexed by Google was preferred, it is also intended to expand the survey in bibliographic databases with restricted access in order to improve the research, expand the debate and confirm this possible negligence, given that there is still a wide scope of inquiries on the topics dealt with here that need further investigation in ethical, epistemological, behavioral and infrastructural aspects.

The most obvious development of failure to publish and share negative results is the waste of resources, time and intellectual effort. This means not communicating to other researchers the “dead ends”, the research flows that can be avoided or circumvented by alternative strategies. But in addition, the opacity of negative results hides important, innovative and disruptive experiences, which help in the critical challenge of refuting prevalent paradigms and current assumptions.

On the other hand, a huge gap is also created in the scientific memory of research institutions when the intricacies of a scientific finding and its trajectory of errors and successes are no longer recorded, thus impacting the principle of reproducibility of scientific experiments and self-correction of science.

Finally, it should be emphasized that this scientific scenario, biased by the key role of positive results, can lead research projects to spurious results. All this summed up, configures a narrowing of the boundaries of scientific transparency and of the assumptions of open science.

Como citar este artigo/How to cite this article

• Sayão, L. F.; Sales, L. F.; Felipe, C. B. M. Invisible science: publication of negative research results. Transinformação, v. 33, e200009, 2021. https://doi.org/10.1590/2318-0889202133e200009

References

• Anderson, G.; Olsen, B. R.; Sprott, H. Opinion: publish negative results. The Scientists, Wilmington, Jan. 15, 2013. Available from: https://www.the-scientist.com/opinion/opinion-publish-negative-results-39928 Cited: Aug. 6, 2019.
  » https://www.the-scientist.com/opinion/opinion-publish-negative-results-39928
• Baker, M. Is there a reproducibility crisis? Nature, v. 533, n. 7604, p. 452-454, 2016. Available from: https://www.nature.com/news/polopoly_fs/1.19970!/menu/main/topColumns/topLeftColumn/pdf/533452a.pdf Cited: Dec. 10, 2019.
  » https://www.nature.com/news/polopoly_fs/1.19970!/menu/main/topColumns/topLeftColumn/pdf/533452a.pdf
• Dickersin, et al Publication bias and clinical trials. Controlled Clinical Trials, v. 8, n. 4, p. 343-353, 1987. Available from: https://www.sciencedirect.com/science/article/abs/pii/0197245687901553 Cited: Aug. 6, 2019.
  » https://www.sciencedirect.com/science/article/abs/pii/0197245687901553
• Einstein, A.; Infeld, L. Lévolution des idées en physique Paris: Flammarion, 1936.
• Fanelli, D. Do pressures to publish increase scientists’ bias? An empirical support from US states data. Plos One, v. 5, n. 4, p. 1-7. 2010. Available from: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0010271 Cited: Aug. 6, 2019.
  » https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0010271
• Fanelli, D. Negative results are disappearing from most disciplines and countries. Scientometrics, v. 90, n. 3, p. 891-904, 2012. Available from: https://link.springer.com/article/10.1007/s11192-011-0494-7. Cited: Aug. 6, 2019.
  » https://doi.org/10.1007/s11192-011-0494-7
• Guimarães, R. A. B. et al Resultados negativos na pesquisa científica: aspectos éticos. Revista Bioética, v. 26, n. 2, p. 245-250, 2018. Disponível em: http://www.scielo.br/scielo.php?pid=S1983-80422018000200245&script=sci_arttext. Acesso em: 6 ago. 2019.
  » http://www.scielo.br/scielo.php?pid=S1983-80422018000200245&script=sci_arttext
• Hendrix, S. Should I publish negative results or does this ruin my career in science? [S. l]: Smart Science Career, [2016?]. Available from: https://smartsciencecareer.com/negative-results/ Cited: Aug. 6, 2019.
  » https://smartsciencecareer.com/negative-results/
• Kuhn, T. The structure of scientific Revolution Chicago: University of Chicago, 1970. Available from: https://www.lri.fr/~mbl/Stanford/CS477/papers/Kuhn-SSR-2ndEd.pdf Cited: Aug. 6, 2019.
  » https://www.lri.fr/~mbl/Stanford/CS477/papers/Kuhn-SSR-2ndEd.pdf
• Latour, B. O objetivo da ciência não é produzir verdades indiscutíveis, mas discutíveis. [Entrevista concedida a] Juremir Machado da Silva. Correio do Povo Porto Alegre, 11 mar. 2017. Disponível em: https://www.correiodopovo.com.br/blogs/di%C3%A1logos/bruno-latour-o-objetivo-da-ci%C3%AAncia-n%C3%A3o-%C3%A9-produzir-verdade-indiscut%C3%ADveis-mas-discut%C3%ADveis-1.306155 Acesso em: 6 ago. 2019.
  » https://www.correiodopovo.com.br/blogs/di%C3%A1logos/bruno-latour-o-objetivo-da-ci%C3%AAncia-n%C3%A3o-%C3%A9-produzir-verdade-indiscut%C3%ADveis-mas-discut%C3%ADveis-1.306155
• Matosin, N. et al Negativity towards negative results: a discussion of the disconnect between scientific worth and scientific culture. Disease Models and Mechanisms, v. 7, p. 171-173, 2014. Available from: https://dmm.biologists.org/content/dmm/7/2/171.full.pdf Cited: Aug. 6, 2019.
  » https://dmm.biologists.org/content/dmm/7/2/171.full.pdf
• Michelson, A. A.; Morley, E. W. On the relative Motion of the Earth and the Luminiferous Ether. American Journal of Science, v. 34, n. 203, p. 333-345, 1987. Available from: https://history.aip.org/exhibits/gap/PDF/michelson.pdf Cited: Aug. 6, 2019
  » https://history.aip.org/exhibits/gap/PDF/michelson.pdf
• Mudrak, B. Negative results: the dark matter of research. Durham: AjeScholar, 2019. Available from: https://www.aje.com/arc/negative-results-dark-matter-research/ Cited: Aug. 6, 2019.
  » https://www.aje.com/arc/negative-results-dark-matter-research/
• Negative Results. Negative Results Scientific Journal Paris: Negative Results, ([2017]). Available from: https://www.negative-results.org/index.php/about-our-company/ Cited: Aug. 6, 2019.
  » https://www.negative-results.org/index.php/about-our-company/
• Oransky, I.; Marcus, A. Keep negativity out of politics: we need more of it in journals. Stat News, Boston, Oct. 14, 2016. Available from: https://www.statnews.com/2016/10/14/journals-negative-findings/ Cited: Aug. 6, 2019.
  » https://www.statnews.com/2016/10/14/journals-negative-findings/
• Pereira, L. V. A biblioteca de Lygia. Gol Revista, n. 210, p. 80-85, 2019. Disponível em: https://www.voegol.com.br/pt/servicos-site/Magazine/GOLREVISTA210.pdf Acesso em: 11 dez. 2019.
  » https://www.voegol.com.br/pt/servicos-site/Magazine/GOLREVISTA210.pdf
• Pfeffer, C.; Olsen, B. R. Editorial. Journal of Negative Results in Biomedicine, v. 1, p. 1-2, 2002. Available from: https://jnrbm.biomedcentral.com/track/pdf/10.1186/1477-5751-1-2 Cited: Aug. 6, 2019.
  » https://jnrbm.biomedcentral.com/track/pdf/10.1186/1477-5751-1-2
• Piropo, B. Um resultado negativo nem sempre é um mau resultado. Techtudo, São Paulo, 26 dez. 2014. Disponível em: https://www.techtudo.com.br/artigos/noticia/2014/12/um-resultado-negativo-nem-sempre-e-um-mau-resultado.html Acesso em: 10 dez. 2019.
  » https://www.techtudo.com.br/artigos/noticia/2014/12/um-resultado-negativo-nem-sempre-e-um-mau-resultado.html
• Plos One. The missing pieces: a collection of negative, null and inconclusive results. Plos One, San Francisco, Feb. 25, 2015. Available from: https://collections.plos.org/missing-pieces Cited: Sep. 12, 2019.
  » https://collections.plos.org/missing-pieces
• Popper, K. Conjectures and refutations: the growth of scientific knowledge. New York: Basic Books Publishers, 1962. Available from: http://www.rosenfels.org/Popper.pdf Cited: Dec. 10, 2019.
  » http://www.rosenfels.org/Popper.pdf
• PubPeer. PubPeer: the Online Journal club. [S. l]: PubPeer Foundation, 2017. Available from: https://pubpeer.com/ Cited: Dec. 15, 2020.
  » https://pubpeer.com/
• Smaldino, P. E. et al Open science and modified funding lotteries can impede the natural selection of bad science. Royal Society Open Science, v. 6, n. 8, p. 1-19, 2019. Available from: https://royalsocietypublishing.org/doi/pdf/10.1098/rsos.190194. Cited: Jan. 20, 2020.
  » https://doi.org/10.1098/rsos.190194
• Timbie, K. The stigma of negative data and how preprint servers can help. Blog Focused Ultrasound Foundation, Charlottesville, Feb. 14, 2019. Available from: https://www.fusfoundation.org/the-foundation/news-media/blog/the-stigma-of-negative-data-and-how-preprint-servers-can-help Cited: Jan. 20, 2020.
  » https://www.fusfoundation.org/the-foundation/news-media/blog/the-stigma-of-negative-data-and-how-preprint-servers-can-help
• van Hilten, L. G. Why it´s time to publish research “failures”. Elsevir Connect, Amsterdam, May 5, 2015. Disponível em: https://www.elsevier.com/connect/scientists-we-want-your-negative-results-too Cited: Dec. 12, 2019.
  » https://www.elsevier.com/connect/scientists-we-want-your-negative-results-too
• Zago, M. A. Pesquisadores discutem estágio atual das pesquisas com células-tronco. Fapesp na Mídia, 9 fev. 2011. Disponível em: https://namidia.fapesp.br/pesquisadores-discutem-estagio-atual-das-pesquisas-com-celulas-tronco/59463 Acesso em: 12 dez. 2019.
  » https://namidia.fapesp.br/pesquisadores-discutem-estagio-atual-das-pesquisas-com-celulas-tronco/59463

Publication Dates

• Publication in this collection
  08 Feb 2021
• Date of issue
  2021

History

• Received
  11 Mar 2020
• Reviewed
  23 Sept 2020
• Accepted
  02 Dec 2020