I protest against the method used to evaluate scientific production used in Brazil

By contributing new knowledge to a particular area of science, the publication of scientific articles is the primary objective of research projects and of the researcher. The systematic planning of a project, obedience to every step of the research protocol makes sure that the results found, whether relevant or not, are accepted by the scientific community. The result of this work will always be the publication of a scientific paper. Afterwards the author submits his contribution to a peer-reviewed scientific journal, either within his field or to a more prestigious journal, for publication.

In several editorials, the Revista Brasileira de Hematologia e Hemoterapia (RBHH) has discussed the Qualis classification system of the Coordenação de Apoio aos Profissionais do Ensino Superior (CAPES) and its effects on the Brazilian academic community^{( 1 , 2 )}. Unhappily this system punishes almost all Brazilian authors but, in particular, those that publish in journals dedicated to their specialty such as the RBHH. As the lay press reported the classification system puts Brazilian scientific journals at "risk of extinction"^{( 3 )}.

On the subject of Brazilian scientific journals, editors on several occasions have disagreed with the use of the impact factor of Institute for Scientific Information (ISI) Thomson Reuters JCR to evaluate Brazilian scientific production^{( 4 )}. Moreover, data showing the geometric growth of the number of scientific publications in Brazil and relatively low number of citations of national studies were presented in the last editorial^{( 5 )}.

Recently, via the scientific publishers list of the World Association of Medical Editors on the web, journal editors from diverse areas have been condemning cases of plagiarism and duplicate publications^{( 6 )}. Furthermore they criticize the cutting up of data resulting from a single research project with the goal of increasing the number of publications of one specific author or group of authors, a technique that received the facetious name of 'salami science'.

Although distinct, articles and texts have recently appeared in the lay press on both these subjects^{( 3 , 7 )}. The articles about CAPES were not complimentary to this government institution. In fact, discussions about the position of CAPES in respect to the evaluation of scientific production have been circulating in the scientific corridors for a few years now. Debates condemn its policies, but it seems that nothing really changes.

One of the major concerns of bureaucrats, who have little or no knowledge of true scientific research, is the need to systematically measure the worth of scientists. How do we decide, for example, whether researchers deserve promotion, pay increases, more political clout or even that grant that they have applied for? To do this, government intuitions around the world are using the number of publications and the impact factor, an index developed to compare journals and not authors. Does this make a difference?

This has created the 'Publish or Perish' culture which includes 'Salami Science'. Whether duplicate publications (which the RBHH does not accept) are ethically correct or not can be argued. Although this may waste the reader's time and, for example, distort the overall results of database searches, it seems that there may be ethical support for this practice when the author truly believes that his work will make a difference in the quality of patients' treatment. Certainly publications in different languages should not be frowned upon as most practitioners in non-English speaking countries do not speak English. This is definitely the case of Brazil where even the aforementioned government institution, CAPES, encourages all to publish in English with no importance whatsoever given to the native language. This it does by a points system where articles that are not published in ISI are assigned a very low number of points and only three articles are included in an author's scientific production evaluation. Even articles published in PubMed, for many researchers the preferred source database for research, receives half the points of an article published in the lowest ISI classification.

Furthermore, splitting a long arduous paper into simpler parts can be helpful not only to the understanding of readers but also to increase the spread of knowledge. Unfortunately today we are living in a world where people are reading less and less due to culture changes related to the modern way of life. Many individuals, including physicians, are not used to reading complicated articles anymore and either get lost in the middle, lose patience or both and so, many long articles remain relatively unread. Perhaps we should change our opinions on this subject.

And this leads me to the most important question - why do we publish? There are numerous reasons but the most common in medicine are to improve patient care, to exchange knowledge with other researchers and to improve academic careers. Of course our egos do not suffer from a good publication too.

One imagines that the primary goal of medical publications should be to improve treatment for patients. It seems fairly clear that the best way to achieve this end is to make knowledge widely and freely available to all practitioners, independent as to whether they are university professors, researchers or the ordinary physician in a small town in the middle of nowhere. We know that today the changes in medicine are rapid and that lifelong learning is essential to keep up, but for many this is not possible as the information is not readily available.

All researchers would like to publish in journals with the highest impact factors. Of course this, as renowned researchers seek to publish in The Lancet, JAMA, Nature and the New England Medical Journal, perhaps artificially and definitely wrongly, consolidates the position of the elite journals at the top. The reason I suggest this is as follows. As initially the ISI, the fore runner of Thompson Reuters, the 'owner' of the Impact Factor, only included journals published in English, this gave the North American and British journals a head start, they had impact factors before any 'foreign' journal even thought about publishing in English. As the impact factor became important to their careers, 'top' researchers obviously invested in trying to publish in these journals.

Moreover, the Lancet, JAMA, Nature and the New England Medical Journal are journals that cover all fields in medicine. This gives them a much greater opportunity of finding articles that they believe will be cited. However, for most everyday practitioners, these 'elite' journals are not the first choice. Most practitioners want all the information in one place, concentrated and at a low cost (or free) and therefore subscribe to a journal of their specialty. Hence, it seems that most authors who submit to these 'top name journals' perhaps do so, not to spread knowledge to improve the health of the population, but for their own careers. With the attitude of CAPES can we blame them?

Even HINARI, the Access to Research Initiative, in which the RBHH participates, started in 2002 by the WHO and major publishers, although a very good start, has its flaws as only medical and nursing schools, universities and research institutions in developing countries have access and not to all journals^{( 8 )}. After a physician leaves medical school, is he no longer entitled to have access to new advances in medicine?

But perhaps the solution to this problem is around the corner. Until now many articles are published in more than one database (SciELO, PubMed, ISI and in smaller databases) and also in the site of the journal itself. This distorts any calculation of the impact factor and limits it only to articles published in ISI. With the creation of the digital object identifier (DOI) the next logical step is to create a new manner to really identify the importance of scientific publications that is independent of the database or databases in which an article is 'published'. Furthermore, with the simplicity of analysis using the World Wide Web, all articles can be broken down to their source articles giving all due respect to the real discoverer of new knowledge and not just to the people who report it in review articles.

References

• ¹
  Ruiz MA. Brazilian Editors of Scientific Magazine. Changes in the criteria of Qualis. Rev Bras Hematol Hemoter. 2010;32(3):189-91.
• ²
  Ruiz MA. The journal's performance in 2011. Rev Bras Hematol Hemoter. 2011;33 (6):399
• ³
  Escobar H. Ranking coloca revistas científicas brasileiras em "riscode extinção" O Estado de São Paulo [Internet]. [cited 2013 Jun 21]; Available from: http://www.estadao.com.br/noticias/impresso,ranking-colocarevistas-cientificas-brasileiras-em-risco-de-extincao,398294,0.htm
  » http://www.estadao.com.br/noticias/impresso,ranking-colocarevistas-cientificas-brasileiras-em-risco-de-extincao,398294,0.htm
• ⁴
  Rocha e Silva M. Reflexões críticas sobre os três erres nos periódicos brasileiros excluídos. Clinics. 2011;33 (1):3-7.
• ⁵
  Ruiz MA. Reviews and guidelines: evidence of progress and a starting point for standardization and changes. Rev Bras Hematol Hemoter. 2013;35 (2):75-6.
• ⁶
  World Association of Medical Editors [Internet]. [cited 2013 Jun 21]; Available from: http://www.wame.org/
  » http://www.wame.org/
• ⁷
  Reinach F. Darwin e a prática da 'Salami Science'. O Estado de Sao Paulo[Internet]. [cited 2013 May 10]; Available from: http://www.estadao.com.br/noticias/impresso,darwin-e-a-pratica-da-salami-science-,1026037,0.htm
  » http://www.estadao.com.br/noticias/impresso,darwin-e-a-pratica-da-salami-science-,1026037,0.htm
• ⁸
  WHO. HINARI Research in Health. WHO. [Internet]. [cited 2013 May 11]; Available from: http://www.who.int/hinari/en/
  » http://www.who.int/hinari/en/

Publication Dates

• Publication in this collection
  June 2013

History

• Received
  20 June 2013
• Accepted
  24 June 2013