A closer look at the Impact Factor (JCR 2012): problems, concerns and actions needed

The present editorial is a little bit different from the ones regularly published in the Anais da Academia Brasileira de Ciências (AABC). Instead of focusing on contributions of the current issue as we normally do (e.g., Kellner 2011), here we report a situation that occurred with the AABC in 2012 and that might be of general interest for editors and anyone else that uses bibliometric parameters to evaluate scientific performance (as CNPq, CAPES and Graduate Programs do in Brazil), since it is directly related to the Impact Factor (IF), Journal Citation Reports^® (JCR) by Thomson Reuters JCR and the Web of Knowledge^SM (WoS). We do not intend to discuss the JCR or the importance of the IF and all the useful data published about each journal indexed in the JCR. However, in the light of the present case, we would like to point out that problems with the JCR do exist, that they are serious and that new actions and measures should be taken in order to mitigate the resulting complications.

For quite some time now, June has become the month of the year in which the anxiety of journal's Editors reaches the highest level because of the release of the Journal Citation Reports^® (JCR) by Thomson Reuters. The AABC is the sole multidisciplinary journal produced in Brazil, publishing mainly original research in distinct areas such as Mathematical Sciences (e.g., Kwakkel et al. 2011), Paleontology (e.g., Jiang and Wang 2011), Biomedical Studies (e.g., Cruz et al. 2012), Agriculture (e.g., Monteiro et al. 2012), Social Sciences (e.g., Rocha et al. 2012) and everything in-between. The four issues per year have gradually increased from about 15 to 35 articles per issue since 2007. The year 2012 was not different and the volume 84 was published with four issues. However, by the end of July 2012, we noticed that the issue 84(2) had not yet appeared in the WoS. A few months later the issue 84(3) was included and all papers published made available in the WoS, but still no sign of issue 84(2). When issue 84(3) was made available in the WoS, AABC contacted Thomson Reuters and submitted several requests alerting about the missing issue, most only resulting in their standard e-mail response. With the help of the Thomson Reuters office in Brazil and the organization Scientific Library On Line (SciELO, based in São Paulo), it was only in May, 1^st 2013, that the issue 84(2) finally appeared in the WoS. Problem solved and 7 weeks later the JCR 2012 was published (mid-June) with the new IFs, which in the case of the AABC, was down from 1.094 to 0.851. A revision of AABC's IF was requested but Thomson Reuters informed AABC that there is nothing they could do about this at this point, a quite disappointing notice. AABC was not expecting any significant changes in its IF, but we were aware that at least 3 valid citations (articles published in 2010-2011) from papers published in the 84(2) issue were not used in the IF calculation because of the delay in having the issue included in the WoS. If they had been counted the AABC's 2012 IF would only change to 0.865, a modest 1.64% increase over the published IF. The current IF actually reveals that the variation is within the normal AABC's IF trend over the last 5 years: 0.881, 1.074, 0.925, 1.094 and 0.851 for JCR 2008, 2009, 2010, 2011 and 2012, respectively (data from the JCR).

What was interesting and somehow worrying in reviewing all 25 articles published in the AABC 84(2) issue is the fact that from over 700 listed references, 55 of them concerned papers published in 2010-2011 in 45 different journals (Rev Palaeobot Palynol, Remote Sens Environ, Brazil J Geod, Sedim Geol, J Geol, Geology, J South Am Earth Sci, Mineral Mag, Environ Sci Pollut Res, J Soils Sedim, Mar Environm Res, Mar Pollut Bull, Conserv Genet Resour, PLoS One, Neurosci Lett, Blood, Nature, J Biol Chem, Int J Parasitol, Biochim Biophys Acta, J Pineal Res, Antioxid Redox Signal, Thromb Haemost, Purinergic Signaling, Cell Microbiol, Arthritis Rheum, Science, Eukaryot Cell, Malar J, PLoS Pathog, Exp Parasitol, Parasitology, J Agr Food Chem, Plant Physiol Biochem, Plant Methods, Planta, Environ Exp Bot, Process Biochem, Mol Breed, Plant Growth Regul, Ann Appl Biol, New Phytol, Plant Sci 2010, Plant Biol, Analyst). All of them would have counted to these journal's IFs published in the JCR 2012 if this issue would have been processed. Once again, we suspect that those extra citations would not have resulted in significant changes to these journals' IFs although the actual impact might vary. For instance, in an extreme case, Nature received 3 valid citations, which is insignificant compared to the 65,731 citations the journal has received in 2012. Yet, it is amazing to see the number of citations that were not used for IF calculation due to the delay in adding the AABC 84(2) issue to the WoS database. A true “bibliometric domino effect”!

The present case raises the question if such incidents are rare or if they occur more regularly with volumes/issues from other periodicals as well. If similar “missing issues” or “late issue additions” regarding this prestigious database are more common and not corrected, than the credibility of the system is at stake. Perhaps it might be interesting that someone would look more deeply into this question and perform a fully evaluation of what can be called the “missing issue” or “late issue addition factors” in the IF of periodicals with distinct IFs (e.g., IF higher than 3, from 3 to 1, and lower than 1). Curiously, after the JCR data were published in June 2013, the section “2012 JCR Data Update” of the JCR Notices (http://admin- apps. webofknowledge.com/JCR/static_html/notices/notices.htm) has been updated weekly with additions or adjustments to the JCR data. This section is set to be closed in September 2013. By checking the list changes carried out to the JCR 2012 data, 98 updates were performed and for 58 (59.2%) of them the “reason for update” was “missing issues”. So, adjustments and corrections are possible, but, as already mentioned, an update of the AABC's IF due to the missing issue 84(2) was not granted so far.

One should note that the editorial board of several periodicals produced in the so-called ‘developing world’ are working very hard to get their journals as part of the international mainstream journals, what necessarily passes through the IF. Efforts vary from inviting special papers on hot topics (e.g., Verjovski-Almeida 2011) and organizing complete volumes on relevant subjects for a particular field (e.g., Kellner and Tomida 2011), to showing directions in a specific area where the journal would like to be heading (e.g., Azevedo 2012). In what can be perhaps regarded a more extreme case, some journals even changed their titles into English, the lingua franca of science, hoping to make the journal more attractive to both, authors and readers. Furthermore, there is the growing trend to evaluate and compare science achievement of authors, departments, institutions and even countries. Despite the complexity and the inevitable controversy associated with this issue (e.g., Pinto and Andrade 1999, Hirsch 2005, Jin et al. 2007, Ball 2007, Mugnaini et al. 2008, Kellner and Ponciano 2008), scientific publications have become paramount in this matter. For all this to work properly, there must be a reliable database where the presented IF of journals can be trusted.

One of the key concerns of the AABC case reported here is the fact that the 84(2) 2012 issue entered the WoS some 8-10 months later (1^st of May 2013) than it should have normally entered, which means that the 24 articles and 1 editorial published were not made available to the users of this important database for a much longer period than expected. As a consequence this specific issue of AABC most likely has lost the chance of being cited more often in 2013 papers since its visibility to the users might have been reduced. Consequently, we can assume that the 84(2) issue may also contribute with fewer citations than normally expected and possibly with a smaller impact on the journal's 2013 IF to be published in June 2014. Still part of the “bibliometric domino effect”.

Finally, we have also noticed that 17 citations granted to papers published in 2010-2011 of WoS 2012 were incorrectly cited. Among the mistakes are spelling errors of author names, volume numbers, issue numbers, page numbers and the year of publication. As a matter of fact, these gross errors by authors are a major problem. Because of these errors, the 17 citations that would have counted for the 2012 IF were not used in the calculation. The good news is that the WoS offers the possibility to suggest corrections to their database, improving the quality of the data for the readers and statistics generated by the system. Therefore, it is important that editors and their staff check regularly the citations received by the journal and make sure that any corrections needed are sent to the Thomson Reuters as soon as possible so all citations are counted. If the 17 errors would have been included in the IF calculation, plus at least the 3 citations from the 84(2) issue, the IF would have been 0.947, a 11.3% increase in the 2012 IF.

As a consequence, the editorial office of the AABC is now considering to establish a routine check of every citation granted to the papers published in the journal of the preceding year, as well as working more closely with authors of accepted papers to ensure that the cited references will be free of errors. Perhaps other periodicals that are not already doing so might also follow this initiative to ensure more accuracy regarding citations. Just another task for the editorial board of scientific journals in what is becoming an increasingly complex scientific world where an expressive number of researchers (particularly the younger generation) are being driven by what has been defined as the “bakery effect” (= the need for constantly producing new and “fresh” publications, preferably in journals with a high IF; see Kellner and Ponciano 2007). This reality arguable leads to the production of “salami-slices-science” (e.g., Castiel and Sanz-Valero 2007), not to speak of the mounting plagiarism and self-plagiarism. Furthermore, editors have to deal with the high importance given to indexes and other bibliometric parameters, to which the evaluation of databases' reliability has just been now added.

REFERENCES

• Azevedo RA. 2012. Publications in the field of Agrarian Sciences in the Anais da Academia Brasileira de Ciências: what next? An Acad Bras Cienc 84: 1-3.
• Ball P. 2007. Achievement index climbs the ranks. Nature 448: 737.
• Castiel LD and Sanz-Valero J. 2007. Entre fetichismo e sobrevivência: o artigo científico é uma mercadoria acadêmica? Cad Saúde Pública 23: 3042-3050.
• Cruz LN, Wu Y, Craig AG and Garcia CRS. 2012 Signal transduction in Plasmodium-Red Blood Cells interactions and in cytoadherence. An Acad Bras Cienc 84: 555-572.
• Hirsch JE. 2005. An index to quantify an individual's scientific research output. PNAS 102: 16569-16572.
• Jiang S and Wang X. 2011. A new ctenochasmatid pterosaur from the Lower Cretaceous, western Liaoning, China. An Acad Bras Cienc 83: 1243-1249.
• Jin BH, Liang LM, Rousseau R and Egghe L. 2007. The R-and AR-indices: complementing the h-index. Chinese Sci Bull 52: 855-863.
• Kellner AWA. 2011. Vegetation replacement in Rio de Janeiro during the last 7,000 years B.C., genetic variation in the medicinal plant Lippia alba (Verbenaceae), and short review of Chagas disease in prehistory. An Acad Bras Cienc 83: 761-762.
• Kellner AWA and Ponciano LCMO. 2008. H-index in the Brazilian Academy of Sciences - comments and concerns. An Acad Bras Cienc 80: 771-781.
• Kellner AWA and Tomida Y. 2011. Proceedings of the Third Gondwanan Dinosaur Symposium. An Acad Bras Cienc 83: 1-2.
• Kwakkel F, Martens M and Peixoto M. 2011. Focal rigidity of flat tori. An Acad Bras Cienc 83: 1149-1158.
• Monteiro CC et al. 2012. Biochemical and histological characterization of tomato mutants. An Acad Bras Cienc 84: 573-585.
• Mugnaini R, Packer AL and Meneghini R. 2008. Comparison of scientists of the Brazilian Academy of Sciences and of the National Academy of Sciences of the USA on the basis of the h-index. Braz J Med Biol Res 41: 258-262.
• Pinto AC and Andrade JB. 1999. Fator de impacto de revistas científicas: qual o significado deste parâmetro? Química Nova 22: 448-453.
• Rocha MSP, Santiago IMFL, Cortez CS, Trindade PM and MourÃo JS. 2012. Use of fishing resources by women in the Mamanguape River Estuary, Paraíba state, Brazil. An Acad Bras Cienc 84: 1189-1199.
• Verjovski-Almeida S. 2011. Schistosoma mansoni at the Molecular and Cellular Biology of Helminth Parasites VI Meeting. An Acad Bras Cienc 83: 355-356.

Publication Dates

• Publication in this collection
  Sept 2013