Scientometrics: Nature Index and Brazilian science

A recent published newspaper article commented on the (lack of) quality of Brazilian science and its (in) efficiency. The newspaper article was based on a special issue of Nature and on a new resource for scientometrics called Nature Index. I show here arguments and sources of bias that, under the light of the principle in dubio pro reo, it is questionable to dispute the quality and efficiency of the Brazilian science on these grounds, as it was commented on the referred article. A brief overview of Brazilian science is provided for readers to make their own judgment.

A recent published article by Dr. Rogério Cezar de Cerqueira Leite in the Folha de S. Paulo (Leite 2015) commented on the (lack of) quality of Brazilian science and its (in) efficiency.The newspaper article was based on a supplement of Nature [2014;515 (7526): S49-S108; see http:// www.nature.com/nature/supplements/natureindex-2014-global/index.html] and on a new database for scientometrics called Nature Index (NI; Nature Index 2016), which encompass three metrics.In one of the alarming highlights of the article, Brazil's scientific efficiency was compared with Chile's (15 times more efficient than Brazil): in 2013 Brazil spent US$ 30 billion in Science and published 670 articles "in the most rigorous journals regarding to its scientific quality," while in the same period Chile published 717 articles in those same journals while investing only US$ 2 billion.
The Nature Index (2016) encompass three metrics: 1) Article count (AC): A count of one is assigned to an institution or country if one or more authors of the research article are from that institution or country, regardless of how many co-authors there are from outside that institution or country; 2) Fractional count (FC): A fractional count that takes into account the percentage of authors from that institution (or country) and the number of affiliated institutions per article.For calculation of the FC, all authors are considered to have contributed equally to the article.The maximum combined FC for any article is 1.0; 3) Weighted fractional count (WFC): A modified version of FC in which fractional counts for articles from specialist astronomy and astrophysics journals have been down VALTER SILVA weighted.These journals encompass a much larger proportion of the total publication output of these fields than any other field covered by the Nature Index.The WFC allows ordering of institutions and countries so as not give emphasis to these fields.The weighting is achieved by multiplying the count from these astronomy and astrophysics journals by a factor of 0.2.This down weighting is in proportion to an approximation of the level to which astronomy and astrophysics articles are overrepresented compared to the total publication output of other fields covered by the Nature Index.These three metrics use the number of publications that come from institutions and countries in selected journals.The selection of these 68 journals was carried out by polling 68 scientists for their 10 favorite journals in which they would publish their best scientific works.There was an attempt to confirm the 68 journals by polling via email 100,000 life, physical, and medical scientists, allegedly from a broad geographical mix.Only 2,800 (2.8%) out of 100,000 scientists answered the questionnaire.The 68 journals, which are part of the NI, constitute less than 1% of journals in the Journal Citation Reports (2013 JCR, ISI Web of Knowledge, Thomson Reuters), but are responsible for about 30% of the citations in natural sciences.Interestingly enough, journals that are recognized as the top 10 of the 100 most influential journals of the 20th century in biology and medicine are not part of the journals list of the NI (e.g., The New England Journal of Medicine, JAMA, BMJ, The American Journal of Botany, The Journal of Zoology, The American Journal of Physical Anthropology and The Journal of Paleontology; see http://dbiosla.org/publications/resources/topten.html).
Similar to other metrics of science (e.g., Impact Factor, h-index), NI has limitations: 1) The main criticism for NI is that there is no apparent reason for the selection of journals.3) The indicators depend more on the absolute number of publications of an institution or country in the selected journals than on the methodological quality or impact of these publications.Furthermore, it would be more logical to measure performance based on individual activities rather than journals.
Given the limitations discussed here and based on principle of in dubio pro reo, it is questionable to dispute the quality of Brazilian science and its efficiency based on the assertions given by Dr. Leite.Moreover, there is a high risk of ecological fallacy in the comparison regarding efficiency in the use of resources applied in research.The US$ 30 billion invested should yield results in the middle and long term, so probably not implying in the 670 published articles.Considering only the NI, there is limited evidence regarding the quality and efficiency of Brazilian science; although the 2012-2015 change in WFC, the main metric of NI, showed an increase of 18.9% for Brazil, ranked 24th globally, 8 places ahead of Chile (Nature Index 2016).

NATURE INDEX AND BRAZILIAN SCIENCE 1599
To be fair, Chile is a noteworthy example of science internationalization. Chilean scientists tend to publish high-impact papers in collaboration with international teams in large expensive coordinated funded from abroad.Based on the ratio of AC to FC (Nature Index 2016), there is an of the internationalization degree: Chile: 1069/212.70= 5.0; Brazil: 960/304.33 = 3.2.The superiority of Chile's science internationalization is similar when analyzed based in data from SCImago (2016) (see Figure S6).On the other hand, it may also indicate that Chile's dependency on external resources to do science is about 1.6 times higher than Brazil.
A brief overview of Brazilian science production is provided for readers to make their own judgment (see Supplementary Material).The data are based on 29,713 journals and 239 countries/ settings contained in the Scopus® database (Elsevier B.V.) and retrieved by the scientometric portal SCImago Journal & Country Rank (SCImago 2016).Figures S1 to S3 are exclusively about Brazilian science production and show analyses of the citable papers (Figure S1), number of citations received (Figure S2) and average of cites per paper (Figure S3).Figures S4 to S6 allow comparing Brazilian science with world and Latin America (Figure S4 and Figure S5), as well as with Chile (Figures S5 and Figure S6).
Finally, I agree with Dr. Leite that much can be done to help Brazilian science grow not only in quantity, but also in quality (e.g., less bureaucracy, more internationalization, more meritocratic and well-managed financial incentives).Another important issue to highlight and solve is that, formally, the profession of scientist in Brazil is not a recognized position by the Ministry of Labor and Employment.