H-index in the Brazilian Academy of Sciences: comments and concerns

HUMAN SCIENCES

H-index in the Brazilian Academy of Sciences - comments and concerns

Alexander W.A. Kellner^I; Luiza C.M.O. Ponciano^{I, II}

^IDepartamento de Geologia e Paleontologia, Museu Nacional, Universidade Federal do Rio de Janeiro Quinta da Boa Vista, São Cristóvão, 20940-040 Rio de Janeiro, RJ, Brasil

^IIDepartamento de Geologia, Instituto de Geociências, Universidade Federal do Rio de Janeiro Av. Athos da Silveira Ramos, 274, CCMN, Cidade Universitária, 21941-916 Rio de Janeiro, RJ, Brasil

^{Correspondence to}

ABSTRACT

Bibliometric parameters have been used in order to evaluate a scientist's performance. The h-index has been gradually accepted as the most adequate parameter for this purpose. To have an idea of this index among Brazilian scientists, we performed an analysis of this parameter for the full members of the Brazilian Academy of Sciences (BAS). The h-index of 402 members listed in 10 distinct categories by the BAS was determined, cross-checked with the curriculum vitae of each of them listed at the Plataforma Lattes database (CVL) and compared with each other. Despite the large production, mostly in journals without impact factor, the h-indexes among the BAS members are comparatively low and show a large variation in all of the 10 categories, particularly in Biomedical and Physical sciences. The highest average of h-index values was found in Biomedical, Health and Chemical sciences; the lowest values were found in Human sciences where this index is meaningless. Several problems due to the trend that new and "fresh" publications need be constantly produced (the "bakery-effect") are discussed. This study points to the need of developing countries such as Brazil to invest in national scientific journals in order to make them gradually part of the mainstream journals. This would have a positive effect on bibliometric parameters of Brazilian researchers, including the h-index.

Key words: scientometrics, bibliometric indexes, h-index, Brazilian Academy of Sciences.

RESUMO

Índices bibliométricos estão sendo utilizados como ferramentas na avaliação do desempenho de cientistas, sendo o índice h o mais empregado atualmente. Com o propósito de tecer considerações sobre o índice h de cientistas brasileiros, foi realizada uma análise do mesmo entre os membros titulares da Academia Brasileira de Ciências (ABC). Foram calculados os valores do índice h para os 402 membros titulares separados nas 10 áreas do conhecimento distintas listadas pela ABC. Concomitantemente os trabalhos de cada pesquisador foram conferidos através da comparação com os currículos apresentados na Plataforma Lattes. Apesar da grande produção científica, a maioria em jornais sem fator de impacto, os índices h dos membros da ABC demonstram uma grande variação em todas as 10 áreas, particularmente nas Ciências Biomédicas e Físicas. As maiores médias do índice h foram encontradas nas Ciências Biomédicas, da Saúde e Químicas; os menores valores estão nas Ciências Humanas, onde este índice não reflete a produção dos respectivos pesquisadores, revelando-se inútil nesse caso. Devido a uma pressão por contínua publicação (the bakery-effect), diversos problemas relacionados são discutidos. A conclusão principal deste estudo corrobora a necessidade de países em desenvolvimento, como o Brasil, investirem em periódicos científicos nacionais, possibilitando a incorporação gradual dos mesmos nas principais listagens dos periódicos com fator de impacto, causando um efeito positivo sobre os índices dos pesquisadores atuantes no país.

Palavras-chave: cienciometria, índices bibliométricos, índice h, Academia Brasileira de Ciências.

INTRODUCTION

There is a fundamental question that gradually gains more importance in the Academia: how can we evaluate a scientist's performance and his contribution tothe field? As well known, there is a steady increase in the number of active scientists worldwide resulting in a large output in terms of publications. On the other hand, there are limitations of funding and permanent jobs. Therefore, evaluation procedures that lead to comparisons are inevitable, which have a direct effect on a researcher's career.

In order to make decisions more empirical and less subjective, bibliometric parameters were introduced,ranging from number of publications (e.g., productivity) to total citations (e.g., total impact). Some even use the amount of papers published in high profile publications such as Nature and Science to evaluate scientific performance (Ball 2007).

One of the indexes that receive most attention is the h-index, which is allegedly considered to have several advantages over previous indicators (see Hirsch 2005). Essentially, this index is based on the citations of publications received in periodicals at the Thomson-Reuter Institute of Scientific Information/Web of Science database (here abbreviated as ISI). To simplify, an h-index of 10 for a scientist means that he/she has 10 papers cited at least 10 times in periodicals of ISI. If one or more of his/her papers are cited over 100 times, his/her h-index will still remain the same; if he/she has 20 other papers cited only 10 times or less, the h-index will not change either. No other kind of production such as books, chapter in books or proceedings is being considered for this index, neither publications in journals that are not part of ISI, no matter how many times they are cited.

The h-index has gained wide acceptance and is being progressively more influential in scientific rulings such as grant proposals, promotions, general advances in careers, fellowships and post-doctorate decisions. When introduced, there was a suggestion to rank whole departments or institutions based on this index (Hirsch 2005) and it is apparently being applied by some committees, even in an unofficial and non-systematic way. Bornmann and Daniel (2005), for example, have shown that successful post-doctoral research fellowships had consistently higher h-indexes than the non-successful applicants.

In Brazil, the use of bibliometric measurement is also being gradually more considered although apparently it is not yet determinant in most areas. Publications with impact factor, which are restricted to those included in ISI, are presently the main parameter applied by CAPES to evaluate Brazilian graduate programs, and it probably will not take very long until the h-index becomes paramount in such evaluations.

To realize the range of h-index among Brazilian scientists, we determined the index values for all full members of the Brazilian Academy of Sciences (BAS). Although certainly not having every prominent scientist of the country as a member of the BAS, we believe that this analysis serves as a proxy for the scientific production in the country. Our aim is not to provide a definitive work on this subject, but to achieve some information and make comments that might help in discussions of the problems involved when applying bibliometric indexes used in scientific developed countries (e.g., USA, Germany, and England) to researchers active in developing countries.

MATERIALS AND METHODS

Only full members of the BAS listed in the academy's official site (http://www.abc.org.br/buscacad.html) in 03/January/2008 were used in the present study. A total of 405 researchers divided into 10 distinct categories as shown by the BAS (see results) were found. During the development of this study, three members passed away and were not considered due to difficulties in retrieving the necessary information, reducing the number to 402 BAS members. In order to minimize problems of homonymous authors, all publications listed in ISI were checked with the curriculum vitae of the Plataforma Lattes database (CVL). A total of 45 BAS members lacked a CVL (11,19%) and therefore their publication list could not be confirmed. The h-index for 357 BAS members was obtained from ISI through the site Periódicos Capes (http://www.periodicos.capes.gov.br).

We observed how the author has signed his scientific paper both at the BAS and in CVL database for the ISI search. In several cases, the author was listed in ISI with a distinct combination of names (e.g., Diogenesde Almeida Campos could be found as Campos, D;Campos, DA or Campos, DD; Sylvio Ferraz-Mello as Ferraz-Mello, S; FerrazMello, S or Mello, SF; Celso Pinto de Melo as de Melo, CP; de Melo, CP and Melo, CP). We tried every combination and calculated the h-index for each of them, combining the results. We did not come across a situation where the same paper was listed twice with a different combination of the author's name.

Once the h-index was established, the papers listed by ISI were checked at the site of the Plataforma Lattes in order to ensure authorship and avoid problems of homonymy. When the paper was not listed there, it was also checked at the BAS site. If not found, the paper was excluded and a new h-index obtained. This procedure may have produced results lower than the real values of the h-index, but it did not influence the general results of the present study.

The average h-index of all members in each category was obtained by adding the total of the h-indexes and dividing the sum by the total number of the members of a category. All searches were done in alphabetical order of the first name, as listed in the BAS site. All data were stored either electronically or in printed version.

RESULTS

The BAS members of Mathematical sciences (data collected between 03-08/January) show the second-lowest h-index average (7), raging from one (one member) to 15 (three members). Four members (8.5%) lack a CVL, two have disclosed their h-index and another two provided the total citation number.

In Physical sciences (data collected between 09-15/January) there was a surprisingly high number of BAS members without CVL (14,92%). The lowest h-index obtained was one (two members) and the maximum 55 (one member), with an average of 16. Four members have presented their h-index in the CVL and another one provided the total citation number.

In Chemical sciences (data collected between 16-28/January) the average h-index was 19. However, no member in this category shows values above 40, with the lowest and highest values of one (one member) and 38 (two members), respectively. A great number of members (16) in this category provided the h-index in the CVL and another presented the number of citations.Six members (13.04%) lack a CVL.

The Earth sciences category (data collected between 08-11/March) is the second area where the minimum h-index number is higher than 1: three members have values of two and the maximum was 32 (one member). Despite this, the average h-index value is comparatively low (9). Seven members (16.28%) lack a CVL, four have disclosed their h-index in the CVL and two provided the total citation number.

In Biological sciences (data collected between 15-16/March) four members lack a CVL and one did not list any publication on this database, being considered as lacking a CVL for the purpose of this study (total five, 19,23%). The lowest and highest h-index are one (one member) and 40 (one member), respectively, and the average h-index value is 13. Two have shown their h-index in the CVL and another provided the total citation number.

Biomedical sciences (data collected between 30/ January-14/March) form the largest BAS community with 96 members, eight (8.33%) of which lack a CVL. This category also displays the overall highest h-index, ranging from four (one member) to 67 (one member), and an average of 23. This category also had the largest number (26) of scientists that presented their h-index in the CVL and another presented the total citation. It should be noted that there is a comparatively high number of Brazilian journals in ISI from the Biomedical area which might explain the comparatively higher h-index.

Health sciences form one of the smallest memberships in the BAS (data collected between 16 and 30/ March). All presented a CVL, with the lowest h-index of one (one member) and the highest of 36 (one member). The average h-index is surprisingly high (20). Curiously,except for one, all have an h-index above 10 (the second lowest h-index = 13). Some BAS members in this category did not list all their publication in the CVL,some of which could be retrieved from the BAS files. Six have disclosed their h-index in the CVL, none presented the total citation number.

The Agrarian sciences have the second smallest member number in the BAS (data collected on 31/ March). Three had no CVL (16,66%). The lowest and highest h-index are one (one member) and 26 (one member), respectively, and the average is 12. Two have presented their h-index,none the total citation number.

The Engineering sciences (data collected on 8/ April) show the third lowest average values of h-indexes (8), with the lowest value of one (three members) and highest of 23 (one member). Four have shown their h-index, none the total citation number and only one member (4.45%) lacks a CVL.

The last category, Human sciences (data collected on 8/April), is also the newest at the BAS. The h-indexes are surprisingly low, with seven members lacking any. One member shows an h-index of four, which is the highest value in this category. The average of this index considering all members is one. Only one member has presented the total citation number, another one (5.88%) lacks a CVL.

While searching the CVL, we noted a variation in how accurately scientists presented their data. There are some cases in which the author presented all combinations of how he could be cited in each listed publication, what does not reflect how he was actually cited. Cases of discrepancies regarding titles and publication dates were also found. Some changed the order of their real names or omitted middle names and there have been instances of differences between the combination of the names at the BAS and CVL databases. In one occasion the author's name was misspelled in the ISI and his paper was only found due to cross-checking between the BAS and CVL data bases. It was also noted that some authors have not provided the full list of their publication in the CVL.

Based on the publications listed at the CVL, the BAS members have a very large production but comparatively low h-indexes. A general search indicates that a significant part of the scientific output of Brazilian researchers is in periodicals that are not surveyed by ISI, which was particularly observed in Physical and Earth sciences and at a lesser extent in Biomedical sciences. The extreme situation was found in Human sciences, in which several researchers listed a numerous quantity of publications, most in books and proceedings of meetings that are not covered by ISI. The average h-index of all the categories of the BAS is 15.

DISCUSSION

The comparatively high number (45 out of 402; 11,19%) of the BAS members that do not provide their information on the CVL database was rather unexpected. One possible explanation is that those researchers might have significantly reduced their scientific activity since having a CVL is mandatory in order to compete for grants at the Brazilian Scientific Council (CNPq). An age survey indicates that all scientists without a CVL are older than 57 years, but this does not necessarily confirm our previous explanation.

One striking result from our study is the remarkable variation of the h-index in all areas, which is highest in the Biomedical and Physical sciences. Except for Biomedical and Earth sciences, all other categories have at least one member with an h-index value of one. The case of Human sciences is dramatic, since seven members have no h-index at all despite their high productivity according to their CVL. The percentage of scientists with no or an h-index of one in all areas of the BAS is 2,8% and in Human sciences is 75%.

In an earlier study and using a different methodology Mugnaini et al. (2008) also observed this h-index variation in the BAS, which according to them is higher than in the members of the National Academy of Sciences of the USA (NAS). The authors pointed out that there might be other criteria for membership selection at the BAS and the possible existence of sub-areas in the same category with distinct trends of citation. Based on the present study, the authors appear to be correct in both assumptions. Regarding the first point, the final step to become a member of the BAS is to be voted by all other full members that receive a two-page information about the candidate, including honors, the five more important publications, participation in committees and a brief summary of the candidate's achievement. No information about the h-index is presented (unless specifically mentioned in the brief summary which is rarely the case), suggesting that this index is not the main criteria in the BAS selection process.

Regarding the existence of sub-areas, a brief survey among the Earth Sciences members of the BAS readily identifies two (and there are more), one related to isotope studies and other to Paleontology; the h-index of scientists active in those areas is quite distinct, favoring the former.

There is at least another possible explanation that can account for the large h-index variation within the same category in the BAS and deserves closer attention. Most senior researchers lived in a quite "different time", when the obligation for scientific survival (e.g., grants, jobs, promotions) of new publications at all times (the "bakery-effect" - see comments below) in journals with impact factor was not stressed in the same fashion as nowadays. Furthermore, bibliometric indexes were also not applied to evaluate a scientist's performance as they are these days.

The striking low h-indexes within the category of Human sciences have traditionally been explained by the predisposition of researchers in this field to favor the publication of their results in books, chapters of books and proceedings of meetings that do not make it into ISI. This is apparently a general characteristic of this area and not restricted to the BAS. However, in the NAS there is now a tendency to present more quantitative studies that are being published in significant journals (Mugnaini et al.2008). The indexes are so low within the BAS members of this area that any evaluation based on the h-index cannot even say to produce a distorted view of the field; it just does not reflect scientific performance at all.

Despite the general notion that the h-index is"here to stay" (see Ball 2007), there have been several critics of this and other bibliometric measures. They are of various nature, starting with the problem that not all relevant journals make it into ISI (Porta et al. 2003) toperiodicals with high impact factors due to an editorial line more concerned with reviews than original research (e.g., Pinto and Andrade 1999). Even the initial of an author's last name has been regarded as influential in citation potential (Tregenza 1997) and double-barreled names could end up being incorrectly listed in publication databases (Lorenzini 1996), including ISI. New indexes that could complement the h-index have also been suggested (Jin et al. 2007) but are not yet applied in a larger scale.

Other problems are more concerning. According to Glänzel and Schubert (2005), researchers in some countries such as the USA (who produces by far the largest amount of published papers in periodicals presented in ISI) and Germany tend to be more "nationalistic" when it comes to citation, what potentially has a negative effect on the h-index of scientists working in other countries (particularly in developing ones).

There is also the issue of comparing scientists that work in institutions with different infrastructures. Traditionally well funded institutions provide more research facilities than new and smaller ones. This is an international problem since there are tremendous differences in resources that countries do provide for science, but also a domestic one for the same reason (i.e., different regions, different research facilities). Large labs with lots of resources have more students, more possibilities to produce science and therefore more opportunities to generate papers in the mainstream journals. Therefore, comparing scientists working in those quite distinct realities can be regarded as comparing the time achieved by swimming athletes in a 100 meter free-style competition, one in the open sea and the second in a swimming pool situated in a covered gym and having in hand a trainer and the latest "fast swimming" suit. However it must be pointed out that one should not punish researchers of high-profile scientific institutions just because they have better working conditions. Merit and achievement should always guide scientific decisions. One of the measures that could ease this situation is to provide special funds for young scientists and for those working in new and developing research institutions.

Regarding the h-index, there is not only the question of comparing scientific achievement of individuals working in countries with distinct scientific realities, but also the problematic of using ISI only. Granted, this particular database is easy to use and widely accessible. However it does not take into account that most researchers in developing countries publish in so-called local journals that, despite their good quality and strong editorial policies regarding evaluation of papers, are not considered by ISI. Keeping in mind that scientific periodicals of the USA and several countries in Europe(e.g., England, Germany) traditionally have more possibilities to join ISI, the h-index can potentially introduce large distortions when scientists of those different countries are compared, particularly if the "nationalistic effect" observed by Glänzel and Schubert (2005) is taken into account.

There is a general notion that the h-index of researchers working in distinct scientific fields is not comparable (Hirsch 2005). This was demonstrated for the BAS before (Mugnaini et al.2008) and was confirmed by the present study. The main reasons for that is the different turnover of papers, which is regarded as independent of scientific performance but rather a peculiarity of one particular field. As a general example, in some (but not all) cases isotope analyses can be done in months and be significant for publication, while to excavate and prepare a dinosaur can take several years before any meaningful result can be achieved (Calvo et al. 2007). More important in our view is the amount of scientific activity in a certain field: the greater the number of scientists, the higher the chances (and the times) that a paper gets to be cited, which has a direct effect on theh-index value.

As pointed out before, one (and perhaps the main) reason of the large variation regarding the h-index among the BAS members may be a result of sub-areas within the same category that have a different publication turnover. This is quite concerning since one of the advocated advantage of the h-index is to compare scientists classified in the same scientific category. Therefore all arguments used to point out the difficulties and ineffectiveness of comparing such index from scientists of different fields does also apply in this case. We think that a profound study of each scientific field should be done in order to identify main sub-areas that should be separated at least for comparative purposes. In Earth sciences category of the BAS, Atmospheric sciences, Geochemistry and Paleontology are some of such sub-areas.

Another issue of great concern is the change in behavior that bibliometric indexes can introduce as they are applied more regularly, causing authors to adjust their publication and citation strategies, some of which being questionable. One problem is self-citation. Fowler and Aksnes (2007) have pointed out that more than50% of the citation received by a researcher is linked directly or indirectly to self-citation. Regarding the h-index, authors can burst it up by selectively citing their own papers, giving preference to those that have not been cited very much. This practice can have an effect on researches with low h-indexes, but is more complicated to achieve as the index goes higher. A rather simplistic approach would be to eliminate self-citations from the indexes calculations (Schreiber 2007). Concerning this measure, it has been pointed out that the removal of self-citations does not affect the comparisons between scientists (Fowler and Aksnes 2007). On the other hand, should an author be penalized for using his own work in his studies? A normal consequence of scientific activities, particularly for young researchers, is to find his own scientific niche to produce original work (and avoid "stepping on others toes"). If a research group finds such a scientific slot and is making important progress that leads to several publications, is it not expected that they will use their own work in their future studies?

With the increasing competition and the inevitable obligation of a gradual higher productivity, there has been a general trend to publish smaller papers with one aspect of the research rather than longer monographs. The space reduction of high-profile journals (normally with the highest impact factors) also stimulates this behavior, even if unintentionally. This "salami-slices-science" (e.g., Ball 2007, Castiel and Sanz-Valero 2007) as a result of the need imposed by the Academia (through funding agencies, career advancement and hiring committees) to have regularly new and "fresh" publications at "all times" (what we here call the "bakery-effect") is another problematic issue that potentially affects the h-index as well as other bibliometric parameters. One potential negative side effect of this situation is that researchers might not be willing to take many risks to end up without results and therefore direct his or her studies to areas where results are more guaranteed, and to focus on scientific topics that are in vogue rather than to explore new areas.

Perhaps the most effective solution that can at least minimize the unwanted "bakery-effect" might be the peer-review system of grants that could in some cases favor the originality of the problem to be investigated.In terms of publications, editors and reviewers can get more severe in pointing out if the results presented are enough to provide a significant advancement or observation to a certain scientific problem.

Still regarding the "bakery-effect", it has been noted that plagiarism is becoming quite common, particularly self-plagiarism.This damaging behavior affects the credibility of science (as well as potentially the bibliometric parameters) and appears to be much larger than one would expect, being a rather global phenomenon (see Errami and Garner 2008 for a detailed analysis). It should be noted, however, that the trend of high-profile journals to only consider short papers (albeit of extreme relevance) basically demands in certain scientific fields that a more complete study is published subsequently, which inevitable will in part repeat the main results already published. The same applies to review articles that, by definition, must represent the advancement of certain fields showing already published results. To correctly address this issue, it has also to be considered the significant divergences of scientific ethical conduct in different cultures. Notwithstanding those observations, we are convinced that the majority of scientists are well aware of what might constitute plagiarism, including self-plagiarism. Therefore, if tools might be given to editors (which in several cases are already available) to detect duplication of papers and this becomes a standard submission procedure, most scientists that behave in such a way might fear being exposed and therefore refrain from such a malpractice (Errami and Garner 2008).

Another major issue regarding publication is multi-authored papers. A random examination of the CVL presented by the BAS members shows that the majority of publications (particularly in certain areas) fall under this condition. This is a global phenomenon and, in extreme cases, publications show dozens of authors(e.g., The International HapMap Consortium).

These multi-authored papers constitute a quite complex and difficult subject to deal with. On one side, several countries have developed scientific policies to encourage international collaborations. This is also the case of the Brazilian funding agencies that also encourage domestic cooperation between developed research centers (generally placed in Rio de Janeiro and São Paulo) and smaller or new ones, particularly those located in the northern and northeastern parts of the country. This is a quite beneficial step forward to the development of science (e.g., changing know-how and experiences), being the generation of multi-authored papers an expected consequence. On the other hand, it has also been noted that, perhaps driven by the "bakery-effect", there seems to be a more "easy-going" attitude in including a co-author in a paper that might not have provided a substantial contribution (e.g., Ball 2007),even bordering the line of standard scientific ethical procedures (see Castiel and Sanz-Valero 2007). To make distinctions of misconduct regarding multi-authored paper worse, there have been more and more significant advancements with interdisciplinary research that will obligatorily include researchers of different fields of expertise.

How shall one proceed regarding this issue? There are no doubts about the great number of easily recognized benefits of collaboration among scientists (particularly of different disciplines) by far outweighing the unwanted side effects. Apparently this question seems to already concern several periodicals that are now asking authors to provide information regarding their role in the published study (e.g., Nature, PNAS, Science, Plos One). This procedure is expected to be followed sooner or later by all mainstream publications. How this information might end up being evaluated is still an unanswered question. In any case, the basic notion of authorship is that the contribution to a particular research project was substantial (see Syrett and Rudner 1996) and that the results could not have been achieved in that form (or speed) without it. Therefore, one might be tempted to somehow divide the indexes attached to such a publication (e.g., the number of citations, directly affecting the h-index) between all authors. It would have the advantage of discouraging the inclusion of names that have actually contributed little (if anything) to the publication. If such measures are introduced, one has to avoid the damaging effect to restrain legitimate collaborations.

A more pressing issue that surely can burst up bibliometric indexes (including the h-index) is linked with scientific journals, particularly those in developing countries. As common knowledge, there are few Brazilian periodicals that take part in the ISI database (not always related to problems regarding quality). Consequently, the remarkable difference between the h-index of the BAS and the NAS members is not surprising (Mugnaini et al. 2008). Although differences are expected due to distinct working conditions (such as funding and infra-structure) that also includes the general importance of science within the respective societies, one might wonder how those numbers would look-like if the Brazilian periodicals not in ISI would be taken into consideration.

It has been pointed out during the review process of this paper that the h-index is not necessarily linked to ISI and another database could be used, such as Google Scholar (GS; http://www.scholar.google.com). Although we will not address the discussion if this database (supposedly much larger than ISI) is more appropriate, we have made an exercise searching the papers of the first author (AWAK) in GS. Searching for the exact phrase "AWA KELLNER", 238 entries were found (as opposed to 31 in ISI). As expected, the publication list includes chapters in books and articles of journals not covered by ISI. In order to calculate the h-index, the program "Publish or Perish" (http://www.harzing.com/pop.htm) was used, despite the fact that it may give only an approximate value, as it lists some references more than once whenever it finds typos in the citation (therefore potentially lowering the h-index). This analysis resulted in an h-index of 15, almost twice the one obtained by ISI (eight). Most papers listed by GS present a higher number of citations than in the ISI database such as Kellner et al. (1994). Surprisingly, in at least one instance, the listed paper shows a lower number of citations: Fara et al. (2005) was cited five times according to ISI against three times in GS. This result indicates that a more detailed analysis has to be made in order to evaluate the consequences of using GS instead of ISI for the h-index or any other bibliometric parameter calculation.

A survey reveals that only 27 periodicals published in Brazil are considered by ISI (JCR Science Edition 2007). The highest impact factor belongs to the Journal of Brazil Chemical Society (1.539). The periodical Anais da Academia Brasileira de Ciências (AABC) has an IF of 0.895 and publishes four volumes (776 pages in 2007) a year. The correspondent journal in the USA is the PNAS (Proceedings of the National Academy of Sciences) that has an IF of 9.64 and publishes 52volumes a year (21.021 pages in 2007). It is easy to realize the brutal differences between those journals.

If bibliometric indexes (including the h-index) are the way to go, what can be done to make the scientific output of developing countries such as Brazil more accessible worldwide? Having in mind that it is natural for scientists to publish a significant part of the research outcome in the country where they live, and considering that it might not be of best interest for a country to ask every scientist only to publish abroad (what would not be feasible anyway due to space limitations of the already over-demanded international mainstream journals), the solution is to make national periodicals more competitive and to be considered by ISI. Despite the fact that Brazilian funding agencies have a program for national periodicals, presently it seems that the "cake" is being split evenly, without much consideration of the journal's importance or efforts to meet higher international standards. This would not be a problem, if the "cake" would be large enough, what is not the case. Since there is an increasing tendency to apply bibliometric factors to evaluate scientific performance, why not consider them when decisions are made for funding periodicals? Just as an example, the resources recently approved by the CNPq to support the AABC (of free access at www.scielo.br/aabc) provide less than 40% of the total costs. Periodicals that cannot find additional support (luckily not the case of the AABC, at least for the next years) have to adapt, reducing the number of issues or finding other publication alternatives (e.g., changing from printed to fully on-line versions, reducing staff).

At the same time, there must be a change in attitude of some journals regarding their editorial policies. There is no doubt that English is the current lingua franca of science and even countries where the language barrier is large (e.g., China) are working to adjust their main scientific periodicals to this reality. Therefore periodicals should provide editorial help by a professional fluent in English to assist editors and authors in improving the language of the manuscript after its acceptance. Furthermore, periodicals should make an effort to attract important papers, always keeping rigorous editorial standards for quality control. One of such attempts has recently been made by the AABC, whose outcome remains to be evaluated (Kellner and Meneghini 2008).

CONCLUDING REMARKS

Evaluation of a scientist's performance has been done in the past and will continue to be done in the future.In a world with limited resources competition is inevitable and scientists hope that comparisons are fare and less subjective as possible. In order to assist those responsible for such evaluations, bibliometric indexes have been introduced based on the notion that the publication record is a valid and determinant parameter of scientific achievement. The most used database is the one of ISI. As an example on how bibliometric measures are becoming imperative, the British government has announced that all funding assessments for universities will be based solely on such measures after 2008(Ball 2007). An example from Brazil is the study made by Packer and Meneghini (2006), who analyzed the most cited papers with at least one author affiliated to a Brazilian institution and used this information to subsequently identify nuclei of excellence in the country (Meneghini and Packer 2006). From all indexes, the h-index is regarded as the most interesting one, not yet very used in Brazil.

Notwithstanding, there are limitations in the way the h-index is currently obtained that must be taken into account, and it seems problematic to evaluate a scientist performance solely on this index. According to Hirsch (2005), a membership of the National Academy of Sciences of the USA may be associated with an h-index of 45. If such a value would be applied for the BAS,the total membership would resume to five members. Just as an exercise, even if this number would be lowered down to an h-index of 10, 35% of all present BAS members would not have been elected, including half the mathematicians, engineers and researchers of Earth sciences. The situation of Human sciences is even worse and no current BAS member of that field would have been admitted.

Interestingly an h-index of 10 would have almost no effect in the membership of Health sciences (see Table I). The reason for that is unknown, but it is possible that Brazilian scientists of that field are already more"adapted" to this index, concentrating their publication in periodicals present in the ISI database.

As pointed out before, the base of the h-index and other bibliometric parameters is ISI. This poses a problem for the Brazilian scientific community since there are only a few journals printed in the country considered by this database and perhaps one explanation for the comparatively low h-indexes of scientists of the BAS despite their high productivity. An interesting exercise might be to examine the total publications of prominent Brazilian scientists and compare them with colleagues of the USA (and perhaps also from other countries like Germany, England, China, and Argentina); this would give a general sense of productivity.

In any case, if the so called developing nations(including Brazil) want to compete regarding bibliometric indexes, their government must find a way to provide not only more funding for science in general, but also to invest in scientific periodicals published in the country, aiming to be included in ISI. In the case of Brazil, although it is important to encourage researchers to publish in mainstream journals abroad, it is paramount to make national periodicals part of those mainstream journals. Hiring professional editors that are perhaps less passionate and might be in some cases more objective in accepting or refusing contributions is one way to go. At the same time there is a need of a competent staff which must include professionals fluent in English to assist editors and authors in improving the language of the manuscript after its acceptance.

Another important measure which does not only concern Brazilian researchers is to provide ways to include books and chapters in ISI (as long as this database will continue to be used to calculate bibliometric indexes). This would be a step forward not only for Human sciences, where the h-index is useless (at least for the BAS), but for several other areas as well since many relevant and original work is published (and cited!) in such publications.

The necessity to join the ISI for what one can say "survival" has moved journals to organize and attract important papers. Some initiative, such as asking authors that publish in the periodical to cite other articles of that periodical (e.g., Plasència 2002), should be avoided, coming quite close to the line of scientific ethics.

Another problem regarding citations that has bearing on the h-index is the apparent "bad habit" of several Brazilian researchers active in the field of Chemical sciences not to cite their own Brazilian colleagues (Pinto and Andrade 1999). If this malpractice and unwise attitude (to say the least) is a general trend in the country, we would have the worst of the worlds regarding any bibliometric measurement. The firm action of the editor and the peer-review system might avoid such"habits" to prosper.

A last topic that we would like to address: how shall a scientist in modern times advise his students? There is no obvious answer to that question. What has to be made clear is that the time of Charles Darwin is long over. If this worldwide recognized English naturalist would be active today, he might have published his most famous work about the origin of species quite faster and due to the "bakery-effect" perhaps in dozens of papers and not in a book. That if, of course, he would have gotten a five-year grant to travel around the world to make his studies being a young and unknown scientist at that time.

ACKNOWLEDGMENTS

Rogério Meneghini (BIREME, OPAS-OMS and USP, São Paulo), Ronaldo Fernandes (Museu Nacional/ UFRJ), Alcides Sial (UFPE), Douglas Riff Gonçalves (Universidade Estadual do Sudoeste da Bahia), Diogenes de Almeida Campos (Museu de Ciências da Terra/ DNPM), and Waltécio de Oliveira Almeida (Universidade Regional do Cariri - URCA, Ceará) are thanked for incentive and criticisms regarding this paper. Fabiana Rodrigues Costa (Museu Nacional/UFRJ) is thanked for the revision of the language. Funding was provided by the Fundação Carlos Chagas Filho de Amparo à Pesquisa do Rio de Janeiro (FAPERJ, #E-26/152.885/2006) and Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq, #304965/2006-5). Luiza Poncianoalso acknowledges the Master fellowship of the CNPq.

Manuscript received on August 26, 2008; accepted for publication on September 9, 2008; contributed by ALEXANDER W.A. KELLNER^*

• BALL P. 2007. Achievement index climbs the ranks. Nature 448: 737.
• BORNMANN L AND DANIEL HD. 2005. Does the h-index for ranking of scientists really work? Scientometrics 65: 391-392.
• CALVO JO, PORFIRI JD, GONZÁLEZ-RIGA B AND KELLNER AWA. 2007. A new Cretaceous terrestrial ecosystem from Gondwana with the description of a new sauropod dinosaur. An Acad Bras Cienc 79: 529-541.
• CASTIEL LD AND SANZ-VALERO J. 2007. Entre fetichismo e sobrevivęncia: o artigo científico é uma mercadoria acadêmica? Cad Saúde Pública 23: 3042-3050.
• ERRAMI M AND GARNER H. 2008. A tale of two citations. Nature 452: 397-399.
• FARA E, SARAIVA AAF, CAMPOS DA, MOREIRA JK, SIEBRA DC AND KELLNER AWA. 2005. Controlled excavations in the Romualdo Member of the Santana Formation (Early Cretaceous, Araripe Basin, northeastern Brazil): stratigraphic, palaeoenvironmental and palaeoecological implications. Palaeogeography, Palaeoclimatology, Palaeoecology 218: 145-160.
• FOWLER JH AND AKSNES DM. 2007. Does self-citation pay? Scientometrics 72: 427-437.
• GLÄNZEL W AND SCHUBERT A. 2005. Domesticity and internationality in co-authorship, references and citations. Scientometrics 65: 323-342.
• HIRSCH JE. 2005. An index to quantify an individual's scientific research output. PNAS 102 (46): 16569-16572.
• JIN BH, LIANG LM, ROUSSEAU R AND EGGHE L. 2007. The R-and AR-indices: complementing the h-index. Chinese Sci Bull 52: 855-863.
• KELLNER AWA AND MENEGHINI R. 2008. A Special year for the AABC. An Acad Bras Cienc 80: 1.
• KELLNER AWA, MAISEY JG AND CAMPOS DA. 1994. Fossil down feather from the Lower Cretaceous of Brazil. Paleontology 37: 489-492.
• LORENZINI CA. 1996. Divided by a common language. Nature 384: 508.
• MENEGHINI R AND PACKER AL. 2006. Articles with authors affiliated to Brazilian institutions published from 1994 to 2003 with 100 or more citations: II - Identification of thematic nuclei of excellence in Brazilian science. An Acad Bras Cienc 78: 855-883.
• MUGNAINI R, PACKER AL AND MENEGHINI R. 2008.Comparison of scientists of the Brazilian Academy of Sciences and of the National Academy of Sciences of the USA on the basis of the h-index. Braz J Med Biol Res 41:258-262.
• PACKER AL AND MENEGHINI R. 2006. Articles with authors affiliated to Brazilian institutions published from 1994 to 2003 with 100 or more citations: I - The weight of international collaboration and the role of the networks. An Acad Bras Cienc 78: 841-853.
• PINTO AC AND ANDRADE JB. 1999. Fator de impacto de revistas científicas: qual o significado deste parâmetro? Química Nova 22: 448-453.
• PLASÈNCIA A. 2002. Lo que Gaceta Sanitaria hace para ti y lo que tú puedes hacer para Gaceta Sanitaria. Gac Sanit 16: 212-213.
• PORTA M, COPETE JL, FERNANDEZ E, ALGUACIL J AND MURILLO J. 2003. Mixing journal, article, and authorcitations, and other pitfalls in the bibliographic impact factor. Cad Saúde Pública 19: 1847-1862.
• SCHREIBER M. 2007. A case study of the Hirsch index for 26 non-prominent physicists. Ann Phys (Leipzig) 16: 640-652.
• SYRETT KL AND RUDNER LM. 1996. Authorship ethics. Practical Assessment. Res Evaluat 5: 1-4.
• TREGENZA T. 1997. Alphabetical orders. Nature 388: 511.

*

Member Academia Brasileira de Ciências

Publication Dates

• Publication in this collection
  19 Nov 2008
• Date of issue
  Dec 2008

History

• Accepted
  09 Sept 2008
• Received
  26 Aug 2008