Scientific Performance of Brazilian Researchers in Pharmacology with grants from CNPq: A comparative study within the Brazilian categories

In Brazil, scientific performance of researchers is one important criteria for decision-making in grant allocation. In this context


Scientific performance of an individual researcher
can be evaluated at different periods of the career (i.e., by the academic position, grants awards and scientific prizes) or more objectively by using bibliometric parameters.Although bibliometric analyses have been criticized due to its limitations in assessing scientific output of an individual scientist (Campanario and Molina 2009), it is generally accepted by the scientific community as an efficient tool for assessing, measuring and comparing the JEAN P. KAMDEM et al. performance/productivity of scientists, entire institutions, departments, countries, etc (Harnad 2008, Haeffner-Cavaillon and Graillot-Gak 2009, Inglesi-Lotz and Pouris 2011, Jia et al. 2014, Díaz-Faes et al. 2015).In addition, bibliometric analyses play a major role in the critical decision of allocating funding resources and in determining policies of research (Katz 2000, Tian et al. 2008, Li et al. 2010, Pinto and Matias 2011, Besselaar and Sandstrom 2015).
In the evaluation processes of scientific performance, bibliometric parameters such as the total number of published papers and their subsequent citations are of utmost importance (Inglesi-Lotz and Pouris 2011, Peng 2011, Rodríguez-Navarro 2012, Roos et al. 2014, Zhu et al. 2014).In Brazil, the evaluation system from the Coordination for the Improvement of Higher Education Personnel (CAPES) requires high performance from scholars in terms of research, publications and, more recently, the citations has been cogitated as potential parameter for individual and institutional scientific evaluation (Meneghini 2011, Baccini et al. 2014).
Few studies have analyzed the profile and scientific output of Brazilian researchers receiving the CNPq (National Council for Scientific and Technological Development) scientific productivity grants in different fields or subfields of knowledge (Santos et al. 2010, Oliveira et al. 2011, 2013, Leite et al. 2012, Martelli-Junior et al. 2010).Those cited studies were more descriptive in nature and, as role, the citation profile of the researchers has not been evaluated in deep.We argue that citation pattern of a given author can give an indication of his/ her influence as generator of new knowledge (by analyzing citations received by original research articles, ORAs) in a field or subfield.
Here, we decided to analyze the citation pattern of researchers currently receiving the productivity fellowship from CNPq in Pharmacology subarea or subfield of biomedical sciences.The selection of this field or subfield in the BF committee (BF is the abbreviation used by the CNPq to describe the field including biochemistry, pharmacology, neuroscience, physiology and biophysics) was based on the fact that authors of the present study have a good knowledge of the area (and subarea) in Brazil, and the choice of parameters was based on the assumed assumption that the citation profile of a researcher may give a better idea about his/her performance than the place where papers were published (i.e., the journal impact factor).Furthermore, since the distribution of CNPq researchers in the levels from D (lowest grade) to A (for consolidate and highly productive investigators) in the category 1 is expected to reflect the degree of independence and the time of experience of the researchers, the analysis of the entire production can give a better idea of the importance of a given researcher in relation to these two aspects.However, to the best of our knowledge, there is no study investigating the profile of CNPq researchers from Pharmacology.
Considering that there is no information regarding the performance of Brazilian researchers in the area of Pharmacology, and that the category 1 of CNPq researchers is attributed only to researchers with scientific productivity (Arruda et al. 2009) and those considered of good quality in terms of comparisons within the Brazilian pairs, the present study was designed to analyze and compare the performance of Brazilian researchers receiving CNPq grants category 1 (1A-D) in Pharmacology.Our research was formulated as follow: (1) What is the scientific profile of those researchers regarding data sources indexed in Scopus?(2) Is there a clear difference within the four levels of CNPq category 1? To answer these questions, we analyzed and compared the total documents; total publications (research articles and review papers) of each researcher by grant category.The bibliometric indicators used here were: total documents, number of original research article (ORA) and review paper (RP), total citation, citation without self-citation, total h-index and h-index without self-citations.The number in parenthesis indicates the percentage in relation to the total number of fellows (N = 82).

data collection
The scientific performance of each researcher was evaluated using the Scival-Scopus database (www.scopus.com).The data were collected from December 2013 to June 2014.Each researcher was evaluated for his/her total production (entire career).
The parameters used to evaluate the entire career of the researchers were: total documents indexed in Scopus database, number of articles (original research and reviews), total citations (with and without self-citations) and h-index (with and without self-citations).

StatiStical analySiS
The results are expressed as mean by researcher category ± S.E.M (standard error of mean).

Scientific Productivity and total citationS
The distribution of total documents published during the entire career of the researchers as retrieved from the Scopus database in April 2014, is shown in Fig. 1a.The total documents included: conference paper, erratum, article, letter, editorial, notes, review, short survey and book chapter.Researchers from grant category 1A published about 250 documents, which is about two times the number published by researchers in the category 1B.A similar number of documents was found for researchers in category 1C and 1D, with an average of 102.83 and 91.39 respectively (Fig. 1a).
From the total documents indexed in Scopus database, we determined the number of original research articles (ORA) and review papers (RP) (Fig. 1b, c).The results demonstrate that the average of ORA published by researchers from grant category 1A (220.4) is almost two times higher than those published by researchers from grant category 1B (111.55), and that researchers from category 1C (88.29) and 1D (83.33) have similar average of published ORA (Fig. 1b).The range of distribution of total ORA by grant category was as follow: 1A (62-437), 1B (49-277), 1C (30-226) and 1D (45-138).
Similar to ORA, researchers from 1A category had the highest number of RP with an average of 9.85 (1-43), followed by researchers from grant category 1B (0-28), with a mean of 7.05 RP per researcher (Fig. 1c).The average number of RP per researcher from grant category 1C and 1D was similar (3.33) with a distribution range of 0-18 and 0-19, respectively (Fig. 1c).
Citations are generally viewed as an important indicator of the impact of the researcher in science, and it therefore serves as an objective and quantitative indicator for evaluating research performance.The h-index measures the cumulative impact of publications of a scientist, considering that the number of citations is a useful variable to evaluate the relevance and/or recognition of a paper.In the present study, the number of citations and the h-index were analyzed as quantitative (and qualitative) indicators of a scientist receiving CNPq fellowships.The results presented in Fig. 2a indicate that the number of total citations received by the researchers during the entire scientific career was related with the category of the researchers.Researchers from 1A category received 2.5 times more citations than researchers of 1B category.The mean of total citations was 5529.5 for 1A and 2207.6 for 1B category, with a distribution range of 1755 to 14349 (1A) and 842 to 6974 (1B).The total citations of researchers in category 1C and 1D presented almost the same pattern of citations, with 1622.08 (226-5865) and 1527.11 (467-3037) for 1C and 1D, respectively (Fig. 2a).

H-indeX
The h-index (with and without self-citations) associated with the total citations of the researchers is illustrated in Fig. 3a, b.The results show that the mean of h-index of researchers from grant category 1A was higher than that of category 1B, while that of researchers from grant category 1C and 1D was almost equal (Fig. 3a).The same tendency was observed when analyzing the h-index without selfcitations (Fig. 3b).In fact, the h-index reflects both the number of publications and the number of citations per publication.Most of researchers in 1A and 1B categories are normally older than that of low categories; however, their "academic age" was not taken into account in this study.Furthermore, the scopus database considers only the citations received from 1996 to the present (in the present study).The h-index grows as citations accumulate and thus it depends on the "academic age" of the researcher.Based on the fact that only the most highly cited articles contribute to the h-index, we can pre-sume that the higher level of the mean of h-index of grant category 1A and 1B, in comparison to grant category 1C and 1D may be due not only on their highly cited papers, but also to differences in the professional age.

FINAL CONSIDERATIONS
In summary, the present study evaluated and compared the profile of CNPq researchers receiving grant productivity 1A, 1B, 1C and 1D, in the area of Pharmacology, based on their total production (i.e., throughout their scientific career) as indexed in the Scopus database.In general, the results showed in this study indicated differences between researchers from grant category 1A when compared to 1B in terms of number of documents indexed, number of research articles and review papers, citations, h-index.On the other hand, there were no noticeable differences between grant category 1C and 1D in most of the parameters analyzed.Based on the bibliometric aspect investigated, it is possible to conclude that researchers from grant category 1A are well ahead of researchers of inferior categories.Consequently, the overall assessment seems to be on the expected path.However, there are some distortions in performance that are difficult to conciliate with relativistic performance analysis.Distortions occurred in all levels of fellowship.In the case of 1A and 1B researchers, it can represent the presence of some old researchers that are now no more active in terms of research, but that had done important contributions to Brazilian science in the past.The low bibliometric performance of those researchers was worsened by the fact that Scopus database only consider the citations received after 1996.Consequently, to some of those old researchers, important contribution to the field in the past might be neglected.About 10 years ago, CNPq created the senior category to encompass the researchers that had contributed in the past and now are no more as active scientist as they were in the past.The existence of such type of distortions in relation to those researchers that have contributed in the past is acceptable and may reflect the relativistic weight in the analysis and distribution of resources in Brazilian sciences.Optimistically, the presence of low performing subjects in all the categories have to be better scrutinized to define distortions that cannot be explained by relativistic performance.Indeed, the presence of researchers with low number of papers, low levels of citations suggests that his or her classification as productive scientist is biased by faults in the judgment system or by corporative interference.Overall, the results presented here may inform the scientific community and the grant agencies on the profile of PQ 1(A-D) fellows of Pharmacology, and may help to define new differences within CNPq grant categories, and consequently, a better allocation of grants.

LIMITATIONS OF THE STUDY
As our searches were performed until June 2014, citation counts of the latest publications may not reflect the actual values of these researchers.Another limitation can be the source of data, because, not all scientific literature are published in sources indexed by Scopus.In addition, Scopus presents some problems in indexing periodicals; however, the problems found in Scopus are similar to those found in other important sites such as Web of Science and Google Scholar.
It should be stressed that there were some difficulties in this work.For instance, one review paper published in Chemical Reviews was found in articles section.In such case, we considered the articles as review paper, because of the knowledge that this journal only publish review papers.

Figure 1 -
Figure 1 -Distribution of total documents (a), original research articles (ORA) (b) and review papers (RP) (c) published by Brazilian researchers from CNPq grant category 1A-D in Pharmacology.The numbers above indicate the mean of total documents, ORA or RP per researcher.

Figure 2 -
Figure 2 -Distribution of the total citations with (a) and without self-citations (b) of Brazilian researchers from CNPq grant category 1A-D in Pharmacology.The numbers above indicate the mean of citations per researcher.

Figure 3 -
Figure 3 -Total h-index with (a) and without self-citations (b) of Brazilian researchers from CNPq grant category 1A-D in Pharmacology.

TABLE II Distribution by institution, region and state of CNPq research grant (PQ 1A-D) in Pharmacology.
SP, São Paulo; RJ, Rio de Janeiro; RS, Rio Grande do Sul; PB, Paraíba; CE, Ceará; MG, Minais Gerais; SC, Santa Catarina.Dezember, 2013.Among the 26 states and the 5 regions of Brazil, the grants were distributed only in 7 states and 3 regions.