Annually, during the evaluation process of the Graduate Programs in Brazil, the discussion about the Qualis journals classification resurfaces. In late 2016, the CAPES’s (Brazilian Federal Agency for Support and Evaluation of Postgraduate Education) pharmacy webpage1 1 https://www.capes.gov.br/component/content/article/44-avaliacao/4671-farmacia published three important documents: (i) the 2017 Document of the Pharmacy Area, (ii) the Considerations on the Qualis Journals Classification 2016, and (iii) the article by Rita de Cassia Barradas Barata “Ten things you should know about the Qualis” (Barata, 2016Barata A. Dez coisas que você deveria saber sobre o Qualis. Rev Bras Pós-grad. 2016;13(30):13-40.). These documents explain in detail how the criteria for the journals classification in the scope of Postgraduate Programs evaluation are defined, including the purposes, applications and limitations of the system.
The system is based on strata pre-established by the CAPES Board of Directors, where A1 <A2, A1 + A2 ≤ 25% and A1 + A2 + B1 ≤ 50% of the total number of journals in which academics published their articles in the time period under review. The criteria for defining and distributing the scientific journals in each stratum are established by respective areas of evaluation, which may result in different classifications for the same journal in each of these areas. In view of that, this is an opportune time to discuss the criteria used by the Pharmacy area towards a classification that more well-balancedly comprises the various fields of knowledge related to Postgraduate Programs in the Pharmacy area.
Concerning the two bibliographic databases used by the area - (1) Web of Science® (former Web of Knowledge)/Thomson Reuters and (2) Scopus/SCImago/Elsevier -, it is crucial to understand some aspects of their structuring and purposes, and the bibliometric indicators used to analyze the journals.
Web of Science® is a comprehensive research platform whose rights belong to Thomson Reuters Publishing, with publications in the areas of science, social sciences, arts and humanities. The database allows evaluating and comparing journals with citation data drawn from approximately 12,000 academic and technical journals and conference proceedings from more than 3,300 publishers in over 60 countries. It is connected to the InCites tool and the Journal Citation Reports (JCR) module so it provides citation metrics and indicators of contents linked to Web of Science.
Scopus is a database whose rights belong to Elsevier Publishing and contains articles, abstracts, conference proceedings, books, among others, with about 67 million records. In 2016, the database counted 22,794 peer-reviewed journals, of which 3,643 are full open access. In the health area, it offers full coverage of the Medline database.
For the journals to be indexed in those databases, they are analyzed according to criteria pre-established by their respective publishers. The initiative is usually from the publisher of the journal and, once approved, maintenance expenses are to be paid. For this reason, the first limitation of such systems concerns the indexed journals databases themselves, and consequently the market dispute between two of the world’s largest scientific publishers (Oosthuizen, Fenton, 2014Oosthuizen JC, Fenton JE. Alternatives to the impact fator. Surgeon. 2014;12(4):239- 243.). The two databases in question offer bibliometric indicators to measure the prestige of the indexed journals. These indicators are essentially based on citation indices, usually from recent years, referred to as “impact factor”.
However, the pattern of article citation is highly variable in the various fields of knowledge. Examples that illustrate this point are the journals in the field of mathematics, which generally accept about ten references and show a tendency of citations of old, usually classic, articles on the subject. Another example is the field of biological sciences, whose journals accept a greater number of references (about 40) and the cited articles tend to be recent ones (Fonseca, 2015Fonseca, J. Para além do fator de impacto: O artigo científico e a disseminação de conhecimento em Medicina. Rev Port Imunoalergol. 2015;23(2):67-9.; Andrés, 2011Andrés A. Evaluating research using impact and Hirsch factors. Europhysics News. 2011:42(2):29-31.; Pendlebury, 2009Pendlebury DA. The use and misuse of journal metrics and other citation indicators. Arch Immunol Ther Exp. 2009;57(1):1-11.). In this sense, the two databases under analysis classify the journals into categories, considering the scope of the journal and its characteristics. For instance, in 2015 the median JCR impact factor for the Mathematics category was 0.614, while for the Biochemistry & Molecular Biology category it was 2,670. In addition, the number of journals is variable and, consequently, the total number of citations.
Therefore, the Journal Impact Factor (JIF) allows comparisons among journals only within a given subject category, as explained by the very databases. The two databases in question provide other metrics for the comparison among different categories, which are displayed in Table I.
Bibliometric indicators available from the databases Web of Science/Thomson Reuters and Scopus/SCImago/Elsevier
Bibliometric data of some periodicals classified in the Qualis 2015 in the area of Pharmacy
According to the document ‘Considerations on the Qualis Journals Classification 2016’, the Pharmacy area used as a classification criterion the Impact Factor indicator from database (1) Web of Science/Thomson Reuters for strata A1 to A2, and a combination of this and the SJR (SCImago Journal Rank) indicator from database (2) Scopus/SCImago/Elsevier for strata B1 to B4. In these cases, the indicator with the highest value was chosen to classify each journal in the mentioned strata. The indexed journals in the databases PubMed/Medline, Scielo and LILACS - instead of ISI/Web of Knowledge/Thomson Reuters and Scopus/SCImago/Elsevier - determined the classification B5.
The ‘2017 Document of the Pharmacy Area’ emphasizes the multidisciplinary and interdisciplinary nature of the Pharmacy area, which is evidenced by the diversity of the professors’ doctoral careers, the engagement of professors in other programs of different CAPES evaluation areas, and the profile of publications in the area.
Regarding the publications in the area, the document shows that the 25 scientific journals most used by professors from Pharmacy programs represent less than 2.0% of the Qualis of the area and more than 15.0% of the articles published in the period under consideration. These journals are classified in a wide variety of categories in both the Web of Science and Scopus databases. These data express the need for indicators that take such a diversity into account. Table I shows some examples of journals, in the 2015 Qualis classification, the categories to which the journals belong in the two databases, and the respective indicators.
Initially, it can be seen that the values attributed to the JIF and CiteScore indicators show a greater proximity between themselves than between those to JIF and SJR. This was expected since the SJR indicator measures the weighted citations, considering the categories.
However, the percentiles of the journals’ ranking in the respective categories indicate a variation, which is a consequence of the variation in the citation profile of the journals, as per the characteristics of the fields of knowledge. Therefore, it would be more appropriate to use the journals’ ranking in the categories to compare journals of different categories. The JCR database, particularly, provides each journal with the Average JIF % indicator; in the case of the Scopus database, the corresponding value can be calculated.
Although the JIF and CiteScore indicators, and respective percentiles, are based on similar assumptions, the values are different for the same journal, since they use their own systems, which depend on the journal’s registration in the database. The two databases under analysis are key for the Pharmacy area, especially in strata A1 and A2, as most journals are registered in the two of them. With respect to the percentiles, among the 25 most cited journals used in the area, the majority of them has a higher average percentile value in the categories of the Scopus database than in that of JCR.
Finally, it is worth noting that these indicators have been internationally discussed as to their applicability for evaluating the quality of publications (Fonseca, 2015Fonseca, J. Para além do fator de impacto: O artigo científico e a disseminação de conhecimento em Medicina. Rev Port Imunoalergol. 2015;23(2):67-9.). The use of the designation ‘Impact Factor’ for the citation index calculation in a recent period leads to the misinterpretation that this represents the impact of the journal. The citation per se does not necessarily mean that the article has quality. The classic example is the publication on cold fusion, which received between the years 1988-1992 approximately 700 citations, but largely negative ones (http://www.scielo.br/pdf/qn/v22n3/1101.pdf). Moreover, these indicators measure only recent citations and do not consider the classic references, which are generally of greater academic impact.
Considering that both the (1) Web of Science/Thomson Reuters and (2) Scopus/SCImago/Elsevier databases provide bibliometric indicators that allow comparing the impact/quality of the journals, it seems to be outdated to classify the journals based on the impact factor using absolute counts instead of relative measures (Pendlebury, 2009Pendlebury DA. The use and misuse of journal metrics and other citation indicators. Arch Immunol Ther Exp. 2009;57(1):1-11.).
Another point to consider about the system is the induced preference for publishing in high-ranked journals, which can be classified in a certain stratum - regardless of the bibliometric indicators used - due to their relevance to the area. The Pharmacy area has four induced journals, two of them are classified as B2, one B3, and the other B5.
Considering that the journals classification criteria in the different areas of the Qualis system evaluation are defined by each corresponding area, it is worth noting that there is space and an urgent need to deeply discuss the fact that, ultimately, the evaluation should underline aspects such as: the current system’s real contribution to the quality assessment of the publications; possible biases; and the perspectives towards proposing a more balanced and consistent system embracing the multi- and interdisciplinary nature of the Pharmacy area.
REFERENCES
- Andrés A. Evaluating research using impact and Hirsch factors. Europhysics News. 2011:42(2):29-31.
- Barata A. Dez coisas que você deveria saber sobre o Qualis. Rev Bras Pós-grad. 2016;13(30):13-40.
- Fonseca, J. Para além do fator de impacto: O artigo científico e a disseminação de conhecimento em Medicina. Rev Port Imunoalergol. 2015;23(2):67-9.
- Oosthuizen JC, Fenton JE. Alternatives to the impact fator. Surgeon. 2014;12(4):239- 243.
- Pendlebury DA. The use and misuse of journal metrics and other citation indicators. Arch Immunol Ther Exp. 2009;57(1):1-11.
-
1
https://www.capes.gov.br/component/content/article/44-avaliacao/4671-farmacia
Publication Dates
-
Publication in this collection
2017
