SciELO - Scientific Electronic Library Online

 
vol.53 issue3Decades of research in drug targeting using gastroretentive drug delivery systems for antihypertensive therapy author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand

Journal

Article

Indicators

Related links

Share


Brazilian Journal of Pharmaceutical Sciences

On-line version ISSN 2175-9790

Braz. J. Pharm. Sci. vol.53 no.3 São Paulo  2017  Epub Nov 13, 2017

https://doi.org/10.1590/s2175-97902017000301001 

Editorial

Impact Factor: an appropriate criterion for the Qualis journals classification in the Pharmacy area?

Mareni Rocha Farias1  2 

Bernd Heinrich Storb2 

Silvia Storpirtis3 

Silvana Nair Leite1  2 

1Departamento de Ciências Farmacêuticas, Universidade Federal de Santa Catarina (UFSC)

2Grupo de pesquisa Políticas e Serviços Farmacêuticos da Universidade Federal de Santa Catarina

3Departamento de Farmácia, Faculdade de Ciências Farmacêuticas, Universidade de São Paulo


Annually, during the evaluation process of the Graduate Programs in Brazil, the discussion about the Qualis journals classification resurfaces. In late 2016, the CAPES’s (Brazilian Federal Agency for Support and Evaluation of Postgraduate Education) pharmacy webpage1 published three important documents: (i) the 2017 Document of the Pharmacy Area, (ii) the Considerations on the Qualis Journals Classification 2016, and (iii) the article by Rita de Cassia Barradas Barata “Ten things you should know about the Qualis” (Barata, 2016). These documents explain in detail how the criteria for the journals classification in the scope of Postgraduate Programs evaluation are defined, including the purposes, applications and limitations of the system.

The system is based on strata pre-established by the CAPES Board of Directors, where A1 <A2, A1 + A2 ≤ 25% and A1 + A2 + B1 ≤ 50% of the total number of journals in which academics published their articles in the time period under review. The criteria for defining and distributing the scientific journals in each stratum are established by respective areas of evaluation, which may result in different classifications for the same journal in each of these areas. In view of that, this is an opportune time to discuss the criteria used by the Pharmacy area towards a classification that more well-balancedly comprises the various fields of knowledge related to Postgraduate Programs in the Pharmacy area.

Concerning the two bibliographic databases used by the area - (1) Web of Science® (former Web of Knowledge)/Thomson Reuters and (2) Scopus/SCImago/Elsevier -, it is crucial to understand some aspects of their structuring and purposes, and the bibliometric indicators used to analyze the journals.

Web of Science® is a comprehensive research platform whose rights belong to Thomson Reuters Publishing, with publications in the areas of science, social sciences, arts and humanities. The database allows evaluating and comparing journals with citation data drawn from approximately 12,000 academic and technical journals and conference proceedings from more than 3,300 publishers in over 60 countries. It is connected to the InCites tool and the Journal Citation Reports (JCR) module so it provides citation metrics and indicators of contents linked to Web of Science.

Scopus is a database whose rights belong to Elsevier Publishing and contains articles, abstracts, conference proceedings, books, among others, with about 67 million records. In 2016, the database counted 22,794 peer-reviewed journals, of which 3,643 are full open access. In the health area, it offers full coverage of the Medline database.

For the journals to be indexed in those databases, they are analyzed according to criteria pre-established by their respective publishers. The initiative is usually from the publisher of the journal and, once approved, maintenance expenses are to be paid. For this reason, the first limitation of such systems concerns the indexed journals databases themselves, and consequently the market dispute between two of the world’s largest scientific publishers (Oosthuizen, Fenton, 2014). The two databases in question offer bibliometric indicators to measure the prestige of the indexed journals. These indicators are essentially based on citation indices, usually from recent years, referred to as “impact factor”.

However, the pattern of article citation is highly variable in the various fields of knowledge. Examples that illustrate this point are the journals in the field of mathematics, which generally accept about ten references and show a tendency of citations of old, usually classic, articles on the subject. Another example is the field of biological sciences, whose journals accept a greater number of references (about 40) and the cited articles tend to be recent ones (Fonseca, 2015; Andrés, 2011; Pendlebury, 2009). In this sense, the two databases under analysis classify the journals into categories, considering the scope of the journal and its characteristics. For instance, in 2015 the median JCR impact factor for the Mathematics category was 0.614, while for the Biochemistry & Molecular Biology category it was 2,670. In addition, the number of journals is variable and, consequently, the total number of citations.

Therefore, the Journal Impact Factor (JIF) allows comparisons among journals only within a given subject category, as explained by the very databases. The two databases in question provide other metrics for the comparison among different categories, which are displayed in Table I.

Table I Bibliometric indicators available from the databases Web of Science/Thomson Reuters and Scopus/SCImago/Elsevier 

Indicator Database Description
JIF Journal Impact Factor Web of Science / Thomson Reuters It measures the citation index of the journal, that is, the number of citations received by articles published in the journal in the two years prior to the evaluation, divided by the number of articles published in the period, including self-citations. It considers original articles, reviews and conference proceedings. According to its creator, this indicator does not allow the comparison among journals of different categories, since these can have great variability of size and citation behaviors.
JIF Percentil Web of Science / Thomson Reuters It calculates the JIF percentiles in each subject category, considering the ranking of the journal in the respective category. Since a journal may appear in more than one category, it will display a JIF percentile value for each category.
JIF Quartile Web of Science / Thomson Reuters As a comparative measure, based on JIF percentiles in each subject category, journals are classified into quartiles (Q1, Q2, Q3 and Q4) in the respective categories. As a journal may appear in more than one category, it can have more than one “Q”.
Average JIF percentil Web of Science / Thomson Reuters With the percentiles of each subject category in which the journal is classified, an average is calculated. It allows, according to the database, comparing journals of different areas. (http://www.istl.org/09-spring/refereed1.html)
CiteScore Scopus/SCImago/ Elsevier It considers the number of citations received in the three years prior to the evaluation, divided by the number of documents published in the same period. It considers original articles, reviews, conference proceedings, editorials, errata, letters, notes, and short surveys.
CiteScore rank Scopus/SCImago/ Elsevier It calculates the CiteScore percentiles in each category of the Scopus database, considering the ranking of the journal in the respective category. A journal will have a percentile value for each category in which it is ranked.
SJR SCImago Journal Rank Scopus/SCImago/ Elsevier It measures the weighted citations received by the journal. The indicator is calculated using an iterative algorithm that distributes prestige values among journals until a stable solution is reached. The weighting considers the category and the prestige of the citation.
SNIP Source Normalized Impact per Paper Centre for Science and Technology Studies (CWTS), University of Leiden/ Scopus/ Elsevier It measures the citation impact of scientific journals. It considers the categories in which the journals are classified, smoothing out differences such as the number of citations per article, the limit of references accepted and the speed of the publication process. It allows comparing among journals of different categories within the Scopus database.

TABLE II Bibliometric data of some periodicals classified in the Qualis 2015 in the area of Pharmacy 

JOURNAL QUALIS JIF (JCR) Cite Score SJR SNIP Average JIF percentile Average CiteScores percentile JIF percentile (in the category) Category Web of Science Cite Scores percentile (in the category) Category Scopus
INDUSTRIAL CROPS AND PRODUCTS A2 3,449 3,70 1,064 1,707 91,3 94 89,286 Agricultural Engineering; 94 Agronomy and Crop Science
93,373 Agronomy
SOCIAL SCIENCE & MEDICINE B1 2,814 3,48 1,894 1,835 86,3 94 90,523 Public, environmental & occupational health SSCI ; 97 Health (social science)
88,462 Social sciences, biomedical SSCI 91 History and Philosophy of Science
80,058 Public, environmental & occupational health SCIE
JOURNAL OF ETHNOPHARMACOLOGY A2 3,055 3,54 1,156 1,551 80,5 83,5 73,729 Chemistry, Medicinal; 86 Drug Discovery;
84,091 Integrative & Complementary Medicine; 81 Pharmacology
70,307 Pharmacology & Pharmacy;
83,014 Plant Sciences
INTERNATIONAL JOURNAL OF NANOMEDICINE A1 4,32 4,88 1,351 1,322 76,7 90,4 69,277 Nanoscience & Nanotechnology; 85 Bioengineering;
84,118 Pharmacology & Pharmacy 86 Biomaterials;
94 Biophysics;
95 Drug Discovery;
92 Organic Chemistry
CYTOMETRY PART A A2 3,181 2,32 1,63 0,917 59,7 68 69,481 Biochemical research methods; 82 Pathology and Forensic Medicine
50 Cell biology; 81 Histology
41 Cell Biology
ANALYTICAL METHODS B1 1,915 1,98 0,623 0,626 55,7 74,67 47,333 Chemistry, Analytical; 60 Analytical Chemistry;
65,2 Food Science & Technology; 77 Chemical Engineering (all);
54,651 Spectroscopy 87 Engineering (all)

According to the document ‘Considerations on the Qualis Journals Classification 2016’, the Pharmacy area used as a classification criterion the Impact Factor indicator from database (1) Web of Science/Thomson Reuters for strata A1 to A2, and a combination of this and the SJR (SCImago Journal Rank) indicator from database (2) Scopus/SCImago/Elsevier for strata B1 to B4. In these cases, the indicator with the highest value was chosen to classify each journal in the mentioned strata. The indexed journals in the databases PubMed/Medline, Scielo and LILACS - instead of ISI/Web of Knowledge/Thomson Reuters and Scopus/SCImago/Elsevier - determined the classification B5.

The ‘2017 Document of the Pharmacy Area’ emphasizes the multidisciplinary and interdisciplinary nature of the Pharmacy area, which is evidenced by the diversity of the professors’ doctoral careers, the engagement of professors in other programs of different CAPES evaluation areas, and the profile of publications in the area.

Regarding the publications in the area, the document shows that the 25 scientific journals most used by professors from Pharmacy programs represent less than 2.0% of the Qualis of the area and more than 15.0% of the articles published in the period under consideration. These journals are classified in a wide variety of categories in both the Web of Science and Scopus databases. These data express the need for indicators that take such a diversity into account. Table I shows some examples of journals, in the 2015 Qualis classification, the categories to which the journals belong in the two databases, and the respective indicators.

Initially, it can be seen that the values attributed to the JIF and CiteScore indicators show a greater proximity between themselves than between those to JIF and SJR. This was expected since the SJR indicator measures the weighted citations, considering the categories.

However, the percentiles of the journals’ ranking in the respective categories indicate a variation, which is a consequence of the variation in the citation profile of the journals, as per the characteristics of the fields of knowledge. Therefore, it would be more appropriate to use the journals’ ranking in the categories to compare journals of different categories. The JCR database, particularly, provides each journal with the Average JIF % indicator; in the case of the Scopus database, the corresponding value can be calculated.

Although the JIF and CiteScore indicators, and respective percentiles, are based on similar assumptions, the values are different for the same journal, since they use their own systems, which depend on the journal’s registration in the database. The two databases under analysis are key for the Pharmacy area, especially in strata A1 and A2, as most journals are registered in the two of them. With respect to the percentiles, among the 25 most cited journals used in the area, the majority of them has a higher average percentile value in the categories of the Scopus database than in that of JCR.

Finally, it is worth noting that these indicators have been internationally discussed as to their applicability for evaluating the quality of publications (Fonseca, 2015). The use of the designation ‘Impact Factor’ for the citation index calculation in a recent period leads to the misinterpretation that this represents the impact of the journal. The citation per se does not necessarily mean that the article has quality. The classic example is the publication on cold fusion, which received between the years 1988-1992 approximately 700 citations, but largely negative ones (http://www.scielo.br/pdf/qn/v22n3/1101.pdf). Moreover, these indicators measure only recent citations and do not consider the classic references, which are generally of greater academic impact.

Considering that both the (1) Web of Science/Thomson Reuters and (2) Scopus/SCImago/Elsevier databases provide bibliometric indicators that allow comparing the impact/quality of the journals, it seems to be outdated to classify the journals based on the impact factor using absolute counts instead of relative measures (Pendlebury, 2009).

Another point to consider about the system is the induced preference for publishing in high-ranked journals, which can be classified in a certain stratum - regardless of the bibliometric indicators used - due to their relevance to the area. The Pharmacy area has four induced journals, two of them are classified as B2, one B3, and the other B5.

Considering that the journals classification criteria in the different areas of the Qualis system evaluation are defined by each corresponding area, it is worth noting that there is space and an urgent need to deeply discuss the fact that, ultimately, the evaluation should underline aspects such as: the current system’s real contribution to the quality assessment of the publications; possible biases; and the perspectives towards proposing a more balanced and consistent system embracing the multi- and interdisciplinary nature of the Pharmacy area.

REFERENCES

Andrés A. Evaluating research using impact and Hirsch factors. Europhysics News. 2011:42(2):29-31. [ Links ]

Barata A. Dez coisas que você deveria saber sobre o Qualis. Rev Bras Pós-grad. 2016;13(30):13-40. [ Links ]

Fonseca, J. Para além do fator de impacto: O artigo científico e a disseminação de conhecimento em Medicina. Rev Port Imunoalergol. 2015;23(2):67-9. [ Links ]

Oosthuizen JC, Fenton JE. Alternatives to the impact fator. Surgeon. 2014;12(4):239- 243. [ Links ]

Pendlebury DA. The use and misuse of journal metrics and other citation indicators. Arch Immunol Ther Exp. 2009;57(1):1-11. [ Links ]

1https://www.capes.gov.br/component/content/article/44-avaliacao/4671-farmacia

*Correspondence: M. R. Farias. Departamento de Ciências Farmacêuticas, Centro de Ciências da Saúde, Universidade Federal de Santa Catarina. Campus Trindade, CEP 88040-900, Florianópolis, SC. E-mail: marenif@pq.cnpq.br

Creative Commons License This is an open-access article distributed under the terms of the Creative Commons Attribution License