QUALIS EVALUATION OF MEDICINE III: ANALYSIS OF ANESTHESIOLOGY AND GYNECOLOGY AND OBSTETRICS JOURNALS

Abstracts

Objective:

To know the current publication of Anesthesiology and Obstetrics and Gynecology subareas, to support the updating of Qualis Journals criteria in these specific subareas.

Method:

Cross-sectional, descriptive study in which was evaluated in quantitatively and qualitatively way the bibliographic production of Anesthesiology and Obstetrics and Gynecology subareas, from January 2010 to December 2012. Were investigated the values ​​of the impact factor; calculated (i) the number (n) and the percentage of journals in each stratum Qualis A1, A2, B1, B2, B3, B4 and B5, and (ii) the median values ​​and their extreme limits (minimum values ​​and maximum) and quartiles (p25; p50; p75; p90) of the impact factors in the different strata.

Results:

The bibliographic production of the three-year period 2010-2012 was published in 69 journals in Anesthesiology subarea and in 345 in Gynecology and Obstetrics. In Anesthesiology, 44% were within the limits of impact factor of superior A1, A2 and B1; in Obstetrics and Gynecology, 42.4% were in those limits and strata.

Conclusions:

Despite lagging behind by international standards, publications of Anesthesiology and Obstetrics and Gynecology showed tendency to improve the quality. In these sub-areas, the median of journals impact factor is beyond the limits defined by the area in the last assessment. Therefore, it must be reconsidered new indicators to assess this aspect.

Journal; Impact factor; Indexation

Objetivo:

Conhecer a publicação atual das subáreas Anestesiologia e Ginecologia e Obstetrícia, para subsidiar a atualização dos critérios Qualis-periódicos, específicos dessas subáreas.

Método:

Estudo de corte transversal, descritivo, onde avaliou-se, de modo quantitativo e qualitativo, a produção bibliográfica das subáreas Anestesiologia e Ginecologia e Obstetrícia, no período de janeiro de 2010 a dezembro de 2012. Foram definidos os valores do fator de impacto das revistas; foram calculados (i) o número (n) e percentual de periódicos em cada um dos estratos Qualis - A1, A2, B1, B2, B3, B4 e B5 e (ii) os valores de mediana e respectivos limites extremos (valores mínimo e máximo) e quartis (p25; p50; p75; p90) dos fatores de impacto nos diferentes estratos.

Resultados:

A produção bibliográfica do triênio 2010-2012 foi divulgada em 69 periódicos na subárea Anestesiologia e em 345 na subárea Ginecologia e Obstetrícia. Na subárea Anestesiologia, 44% dessa produção estava nos limites de fator de impacto dos estratos superiores A1, A2 e B1; na subárea Ginecologia e Obstetrícia, 42,4% estavam nesses limites e estratos.

Conclusões:

Apesar de aquém do desejado para os padrões internacionais, as publicações das subáreas Anestesiologia e Ginecologia e Obstetrícia mostram tendência para melhorar a qualidade. Nessas subáreas, a mediana do fator de impacto dos periódicos está além dos limites definidos pela área na última avaliação. Impõe-se, portanto, novos indicadores para a avaliação desse quesito.

Revistas; Fator de impacto; Indexação

INTRODUCTION

What is Qualis-journals?

By definition, "Qualis is the set of procedures used by CAPES to stratification of the quality of the intellectual production of postgraduate programs"¹. This instrument is based on the information included in the data collection system - today, Sucupira platform -, generating a list of dissemination vehicles, classified into different strata quality. This constitute the Qualis strata for journals, books and book chapters, among others.

In the case of Qualis-journals, stratification of the value of intellectual production on the full articles in journals is performed individually and periodically by the respective areas of assessment and defined by the impact factor of the journals used in the period. The data collection system is classified into groups A1, the highest; A2; B1; B2; B3; B4; B5 and C, the least qualified (zero weight). In the evaluation areas, the same journal can be classified into different strata, depending on the limits of the areas during the study period. According to Capes, "this is not an inconsistency, but expresses the value assigned in each area to the relevance of the journal into it. So, it is not the intent of this classification - which is specific to the evaluation process of each area - to define journals quality in an absolute way"¹.

To access the Qualis of the different areas, Capes created the WebQualis, available on the home page to all people interested and involved with postgraduate in Brazil.

The last evaluation of Qualis journals process in Medicine III

According to the document 001/2012 (Medicine III Area), the last stratification of Qualis journals was based on the indexing databases in ISI-Web of Science and also in Scimago (cites per doc), the correlation between the impact factor (IF) and cites/doc. All impact factors (ISI-Web of Science) and cites per doc of two years (Scimago basis) of all mentioned journals were evaluated. When the journal was contemplated in both bases, it was considered the most valuable impact factor. Using this criterion, there was the distribution of journals, obeying the limits (or locks) established by the CTC - ES/Capes (percentage of A1+A2 ≤ 25%; percentage of A1 <A2 and percentage of A1+A2+B1 ≤ 50%)².

The area considered all journals where articles were published in Medicine III and classified them as journals of the area and other areas (outside the area) and included 585 journals from the area and 933 from outside for a total of 1518 journals to be classified. Then it was decided that production with impact (impact factor or cite per doc) >0.01 (the ISI or Scimago bases) would be distributed till the stratum B3; journals of the area indexed on other bases would be in stratum B4 and B5, journals outside the area and indexed to other bases.

As the vast majority of journals in the area has no IF>4, initially a undergrade simulation in the strata from A1 to A2 in all other areas of journals was performed with FI>3.3 (which is the base of the stratum A1), but this simulation exceeded the limits of percentages of A1, A2 and B1, established by the CTC-ES/Capes. Thus, it was established that the journals from outside the area with 3.3>FI<4.0, would suffer undergrade in stratum A1, moving to A2. Still, this stratification exceeded the percentage of 25% of A1+A2 (=25.16%) and was not approved by Capes. Thus, were raised all journals from outside the area classified in strata A2 and of these, was selected those in which the researchers of Medicine III had not been published in the last two three-year periods (2004-2006 and 2007-2009), resulting in 20 journals. From these 20, were selected three journals of lower impact factor, which suffered undergrade from A2 to B1. The final stratification of periodic considering the IF higher of the ISI-Web of Science databases and Scimago and journals area and outside the area, was defined as Figure 1. These criteria were used to assess the Medicine III programs in the last three years (2010-2012).

FIGURE 1
- Strata and their Qualis journals boundary in Medicine III2

The discussion table: Journals - evaluation of the upgrade and Qualis periodicals

Starting new update Qualis journals process, the discussion table: Journals - evaluation of the upgrade and Qualis-journals, was coordinated by Prof. Francisco Sampaio, Professor at the State University of Rio de Janeiro (UERJ) in the 5^th Meeting of Postgraduate Medicine III programs/Capes (São Paulo, December 2014). One of the objectives of this table was to identify the current bibliographic publication of Medicine III area, to support the upgrade of the Qualis criteria for the next evaluation period. "Analysis of the Journal of Gynecology and Obstetrics and Anesthesiology" was one of the topics covered in the discussion and is the subject of this article.

The aim of this paper was to identify the current publication of the subareas Anesthesiology and Obstetrics and Gynecology (GO), to support the updating of Qualis-Journals criteria in these specific subareas.

METHOD

Study design

Cross-sectional, descriptive cohort study, which assessed is quantitatively and qualitatively the bibliographic production of Anesthesiology and GO subareas, from January 2010 to December 2012.

Search method

The search to define the study sample was to IDENTIFY journals that served as the vehicle for disseminating the research of Anesthesiology and GO and EVALUATE their Qualis strata and IF, the JCR sources (IsiWeb) and Scimago.

Sample definition

It began accessing Capes collection system to identify all the bibliographic production of the last three years (2010-2012) of all postgraduate programs in Medicine III, subarea Anesthesiology and GO. Were included publications in the years 2010, 2011 and 2012 of the following programs: Anesthesiology: Unesp/Bot-Anesthesiology and USP-Anesthesiology; subarea Obstetrics and Gynecology: UFMG-Women's Health, UNESP/Bot-Gynecology, Obstetrics and Mastology, UNICAMP- Obstetrics, UNIFESP- Medicine (Gynecology), UNIFESP- Medicine (Obstetrics), USP-Obstetrics and Gynecology, USP/RP-Gynecology and Obstetrics. Were identified the journals of the subareas, held a consultation with the respective IF (ISI-Web of Science) and cites per doc (Scimago base), considering the greater IF whenever the journal were assessed at both bases.

Data collection

Defined IF values ​​of the journals were calculated (i) the number (n) and the percentage of journals in each stratum Qualis A1, A2, B1, B2, B3, B4 and B5, and (ii) the median values and their extreme limits (minimum and maximum values) and quartiles (p25; p50; p75; p90) of IF in different strata. These calculations were made for journals of both areas.

RESULTS

The bibliographic production of the three-year period 2010-2012 was published in 69 journals in Anesthesiology subarea and in 345 in GO.

Of the total of 69 Anesthesiology journals, 44% were within the limits of IF of superior A1, A2 and B1. The median journal IF was 1.53 [0.15 - 11.99], with P25 and P90 from 0.71 to 4.79. Median values (IF=1.53) correspond to the B1 stratum boundaries (IF≥1,6 and <2.85); the P90 values (IF=4.79) are above the lower limit of stratum A1 (IF≥4,0) (Figure 2).

FIGURE 2
- Anesthesiology subarea distribution in the Qualis strata with median and quartiles of their IF

Of the total of 345 journals in the GO subarea, 42.4% were within the limits of IF of superior A1, A2 and B1. The median IF of used journals as a way for disseminating research was 2.00 [0.11-39.21], with P25 and P90 of 1.14 to 4.16. Median values (IF=2.00) correspond to the B1 strata boundaries (IF≥1,6 and <2.85); the P90 values (IF=4.16) are above the lower limit of stratum A1 (FI≥4.0) (Figure 3).

FIGURE 3
- GO subarea distribution in the Qualis strata with median and quartiles of their IF

DISCUSSION

The results of this study point to some interesting aspects related to the standard of quality and quantity of bibliographic production of Anesthesiology and GO subareas.

The first is the trend towards concentration of the publications in the upper strata of the Qualis-journals, observing that more than 40.0% of production, of both sub-areas is concentrated in the upper strata of A1 to B12. This has been going on since the previous triennium (2007-2009), when there was a predominance of publications in the strata A1-B1, B2 and B3 (ancient strata IA, IB and IC), compared to the three years from 2004 to 2006⁴. However, we must highlight the fact that the quality of publications, although growing, is still below the international standard, important issue to substantiate the internationalization of our research.

The second point is that the bibliographic production is being conveyed in journals outside of Anesthesiology and GO subareas. Today, research is being published in journals with varying scopes, especially of basic/biomolecular and translational areas. This fact is associated with improvement of research quality, the range of new tools opens with technological advances in the search for biomolecular and genetic markers, immunohistochemical and oxidative stress, among others, in addition to the improvement of new techniques and analysis equipment (Elisa, mass spectroscopy and DNA-microarray platforms, for example). This undoubtedly is the global reality of technical and scientific landscape.

These changes favor not only the quality, and make the results of our most competitive research to be published in journals with the highest impact. The end result is the increase in IF publications. As a result, there will be high on metrics and limits of Qualis journal strata. This was perhaps the most controversial issue, because the subareas in which this happens, forms vicious cycle between IF publications and mean and/or median of the subareas and Medicine III, that not always is accompanied by the update of Qualis journals system. On the other hand, the limits of Qualis periodic strata, being updated from the publications of a certain period (two years, three years or four years), may not represent the actual quality of publications at the time of evaluation, which makes this indicator as insufficient to bibliographic production.

In this context, this author proposed the adoption of new indicators to evaluate the quality of bibliographic production. Among them, the H index of each permanent teacher and of all postgraduate program, the evaluation of research communications at national and international conferences, the quantity and quality of fundraising linked to the publication, as well as institutional agencies grants and staff quality involved on the search (IC students, master's, doctoral and post-doc). In addition to these, it was suggested the possibility of defining the limits of Qualis-journal strata considering not only the median but the upper quartile (P75, for example) of the IF of the journals included in the Capes information system.

The discussion of these points was very productive and one of the short-term results is that for the next evaluation period - now four-year - is being considered to include the H index of faculty and programs as an indicator of the quality of the research area.

CONCLUSIONS

Considering the Anesthesiology and GO subareas, the result of this study shows that: 1) although below the desired level by international standards, there is a tendency to improve the quality of bibliographic production; 2) the median journals IF of the bibliographic production is beyond the limits defined by the area in the last assessment; 3) use only the IF of the journals may be insufficient to delimit the Qualis-journal strata. These facts impose the search for new indicators, alone or in combination, to evaluate the bibliographic production of Medicine III area.

REFERENCES

• ¹
  Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / CAPES [http://www.capes.gov.br]. Classificação da produção intelectual. Qualis-Periódicos. [acesso em julho 2015]. Disponível em: http://www.capes.gov.br/avaliacao/instrumentos-de-apoio/classificacao-da-producao-intelectual.
  » http://www.capes.gov.br» http://www.capes.gov.br/avaliacao/instrumentos-de-apoio/classificacao-da-producao-intelectual
• ²
  Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / CAPES [http://www.capes.gov.br]. COMUNICADO no 001/2012 -MEDICINA III. Atualização do webqualis da área. Critérios utilizados para a estratificação de periódicos. [acesso em julho 2015]. Disponível em: http://www.capes.gov.br/images/stories/download/avaliacao/Qualis_Medicina_III.pdf.
  » http://www.capes.gov.br» http://www.capes.gov.br/images/stories/download/avaliacao/Qualis_Medicina_III.pdf
• ³
  ISI-Web of Science. [acesso em julho 2015]. Disponível em: http://apps.webofknowledge.com/
  » http://apps.webofknowledge.com/
• ⁴
  SJR - Scimago Journal & Country Rank [acesso em julho 2015]. Disponível em http://www.scimagojr.com
  » http://www.scimagojr.com
• ⁵
  Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / CAPES [http://www.capes.gov.br]. Cadernos de avaliação / Produção Biliográfica. [acesso em dezembro 2014]. Disponível em: http://conteudoweb.capes.gov.br/conteudoweb/CadernoAvaliacaoServlet
  » http://conteudoweb.capes.gov.br/conteudoweb/CadernoAvaliacaoServlet
• ⁶
  Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / CAPES [http://www.capes.gov.br]. Relatório de Avaliação Medicina III/CAPES 2007-2009 (2010). [acesso em julho 2015] Disponível em: http://trienal.capes.gov.br/wp-content/uploads/2011/03/MEDICINA-III-REL-AVAL.pdf
  » http://trienal.capes.gov.br/wp-content/uploads/2011/03/MEDICINA-III-REL-AVAL.pdf

Publication Dates

• Publication in this collection
  2015

History

• Received
  19 Feb 2015
• Accepted
  12 Sept 2015