Acessibilidade / Reportar erro

SURGERY AND CARDIOVASCULAR SURGERY JOURNALS ANALYSIS

Abstracts

Objective:

To analyze critically the effectiveness and value of bibliometric indicators in journals of Surgery or Cardiovacular Surgery in the context of the postgraduate programs of CAPES Medicine III.

Methods:

A sampling with 16 academic programs and one professional master of Medicine III, encompassing the General and Digestive System Surgery, Cardiovascular Surgery and Multidisciplinary courses with such contents, was evaluated. Thomson Reuters/ISI (JCR), Elsevier/Scopus (SJR), and also Scielo databases were used.

Results:

Only in seven programs, the teachers had an average of Qualis A1 articles greater than the others strata. Eleven journals in the surgical area are in stratum A1 (5%) and it reaches 25% in Cardiovascular Surgery. Among the six journals with the largest number of publications Qualis A1 in area Medicine III, five are from non-specific areas. The Acta Cirúrgica Brasileira represented 58% of the publications in the stratum A2. There are some obstacles in the Qualis classification with little uniformity among the Medicine areas I, II and III.

Conclusions:

A permanent committee should be set to update the Qualis, composed by the three medical areas. It should be considered using other index databases and the unification of the Qualis criteria for journals in medicine. Rating criteria of multi and transdisciplinary journals need to be reviewed. It is essential an institutional financial support for national journals chosen by peers aiming to provide a full computerization process and a professional reviewer of the English language, in order to increase the impact factor.

Postgraduate education; Bibliometrics; Indicators; Educational measurement


Objetivo:

Analisar criticamente a eficácia e valor de indicadores bibliométricos dos periódicos da Cirurgia e Cirurgia Cardiovascular no contexto dos Programas de Pós-Graduação da área Medicina III da CAPES.

Métodos:

Foi avaliada uma amostragem com 16 programas acadêmicos e um mestrado profissional da área de Medicina III, compreendendo a Cirurgia Geral e do Aparelho Digestivo, a Cirurgia Cardiovascular e Cursos Multidisciplinares com estes conteúdos. Utilizaram-se as bases de dados Thomson Reuters/ISI (JCR), Elsevier/Scopus (SJR), além do Scielo.

Resultados:

Somente nos programas 7, os docentes têm média de artigos Qualis A1 maior ou igual que os demais estratos. Apenas 11 periódicos da área de Cirurgia estão no estrato A1 (5%) e na Cirurgia Cardiovascular são 25%. Dos seis periódicos com o maior número de publicações na área Medicina III, Qualis A1, cinco não são específicos da área. A Acta Cirúrgica Brasileira representou 58% das publicações no estrato A2. Há alguns óbices na classificação Qualis com pouca uniformidade entre as Medicinas I, II e III.

Conclusões:

Se impõe a criação de um comitê permanente para atualização do Qualis, composto pelas três áreas da Medicina. Deve-se ponderar a utilização de outras bases de indexação e a unificação dos critérios Qualis para as revistas das áreas de medicina. Critérios de classificação dos periódicos multi e transdisciplinares precisam ser revistos. É imprescindível suporte financeiro institucional aos periódicos nacionais escolhidos pelos pares visando informatização completa e prover revisão profissional da língua inglesa, tendo como meta a elevação do fator de impacto.

Educação de Pós-Graduação; Bibliometria; Indicadores; Avaliação educacional


INTRODUCTION

Understand the assessment system of Qualis Capes Medicine III within the community of the area and its use, is essential to contribute to the evaluation process of the programs in surgery. The Qualis has the function to stratify the quality of the intellectual production of potgraduate programs, assessing scientific journals in medicine. It is one of the parameters for strategies and resource allocation policies, and result in encouraging the faculty with nucleation and ability to consolidate research groups in the country. The use of international databases and the application of bibliometric indicators of our scientific production permit to show, clearly, the domestic production and its specificities, used in all Capes knowledge area.

The objective of this paper was to, critically, analyze the effectiveness and value of bibliometric indicators of journals in Surgery and Cardiovascular Surgery in the context of Medicine III post​​graduate programs.

METHOD

We analyzed a sample of all the programs of Medicine III (36), comprising 16 programs (27 academic courses and one professional master - MP) restricted to the General and Digestive Surgery, Cardiovascular Surgery and any Multidisciplinary Programs that included in its content the aforementioned specialties (Figure 1). Were excluded anesthesiology, obstetrics and gynecology, ophthalmology and otolaryngology, orthopedics, plastic surgery, thoracic surgery and urology. The study was performed in the range of the last triennial evaluation (2010-2012)11. Relatorio de Avaliação 2010-2012 - Trienal 2013. Área de avaliação: Medicina III. Disponivel em: http://conteudoweb.capes.gov.br/conteudoweb/CadernoAvaliacaoServlet?acao=filtraArquivo&ano=2012&codigo_ies=&area=17 Acesso:janeiro/2015.
http://conteudoweb.capes.gov.br/conteudo...
, consulting, year by year, the indicators of each evaluation forms of the respective programs/courses. Were also consulted Thomson Reuters - ISI Web of Science (JCR), Elsevier/Scopus (SJR), and the Scielo databases, on bibliometric indicators22. Sistema Web Qualis. Portal CAPES. 2015. Disponível em:

3. http://qualis.capes.gov.br/webqualis/publico/pesquisaPublicaClassificacao.seam?conversationPropagation=begin Acesso:janeiro/2015.SJR
http://qualis.capes.gov.br/webqualis/pub...
-44. SCImago Disponível em: http://www.scimagojr.com/ Acesso:janeiro/2015.
http://www.scimagojr.com/...
.

RESULTS

In the document area (Medicine III, 2013) it can be seen that only in seven programs, the teachers had an average of articles Qualis A1 equal or greater than the other strata (horizontal lines). In general, the strata decreases in direct proportion to the program note (vertical lines), except concept 6, in which is relevant stage B2 (Table 1). Data from 2010: average of 1.9 publication/teacher with median at 2.21.

FIGURE 1
- Programs and courses used in sampling

Searching in ISI Web of Science the journals in the surgical area, were found only 11 in stratum A1, average 5.7 and median 5.19 (Figure 2).

FIGURE 2
- Representation of 10 journals with high impact factor in the surgery area (ISI/JCR); arrows indicate some of A1, in which area published more frequently

There were 140 A1 publications in the three-year period for the programs analyzed. In the area, there were five publications in The New England Journal of Medicine (FI-JCR/2014, around 54); two in Nature (FI-JCR/2014, around 48); Nature Medicine (FI-JCR/2014 around 28). It was also highlighted the Cochrane Library, a program in which only accounted for 11 articles. There were five publications in the journal Annals of Surgery during the triennial. It is observed that among the six A1 journals, five are not specific in surgery or cardiovascular surgery (Figure 3).

FIGURE 3
- Distribution of preferred journals Qualis A1 in Medicine III in the analyzed sample: only one journal was specific in surgery and had five publications (Annals of Surgery, oval pointed).

Also according to the ISI Web of Science source, the classification of Brazilian journals specific to the surgical area, the Brazilian Archives of Cardiology, the Brazilian Journal of Cardiovascular Surgery and the Brazilian Surgical Acta occupy the 11th, 50th and 55th positions, respectively (Figure 4).

FIGURE 4
- National journals rating in Web of Science

TABLE 1
- Scientific production in the strata A1 to B2 of the Qualis by permanent teaching versus Notes of graduate programs of Medicine III

Regarding the strata A2, in the production of the last triennial were pointed 420 publications in 97 journals, 171 (40%) concentrated in nine. Fifty-eight percent of publications in this stratum were published in the Brazilian Surgical Acta (Figure 5). It can be seen in this stratum that only a periodic (Nutrition) is not specific to surgery.

FIGURE 5
- Preferred distribution of publications classified as Qualis A2 in Medicine III area in the analyzed sample

On the classification Qualis B1 and B2, the Journal of the Brazilian College of Surgeons, the Brazilian Archives of Cardiology and the Journal of Cardiovascular Surgery accounted for 147 publications in the area in the last three years. Two programs have published 22 articles in the Journal of Cardiovascular Surgery in 2012 (Figure 6).

FIGURE 6
- Distribution of three national journals in Surgery and Cardiovascular Surgery classified in Qualis B1 and B2: 77 in the Journal of the Brazilian College of Surgeons, 51 in the Journal of Cardiovascular Surgery and 19 in the Brazilian Archives of Cardiology

In 2013, according to the ISI Web of Science, the surgery had only 5% of journals available for publication in the stratum A1. In turn, the Cardiovascular Surgery (in Heart and Cardiovascular System category) had 21% of all specialty journals classified as Qualis A1. In some other specialties of Medicine III this indicator was zero, ie, there was no journal in the stratum A1, like the otolaryngology. Among the universe of indexed journals in multidisciplinary areas or basic sciences, the percentage in stratum A1 ranged from 15-29% (Table 2).

TABLE 2
- Periodic percentage in stratum A1 by specialty or category classified by the Web of Konowledge in 2013

The constant evolution of Qualis results in the need for constant validation and some obstacles are detected. Stands the distinction between print and online Surgical Brazilian Acta classified in stratum B3 and B4, respectively. Also stands for the same journal, wide variations in the Qualis among medicines. The Neurosurgery (IF, and JCR SJR=3.0) is found in the stratum A2 and B1 in Medicine I, II and III. The Surgical Endoscopy and Other Interventional Techniques which IF is above 3.2, has Qualis B5 in Medicine III and A1 in medicines I and II (source Qualis 2014).

DISCUSSION

Qualitative and quantitative indicators in science, technology and innovation have been strengthened in the last decade, with the recognition of educational managers and national scientific community of the need for instruments to set guidelines and guide resource allocation policies and the formulation of programs and evaluation of activities related to scientific and technological development in the country (Mugnaini).

A broad sampling of this study, covering almost 50% of Medicine III programs allowed to obtain a reliable picture of the surgical area through the analysis of bibliometric indicators of scientific publications in the course of the last three-year (2010-2012). Were used, as sources, the data on the area document and the old Capes mechanism of evaluation (Coleta), with its indicators, as well as the databases consecrated by the academic community11. Relatorio de Avaliação 2010-2012 - Trienal 2013. Área de avaliação: Medicina III. Disponivel em: http://conteudoweb.capes.gov.br/conteudoweb/CadernoAvaliacaoServlet?acao=filtraArquivo&ano=2012&codigo_ies=&area=17 Acesso:janeiro/2015.
http://conteudoweb.capes.gov.br/conteudo...

2. Sistema Web Qualis. Portal CAPES. 2015. Disponível em:

3. http://qualis.capes.gov.br/webqualis/publico/pesquisaPublicaClassificacao.seam?conversationPropagation=begin Acesso:janeiro/2015.SJR
http://qualis.capes.gov.br/webqualis/pub...

4. SCImago Disponível em: http://www.scimagojr.com/ Acesso:janeiro/2015.
http://www.scimagojr.com/...
-55. JCR Web of Science. 2015..

With the evolution of computers and the World Wide Web it was possible to integrate an exponential number of data on specific banks for this purpose around the world. In the field of medicine some bases contain significant part of world production among which stand out the Science Citation Index (ISI- Web of Science), Scopus and the Scientific Electronic Library Online (SciELO), this comprising Latin America and the Caribbean. It is noteworthy that all come from private systems and commercialized. Capes and CNPq adopt, preferably, the ISI. It is thinkable whether this would be the most appropriate way, because it does not include the indexing of a substantial number of journals, often from developing countries, especially those incorporated into PubMed, still without impact factor, whose article already receive international citations. This is the case of the Journal of Brazilian College of Surgeons, whose IF in Scopus/SJR, for example, stands at 0.58. Of course, the impact factor also deserves reflection. It measures the importance or influence of a journal, based on citations of articles as time goes by66. Documento de área 2013. CAPES. Área de avaliação: Medicina III. Disponivel em: http://www.capes.gov.br/images/stories/download/avaliacaotrienal/Docs_de_area/Medicina_III_doc_area_e_comiss%C3%A3o_att08deoutubro.pdf Acesso:janeiro/2015.
http://www.capes.gov.br/images/stories/d...
-77. Mugnaini R, Jannuzzi PM, Quoniam L. Indicadores bibliométricos da produção científica brasileira: uma análise a partir da base Pascal. CiInf., Brasília 2004;33(2):123-31.. The use of the IF, transparent criteria to qualify jurnals, can cause bias. Review articles or containing guidelines can be much cited and not show the impact of an unprecedented production. The excessive self-citations can raise the impact factor without this denotes a greater visibility of the article. A valuable publication and originality in national journal may not have an impact and meritorious recognition. The editorial ethics can be subverted by substantial financial contribution paying for publication, often in the face of its media potential88. Castro RCF. Journals in surgery and gastroenterology: indexing in databases and bibliometric indicators. Acta Cir Bras. 2006;21(3):122-32.. So, it is necessary to discuss about the impact factor. In this context, Petroianu99. Andriolo, A et al. Classificação dos Periódicos no Sistema QUALIS da CAPES - a Mudança dos Critérios é URGENTE!. Arq. Bras. Cardiol.2010;94(3):290-1. put in our minds a relevant questioning: "There are probably good intentions in establishing such standards; however, the assumptions concerning the scientific publication and its dissemination vehicles need to be reconsidered."

It is clear that the Brazilian surgical area is growing in importance in medicine taking in account 2013 SCImago (Elsevier) portal 33. http://qualis.capes.gov.br/webqualis/publico/pesquisaPublicaClassificacao.seam?conversationPropagation=begin Acesso:janeiro/2015.SJR
http://qualis.capes.gov.br/webqualis/pub...
,77. Mugnaini R, Jannuzzi PM, Quoniam L. Indicadores bibliométricos da produção científica brasileira: uma análise a partir da base Pascal. CiInf., Brasília 2004;33(2):123-31.,1010. Petroianu A. Perversidade contra a publicação médica no Brasil. Rev Col Bras Cir. 2011; 38(5):290-1.: the 12th place among the countries with the highest number of citations of their articles in this specialty. It should be noted that in the previous three years (2007-2009) the stratum A1 was defined with the impact factor (IF) greater than 2.96. In the latter three-year period (2010-2012) stratification A1, respecting the institutional recommendations, amounted to a value greater than or equal to 4.05. Therefore, the stratification of journals in Medicine III area has reflected this qualitative leap, including equivalence to the ratings of other areas of medicine (I and II), once far away.

The bibliometric cut shows that Medicine III presents consistent production in A1 periodicals, despite the concentration in some programs, not necessarily 6 and 7. It is observed dispersion evident with the choice of journals with multidisciplinary approaches, with a predominance of other areas of knowledge (such as the gastroenterology and endoscopy) or several other specialties and various fields of knowledge (such as in the journal Plos One and the Cochrane Library). This scenario generates obviously some differences in the classification of journals, according to the database. For example, in both SCImago and Thompson Reuters, the journal Annals of Surgery keeps on top of the ranking, but the British Journal of Surgery reaches the third and fifth places in the top 10 listed on consulting the two bases, respectively.

By choosing between pairs of the area, the Brazilian Surgical Acta and the Journal of the Brazilian College of Surgeons were upgraded to the strata A2 and B1, respectively. In the period 2007-2009 were B2 and B3 and the current criteria are applied would be classified as B3 and B4, since the parameter is based on the JCR impact factor. The validity wonders of this strategy and maintaining the current status quo is beneficial to the area and for the journals. Since the restoration of the stratification of Qualis Capes from A1 to C, there remains controversy with regard to the appreciation of the Brazilian journals. This study showed that both the Surgical Brazilian Acta and the Journal of the Brazilian College of Surgeons were, in fact, the ones that more publications materialized in the strata A2 and B1. It is questionable its practical result. Editors of both journals report that the strategy adopted not only promoted magnification in demand, as well as resulted in increasing the quality of the articles submitted. The Brazilian Surgical Acta published 97.5% of original articles1111. Población DA, Goldenberg S, Montero EFS, Moreira MB, Pellizzon RF. Revistas brasileiras publicadoras de artigos científicos em cirurgia. I - Características estruturais e administrativas das revistas. Acta Cir Bras. 2002;17(6):359-69. last year. It became monthly in 2012 and therefore the impact factor has not expressed the consequences of this change. Moreover, especially on the first (Acta Cir Bras) there was a greater international visibility, given the demand for submission of articles by authors from other nationalities. This consolidation is reflected in the development of both, facilitating in the case of the second (Rev Col Bras Cir), the indexing in PubMed, lying on its way to achieving its IF/JCR in the ISI Web of Science1212. Goldenberg Saul. Acta Cirúrgica Brasileira em 2012: análise retrospectiva. Acta Cir. Bras. 2013;28(1):1-4..

Another controversial aspect concerns the cutoff for stratification in the Qualis. Every year rises the average and median area of ​​publications, and this is reflected in reducing the impact of journals in higher strata (A1 and A2), for some specialties. In 2013, the JCR/ISI and the SJR/Scopus for Brazilian Archives of Cardiology, Journal of Cardiovascular Surgery and Surgical Brazilian Acta were 1,124 and 1:03; 0.632 and 0:58 and finally 0:57 and 0.67, respectively. The mean and median of the journals "top ten" Qualis A1 in the database surgery category Web of Science is located around 5.2. Only 5% of journals specific to surgery are in stratum A1, that is, of the 202 listed only 11 have impact factor greater than or equal to 4. In Cardiovascular Surgery default is broader, since there are 27 regular A1 between 125 journals indexed (21%). Obviously in the basic sciences the percentage of regular A1 rises considerably (around 25-30%) and the surgeon is able to publish in this niche and/or multidisciplinary journals such as in the areas of gastroenterology and hepatology, endocrinology and metabolism, transplantation, biochemistry and molecular biology. Of course, interests the academic surgeon and researcher disclose the article among their peers, but this intention cause permanent conflict, given that, in most cases, or it publishes in journals with lower impact on his area, or ignore it and search for journals with more impact, whose visibility is reduced in his specialty88. Castro RCF. Journals in surgery and gastroenterology: indexing in databases and bibliometric indicators. Acta Cir Bras. 2006;21(3):122-32.-99. Andriolo, A et al. Classificação dos Periódicos no Sistema QUALIS da CAPES - a Mudança dos Critérios é URGENTE!. Arq. Bras. Cardiol.2010;94(3):290-1.. It is clear that, with the search engine on the internet, including PubMed, it is possible to screen the choice of text. However, in the current context, given the shortage of time and the necessary objectivity in their updates, the surgeon restricts reading the very few articles in just one or two reference journals in his specialty.

The Qualis system, although it is open to criticism, is still the best we have to apply bibliometric indexes, but there will always be need to improve it. In the globalized world, with access to PubMed (NIH) and eventually to the full text, it does not make sense to use printing on paper, except for library files or any interests of authors. In the third millennium, the search for articles is done electronically (internet), representing paper and resources savings, with the vaunted of sustainability. So, it´s not well with the common sense allocate the same journal in two distinct strata. An example, can be found in Brazilian Surgical Acta classified differently in Qualis of both Medicines I and II, because of its publishing vehicle (online - B4 or printed - B3). Paradoxically, the online version that offers increased visibility has the lowest Qualis. It has the same scientific text, absolutely unique in its content, evaluated by the same criteria and editorial board. If the journal, still appearing with two ISSN, considering both versions, is indexed in international databases, there is no reason to qualify it unevenly.

Another relevant bias concerns the criteria adopted for the classification of a journal in the Qualis, among the various areas of medicine. If currently the stratification of medicines is almost equal, is not anymore justified a journal with unique impact factor be qualified in different strata, or in contravention disparate ratings in which the same journal receives Qualis A1 in one area of ​​medicine and B5 in another, even with an impact factor equal or above 3.0. This is the case of Neurosurgery and the Surgical Endoscopy and Other Interventional Techniques. Obviously there are obvious differences, such as B1 to A2.

Each area has autonomy to classify its journals in the upper strata given the limits set by the CTC (25% A1 + A2, A1 <A2 and A1 + A2 + B1 <or = 50%) and can better consider or not some journals. Medicine III opted to choose two journals and modified its Qualis.

However, it is emphasized that if the cut for insertion into the Qualis is pretty much the same among the "three medicines" should prevail IF, with unification of this classification, at the risk of producing inconsistencies between areas. It is essential to reevaluate the percentage margin of the strata, because there is great possibility of plastering the incorporation of journals with high impact factor, particularly in multidisciplinary areas, ignoring the evolution of knowledge on the basis of inter and transdisciplinarity in constant growing, so present and necessary to the current scientific world.

CONCLUSIONS

  • 1) It should be mister the creation of a permanent committee to update the Qualis, which comprises the three areas of medicine, given feedback throughout Excel sheets allowing real-time analysis and re-stratification of a journal serving in the same stratum to the three medicines.

  • 2) It is necessary to consider the need to use other index databases without exclusivity for the ISI Web of Science and Scimago to mark the grading in Qualis Capes.

  • 3) It should be considered the assess unification of Qualis criteria for the journals of all fields of medicine, considering in this quadrennium similar stratification.

  • 4) Multi- and transdisciplinary coupled with translational research - actions desirable for science integration on the third millennium - should provide changes in the paradigm used for the classifications of journals in the Qualis Capes in the upper strata.

  • 5) To offer institutional financial support for national journals chosen among peers to be computerized in current international standards, if they are not, as well as provide the costs for professional English language version, enabling the journal in reasonable time to increase its impact factor.

REFERENCES

  • 1
    Relatorio de Avaliação 2010-2012 - Trienal 2013. Área de avaliação: Medicina III. Disponivel em: http://conteudoweb.capes.gov.br/conteudoweb/CadernoAvaliacaoServlet?acao=filtraArquivo&ano=2012&codigo_ies=&area=17 Acesso:janeiro/2015.
    » http://conteudoweb.capes.gov.br/conteudoweb/CadernoAvaliacaoServlet?acao=filtraArquivo&ano=2012&codigo_ies=&area=17
  • 2
    Sistema Web Qualis. Portal CAPES. 2015. Disponível em:
  • 3
    http://qualis.capes.gov.br/webqualis/publico/pesquisaPublicaClassificacao.seam?conversationPropagation=begin Acesso:janeiro/2015.SJR
    » http://qualis.capes.gov.br/webqualis/publico/pesquisaPublicaClassificacao.seam?conversationPropagation=begin
  • 4
    SCImago Disponível em: http://www.scimagojr.com/ Acesso:janeiro/2015.
    » http://www.scimagojr.com/
  • 5
    JCR Web of Science. 2015.
  • 6
    Documento de área 2013. CAPES. Área de avaliação: Medicina III. Disponivel em: http://www.capes.gov.br/images/stories/download/avaliacaotrienal/Docs_de_area/Medicina_III_doc_area_e_comiss%C3%A3o_att08deoutubro.pdf Acesso:janeiro/2015.
    » http://www.capes.gov.br/images/stories/download/avaliacaotrienal/Docs_de_area/Medicina_III_doc_area_e_comiss%C3%A3o_att08deoutubro.pdf
  • 7
    Mugnaini R, Jannuzzi PM, Quoniam L. Indicadores bibliométricos da produção científica brasileira: uma análise a partir da base Pascal. CiInf., Brasília 2004;33(2):123-31.
  • 8
    Castro RCF. Journals in surgery and gastroenterology: indexing in databases and bibliometric indicators. Acta Cir Bras. 2006;21(3):122-32.
  • 9
    Andriolo, A et al. Classificação dos Periódicos no Sistema QUALIS da CAPES - a Mudança dos Critérios é URGENTE!. Arq. Bras. Cardiol.2010;94(3):290-1.
  • 10
    Petroianu A. Perversidade contra a publicação médica no Brasil. Rev Col Bras Cir. 2011; 38(5):290-1.
  • 11
    Población DA, Goldenberg S, Montero EFS, Moreira MB, Pellizzon RF. Revistas brasileiras publicadoras de artigos científicos em cirurgia. I - Características estruturais e administrativas das revistas. Acta Cir Bras. 2002;17(6):359-69.
  • 12
    Goldenberg Saul. Acta Cirúrgica Brasileira em 2012: análise retrospectiva. Acta Cir. Bras. 2013;28(1):1-4.
  • 13
    Manso JEF, Malafaia O, Ferraz EM. A Revista do CBC está no Medline / PubMed. Rev. Col. Bras. Cir. 2009;36(6):470-1.

Publication Dates

  • Publication in this collection
    2015

History

  • Received
    19 Feb 2015
  • Accepted
    12 Sept 2015
Colégio Brasileiro de Cirurgiões Rua Visconde de Silva, 52 - 3º andar, 22271- 090 Rio de Janeiro - RJ, Tel.: +55 21 2138-0659, Fax: (55 21) 2286-2595 - Rio de Janeiro - RJ - Brazil
E-mail: revista@cbc.org.br