DEVELOPMENT OF METRICS FOR ARTICLES PUBLISHED IN PROFESSIONAL MASTER'S COURSE

Abstracts

Objective:

Propose metric to qualify the production conveyed through articles published in Professional Master's Programs and, from there, to establish guidance for the evaluation of postgraduate programs of Medicine III.

Method:

Analysis of the documents of 2013 area graduate programs strict sense concerning the application and measurement of score the articles published, and creation of proposal for metric of the theme with the quadrennial review of Medicine III.

Results:

Were evaluated the medicines area documents I, II and III; Biological Sciences (I) and Interdisciplinary, as well as the 2013 reports of CAPES. All programs establish metrics for "Classification of Published Articles" within its bibliographic production although with different percentages respecting its specificities. With these data collected and correlating their relevance with the surgical areas, was drafted proposal for quantification of the quality of the published articles to "Professional Postgraduate Programs" at surgical area that have specific characteristics according to their guidelines, directing their scientific production to technique journals preferably.

Conclusion:

The metric suggested for published articles, that should be included in intellectual production of the Area Document, should be considered for the extract A1 = 100 points; A2 = 85 points; B1 = 80 points; B2 = 70 points; B3 = 60 points; B4 = 40 points and B5 = 20 points.

Journal article; Education medical; Postgraduation in Medicine

Objetivo:

Propor métrica para qualificar a produção veiculada através de artigos publicados em programas de mestrado profissional e, a partir daí, estabelecer orientação para a avaliação dos programas de pós-graduação da Medicina III.

Método:

Análise dos documentos de área de 2013 dos programas de pós-graduação senso estrito no que concerne à aplicação e mensuração de pontuação a artigos publicados, e criação de proposta para métrica do tema com vistas à avaliação quadrienal da Medicina III.

Resultados:

Foram avaliados os documentos de área das Medicinas I, II e III; Ciências Biológicas I e Interdisciplinar, assim como os relatórios de 2013 da CAPES. Todos os programas estabelecem métrica para "Classificação de Artigos Publicados" dentro de sua produção bibliográfica embora com percentuais diferentes respeitando suas especificidades. Com esses dados levantados e correlacionando-se sua pertinência com as áreas cirúrgicas, foi elaborada proposta de quantificação da qualidade dos artigos publicados para "Programas de Pós-graduação Profissionais" na área cirúrgica que possuem características próprias de acordo com suas diretrízes, direcionando assim sua produção científica preferencialmente para revistas técnicas.

Conclusão:

A métrica sugerida para artigos publicados, que deve ser incluída na produção intelectual do Documento de Área, deverá ser considerada para o extrato A1 = 100 pontos; A2 = 85 pontos; B1 = 80 pontos; B2 = 70 pontos; B3 = 60 pontos; B4 = 40 pontos e B5 = 20 pontos.

Artigo de revista; Educação de pós-graduação em Medicina

INTRODUCTION

The stricto sensu postgraduate professional programs of CAPES have been increasing exponentially since its implantation¹.

With a new philosophical view, shifting the focus of postgraduate studies, from being only scientific to, also, technical and technology development producers, professional masters courses were designed with the purpose of changing the formation of professionals in the health area, creating not only the ones who produces research, but also creative and innovative professionals, that collaborates with the creation and or development of innovations in their daily work¹.

Such a view brought new paradigm, because the existing evaluation criteria is not suitable to programs with other purposes besides academic​​research.

Differently from academic master, the professional one tends to value much more inventive creation, patents generation and copyright deposit, which generally had negligible weight or even was not valued in academic master evaluation.

Thus, becomes urgent to re-establish new evaluation criteria for these programs due they have different disseminating production ways, following different publication lines between the two.

Regarding the publication of articles, while the academic master search publications in journals with ever-increasing impact, the professional also need to promote the production of their techniques and technology in journals that have, in general, lower impact factor than those purely scientific and used in the academic areas of health sciences. Thus, with the current score, the scientific production of articles in professional master remains relatively in lower level, negatively interfering in the final evaluation when the current criteria are used².

Hence, the need to review the criteria and metrics for the evaluation process of intellectual production in professional master postgraduate programs of Medicine III is urgent.

METHODS

This study was conducted at the Postgraduate Program in Professional and Research Experimental Surgery at the University of Pará by members of his collegiate in October-November 2014.

The research had as guidelines the active search of the evaluation criteria of bibliographical production in the form of complete articles published in journals, now used by different areas of the CAPES Health Sciences, verifying its compatibility with main purpose of the master professional and technical journals for which the bibliographic production is naturally directed.

The findings presented here are the result of the presentation made and the subsequent discussion during the V Meeting of Postgraduate Medicine III in December 8 and 9, 2014, in São Paulo, Brazil³.

Analysis of the material was based on the metrics contained in the Area Documents and 2013 reports of Interdisciplinar⁴, Biology Sciences⁵, Medicines I, II and III⁶ areas (Figure 1); reports from the 3^rd Monitoring Seminar realized in July 7 to 9, 2014, by CAPES Education Area; and accompanying seminar data from the postgraduate programs carried out in March 14 to 15, 2013, Brasília².

FIGURE 1
- Ranking of bibliographic production of academic master programs for the triennium analyzed in 2013

The bibliographic and technical production of the postgraduate programs recommended by CAPES were analyzed by comparing the mirrors formed by Academic Master's programs with the Professional Master's (Figure 2).

FIGURE 2
- Bibliographic and technique production comparison of academic and professional master's programs in triennium analyzed in 2013.

After Figure 3 mirrors identification, the designs were superimposed and applied on them using the current scoring criteria.

FIGURE 3
- Mirrors of bibliographic production of academic and professional master programs in the triennium analyzed in 2013.

RESULTS

The analyzes showed the results expressed in Figures 4, 5, 6 and 7 and Table 1.

FIGURE 4
- Percentage of articles published in the triennium analyzed in 2013 by academic and professional master's programs, distributed by Qualis stratum

FIGURE 5
- "Notes" (metric area x number of articles in the triennium analyzed in 2013), obtained by academic and professional master's programs, when measured by the metric of the Interdisciplinary and Biology Sciences area

FIGURE 6
- "Notes" (metric area x number of articles in the triennium analyzed in 2013), obtained by academic and professional master's programs, when evaluated by Medicines I, II and III metrics

FIGURE 7
- "Notes" (metric area x number of articles in the triennium analyzed in 2013), obtained by academic and professional master's programs, when evaluated by Suggested metric

TABLE 1
- "Note" (Metric area x number of articles in the triennium analyzed in 2013) of the academic and professional masters, when assessed by different metrics

	Notes by INTERDISCIPLINARY metrics.	Notes by MEDICINES metrics	Notes by SUGGESTED metrics
Academic	143060	121310	121310
Profissional	74580	58300	91860
Difference	68480	63010	29450

• Source: 3rd Accompanying Reports Seminar July 7/9, 2014, CAPES Education Area

DISCUSSION

Evaluation is a designed process to ensure the acquisition of competencies and skills in a particular area of ​​knowledge or labor field. Has always in view the process of continuous improving⁷.

Taking this concept as valid, CAPES revisional evaluation process of metrics for different types of production currently done is correct to adequate indicators focusing the basic justice principle.

With regard to published articles metrics, journals indicators express the quality of articles judged by two or three referees, or peer-reviewed journals, using criteria universally accepted. In most CAPES area documents the evaluation of journals follows a fairly uniform pattern with low variations, at least within ​​health area.

Thus, one might come to the conclusion that there would be no need to review the metrics, given its concordance among peers.

However, with the creation of the professional master's programs in 1999¹, another reality emerged in the Brazilian postgraduate scenario, since the purpose of them now aimed not only scientific, but also the technical and technological production.

In this new scenario, the application of current metric produces distortion and looses to professional programs (Figures 5 and 6) due to its technical characteristic they are compel to publish in journals of other areas of knowledge, such as bioengineering, biomaterials, biotechnology, nanotechnology which are now, of course, evaluated with a lower Qualis in health areas, clearly demonstrated in the 2013 assessment reports (Figures 2, 3 and 4)⁸.

With this new reality, in the V Meeting of Postgraduate Medicine III on 8 and 9 of December 2014 in São Paulo, Brazil, the issue was brought to discussion being presented, discussed and offered suggestions with new metric to be applied on published articles for professional masters, valuing technical and technological journals, not losing target on Qualis A and excellence ones.

Thus, when applying the metrics suggested in the last three years, there is a clear reduction of this distortion, keeping, however as a goal to be achieved, Qualis A journals. (Figure 7, Table 1).

With the adoption of this proposed metric, there will certainly be a better assessment of the professional master's programs, serving this metric as basic guideline of technical and technological production, correcting the existing distortions when evaluation is made with the metric for academic programs.

It is noteworthy that over the years, with a new evaluation by CAPES of Qualis for technical and technological journals in the health area, as well as the acceptance of technical articles in high Qualis journals, distortion tend to disappear over time with constant reassessment of used metrics.

CONCLUSION

The metric for articles published in the professional master to be included as intellectual production in area ​​document should consider as Qualis A1: 100 points; Qualis A2: 90 points; Qualis B1: 80 points; Qualis B2: 70 points; Qualis B3: 60 points; Qualis B4: 40 points; Qualis B5: 20 points and Qualis C: 0 points.

REFERÊNCIAS

• ¹
  Portal Brasil. Disponível em: http://www.brasil.gov.br/educacao/2014/05/capes-recomenda-574-opcoes-de-cursos-de-mestrado-profissional Acesso em 18/03/2015.
  » http://www.brasil.gov.br/educacao/2014/05/capes-recomenda-574-opcoes-de-cursos-de-mestrado-profissional
• ²
  Relatório do Seminário de acompanhamento dos Programas de Pós-Graduação em Ensino. De 14 a 15 de março de 2013. Área de Ensino da CAPES. Brasília DF.
• ³
  V Encontro da Pós-Graduação da Medicina III. De 8 a 9 de dezembro de 2014 Teatro Marcos Lindemberg, Vila Clementino na cidade de São Paulo, Brasil. 2014.
• ⁴
  Relatorio de Avaliação 2010-2012 - Trienal 2013. Área de avaliação: Interdisciplinar. Disponivel em: http://www.google.com.br/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&ved=0CCMQFjAB&url=http%3A%2F%2Ftrienal.capes.gov.br%2Fwp-content%2Fuploads%2F2011%2F01%2FINTERDISCIPLINAR-RELAT%25C3%2593RIO-DE-AVALIA%25C3%2587%25C3%2583O-FINAL.pdf&ei=SrMNVfGxOcmegwT7yID4DA&usg=AFQjCNHsSZ8-bkPEUWqUmYWRYq4emq0XMg
  » http://www.google.com.br/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&ved=0CCMQFjAB&url=http%3A%2F%2Ftrienal.capes.gov.br%2Fwp-content%2Fuploads%2F2011%2F01%2FINTERDISCIPLINAR-RELAT%25C3%2593RIO-DE-AVALIA%25C3%2587%25C3%2583O-FINAL.pdf&ei=SrMNVfGxOcmegwT7yID4DA&usg=AFQjCNHsSZ8-bkPEUWqUmYWRYq4emq0XMg
• ⁵
  Relatorio de Avaliação 2010-2012 - Trienal 2013. Área de avaliação: Ciências Biológicas I. Disponivel em: <https://drive.google.com/viewerng/viewer?a=v&pid=sites&srcid=Y2FwZXMuZ292LmJyfHRyaWVuYWwtMjAxM3xneDo1YWQzYjdhM2YxODFhZWMx> Acesso em: 20/11/2014.
  » https://drive.google.com/viewerng/viewer?a=v&pid=sites&srcid=Y2FwZXMuZ292LmJyfHRyaWVuYWwtMjAxM3xneDo1YWQzYjdhM2YxODFhZWMx
• ⁶
  Relatorio de Avaliação 2010-2012 - Trienal 2013. Área de avaliação: Medicina I. Disponivel em: <https://drive.google.com/viewerng/viewer?a=v&pid=sites&srcid=Y2FwZXMuZ292LmJyfHRyaWVuYWwtMjAxM3xneDoyNjNjZmQ3OWQ5ZjE5ZTEw> Acesso em: 20/11/2014.
  » https://drive.google.com/viewerng/viewer?a=v&pid=sites&srcid=Y2FwZXMuZ292LmJyfHRyaWVuYWwtMjAxM3xneDoyNjNjZmQ3OWQ5ZjE5ZTEw
• ⁷
  Dicionário informal. Disponível em: http://www.dicionarioinformal.com.br/avaliar/ Acesso em 14/03/2015.
  » http://www.dicionarioinformal.com.br/avaliar/
• ⁸
  Relatorio de Avaliação 2010-2012 - Trienal 2013. Área de avaliação: Biotecnologia. Disponivel em: <https://drive.google.com/viewerng/viewer?a=v&pid=sites&srcid=Y2FwZXMuZ292LmJyfHRyaWVuYWwtMjAxM3xneDo2NjQwNDI1YWExNTg2OWZi> Acesso em: 20/11/2014.
  » https://drive.google.com/viewerng/viewer?a=v&pid=sites&srcid=Y2FwZXMuZ292LmJyfHRyaWVuYWwtMjAxM3xneDo2NjQwNDI1YWExNTg2OWZi

• Source of funding: none

Publication Dates

• Publication in this collection
  2015

History

• Received
  19 Feb 2015
• Accepted
  12 Sept 2015