Acessibilidade / Reportar erro

DEVELOPMENT OF METRICS FOR TECHNICAL PRODUCTION: QUALIS BOOKS AND BOOK CHAPTERS

Abstracts

Objective:

To propose metrics to qualify the publication in books and chapters, and from there, establish guidance for the evaluation of the Medicine III programs.

Method:

Analysis of some of the 2013 area documents focusing this issue. Were analyzed the following areas: Computer Science; Biotechnology; Biological Sciences I; Public Health; Medicine I.

Results:

Except for the Medicine I, which has not adopted the metric for books and chapters, all other programs established metrics within the intellectual production, although with unequal percentages.

Conclusion:

It´s desirable to include metrics for books and book chapters in the intellectual production of post-graduate programs in Area Document with percentage-value of 5% in publications of Medicine III programs.

Books; Book classification; Education, medical, graduate


Objetivo:

Propor a métrica para qualificar a produção veiculada através de livros e capítulos e, a partir daí, estabelecer orientação para a avaliação dos programas de pós-graduação da Medicina III.

Método:

Análise dos documentos de área de 2013 dos programas de pós-graduação senso estrito das áreas: Ciência da Computação; Biotecnologia; Ciências Biológicas I; Saúde Coletiva; Medicina I.

Resultados:

Excetuando-se o programa da Medicina I, que não adotou a métrica para classificação de livros e capítulos, todos os demais estabeleceram-na dentro da sua produção intelectual, embora com percentuais desiguais.

Conclusão:

É desejável inserir a métrica de livros e capitulos de livros na produção intelectual do Documento de Área dos programas, ortorgando a ela percentual de 5% das publicações qualificadas dos programas da Medicina III.

Livros; Classificação de livro; Educação de pós-graduação em medicina


INTRODUCTION

Defined by CAPES, books are printed or electronic products with ISBN or ISSN (for serial works) containing at least 50 pages and published by public or private publishing house, scientific association, cultural/research institution or official organization11. CAPES. Roteiro para classificação de livros - avaliação dos Programas de Pós-Graduação. 24 de agosto 2009. Disponível em: < https://www.capes.gov.br/images/stories/download/avaliacao/Roteiro_livros_Trienio2007_2009.pdf>. Acesso em: 14/11/2014.
https://www.capes.gov.br/images/stories/...
.

Everyone knows that in various areas of knowledge books and their chapters are references to construct knowledge, setting styles and schools of thought. Thus, to evaluate the intellectual production in the format of books and chapters is a peculiar exercise, since there are no examples in other countries to do it. The evaluation involves singularities when compared to journals. In these, the production quality can be inferred a priori from circulation and impact indicators, recognized in consolidated bases and indexes. In the case of books and chapters, these principles are absent. The "Institute Scientific Information" - ISI - which was introduced by Eugene Garfield22. Eugene Garfild. Disponível em: < http://en.wikipedia.org/wiki/Eugene_Garfield>. Acessado em: 26/11/2014
http://en.wikipedia.org/wiki/Eugene_Garf...
in 1960, registers books and articles in its base; however, such records do not get proper and adequate scientometric treatment.

ISI do not register citations among books, as well as citations of articles in books; hence, assess the intellectual production program through books requires the development of specific criteria11. CAPES. Roteiro para classificação de livros - avaliação dos Programas de Pós-Graduação. 24 de agosto 2009. Disponível em: < https://www.capes.gov.br/images/stories/download/avaliacao/Roteiro_livros_Trienio2007_2009.pdf>. Acesso em: 14/11/2014.
https://www.capes.gov.br/images/stories/...
. So, it's a challenge in making uniformity procedures among areas to be more objective in evaluation process and more transparent to the scientific and academic community of the Brazilian postgraduate. The evaluation done by CAPES for the production of books and chapters requires ongoing process like "Qualis Periodicals", which spent more than a decade to reach the current stage of recognition.

The objective of this paper is to propose metrics to qualify the production on books and chapters, and thus to establish guidance for the evaluation of postgraduate programs of Medicine III on these topics.

METHOD

This study was conducted at the Postgraduate Program in Principles of Surgery at the Evangelical School of Paraná and University Evangelical Hospital of Curitiba by members of its Collegiate in November 2014.

This research followed the principles of the classification for books approved for the evaluation process of postgraduate programs at the CTC 2009 CAPES.

The structure also followed the document "The construction of scientific paper: a guide for projects, scientific research and reports"33. Alexandre MJO. A construção do trabalho científico: um guia para projetos, pesquisas e relatórios científicos. Rio de Janeiro: Forense Universitária, 2003. and "Scientific methodology: how to make more pleasant the development of academic work"44. Martins RB. Metodologia científica: como tornar mais agradável a elaboração de trabalhos acadêmicos. Curitiba: Juruá; 2004..

This publication is the result of presentation made during the Fifth Meeting of Postgraduate Medicine III between 8 and 9 December 2014 in São Paulo, Brazil. The material was based on the analysis of the 2013 area ​​documents of various postgraduate programs that aimed to establish the metric for books and book chapters. The programs analyzed were in the following areas: Computer Science55. Relatório de Avaliação 2010-2012 - Trienal 2013. Área de avaliação: Ciência da Computação. Disponivel em: < https://drive.google.com/viewerng/viewer?a=v&pid=sites&srcid=Y2FwZXMuZ292LmJyfHRyaWVuYWwtMjAxM3xneDoxMTdkMTgxYjVjN2M4OTk0> Acesso em: 20/11/2014.
https://drive.google.com/viewerng/viewer...
; Biotecnology66. Relatório de Avaliação 2010-2012 - Trienal 2013. Área de avaliação: Biotecnologia. Disponivel em: < https://drive.google.com/viewerng/viewer?a=v&pid=sites&srcid=Y2FwZXMuZ292LmJyfHRyaWVuYWwtMjAxM3xneDo2NjQwNDI1YWExNTg2OWZi> Acesso em: 20/11/2014.
https://drive.google.com/viewerng/viewer...
; Biological Sciences I77. Relatório de Avaliação 2010-2012 - Trienal 2013. Área de avaliação: Ciências Biológicas I. Disponivel em: < https://drive.google.com/viewerng/viewer?a=v&pid=sites&srcid=Y2FwZXMuZ292LmJyfHRyaWVuYWwtMjAxM3xneDo1YWQzYjdhM2YxODFhZWMx> Acesso em: 20/11/2014.
https://drive.google.com/viewerng/viewer...
; Public Health 88. Relatório de Avaliação 2010-2012 - Trienal 2013. Área de avaliação: Saúde Coletiva. Disponivel em: < https://drive.google.com/viewerng/viewer?a=v&pid=sites&srcid=Y2FwZXMuZ292LmJyfHRyaWVuYWwtMjAxM3xneDo3ZWRlMmUwNzBhOGJlMWNl> Acesso em: 20/11/2014.
https://drive.google.com/viewerng/viewer...
; and Medicine I99. Relatório de Avaliação 2010-2012 - Trienal 2013. Área de avaliação: Medicina I. Disponivel em: < https://drive.google.com/viewerng/viewer?a=v&pid=sites&srcid=Y2FwZXMuZ292LmJyfHRyaWVuYWwtMjAxM3xneDoyNjNjZmQ3OWQ5ZjE5ZTEw> Acesso em: 20/11/2014.
https://drive.google.com/viewerng/viewer...
.

As an evaluation tool for classification of books and chapters was observed the following items: 1) work identification data; 2) the formal aspects of the work, if there was the presence of teachers and students of the program, published in foreign languages, obtaining national and international awards and, also, the link to program line of research; 3) qualitative evaluation of the content, where it was observed the thematic relevance, innovation which highlights the originality of the content, with the innovative contribution to the field of surgery and also the impact of the work for its community.

For the books and chapters metrics the proposal prepared by these authors is shown in Figures 1, 2 and 3. By convention, the sum of chapters should not exceed the book's score, as well as the same author can score a maximum of two chapters in the same book.

FIGURE 1
- Book punctuartion criteria

FIGURE 2
- Book punctuation strata

FIGURE 3
- Book chapters punctuation strata

RESULTS

The survey showed the following results:

Area of evaluation: Computer Science

Books and chapters were evaluated in intellectual production (Item 4 of the 2013 Area Document) without using specific evaluation guideline, according to percentages shown in Figure 4.

FIGURE 4
- Item 4 of the Computer Science Area Document 2013

Area of evaluation: Biotechnology

Books and book chapters were evaluated in the intellectual production (Item 4 of the 2013 Area Document) as shown in Figure 5.

FIGURE 5
- Item 4 of the Computer Science Area Document 2013

Area of evaluation: Biological Sciences I

Books and book chapters were evaluated in the intellectual production (Item 4 of the 2013 Area Document) using percentage as shown in Figure 6.

FIGURE 6
- Item 4 of the Biological Sciences I Area Document 2013

Area Assessment: Public Health

Books and book chapters were evaluated in the intellectual production (Item 4 of the 2013 Area Document) using percentage of the program as shown in Figure 7.

FIGURE 7
- Item 4 of the Public Health Area Document 2013

Area of evaluation: Medicine I

The evaluation area of Medicine I did not adopt the metric for books and chapters because it seldom uses this type of publication.

Area of evaluation: Medicine III

The proposal prepared by the authors of this article is to consider the evaluation of books and book chapters in the intellectual production as shown in Figure 8.

FIGURE 8
- Proposition to include and evaluate books and book chapters in Item 4 of Area Document of Medicine III

DISCUSSION

The production assessment is distinct from the periodic to books and chapters, because it presents singularities compared to journals. The indicators that express quality are the ones using well-defined methods and applied impartially by peers. They have universal characteristics; already on the books and chapters these characteristics are not presented11. CAPES. Roteiro para classificação de livros - avaliação dos Programas de Pós-Graduação. 24 de agosto 2009. Disponível em: < https://www.capes.gov.br/images/stories/download/avaliacao/Roteiro_livros_Trienio2007_2009.pdf>. Acesso em: 14/11/2014.
https://www.capes.gov.br/images/stories/...
.

It is also known that unlike the postgraduate programs in the medical field (Medicines I, II and III) - where books and chapters are not significant and relevant production - in many others, such as Computer Science55. Relatório de Avaliação 2010-2012 - Trienal 2013. Área de avaliação: Ciência da Computação. Disponivel em: < https://drive.google.com/viewerng/viewer?a=v&pid=sites&srcid=Y2FwZXMuZ292LmJyfHRyaWVuYWwtMjAxM3xneDoxMTdkMTgxYjVjN2M4OTk0> Acesso em: 20/11/2014.
https://drive.google.com/viewerng/viewer...
, Biotecnology66. Relatório de Avaliação 2010-2012 - Trienal 2013. Área de avaliação: Biotecnologia. Disponivel em: < https://drive.google.com/viewerng/viewer?a=v&pid=sites&srcid=Y2FwZXMuZ292LmJyfHRyaWVuYWwtMjAxM3xneDo2NjQwNDI1YWExNTg2OWZi> Acesso em: 20/11/2014.
https://drive.google.com/viewerng/viewer...
, Biological Sciences I77. Relatório de Avaliação 2010-2012 - Trienal 2013. Área de avaliação: Ciências Biológicas I. Disponivel em: < https://drive.google.com/viewerng/viewer?a=v&pid=sites&srcid=Y2FwZXMuZ292LmJyfHRyaWVuYWwtMjAxM3xneDo1YWQzYjdhM2YxODFhZWMx> Acesso em: 20/11/2014.
https://drive.google.com/viewerng/viewer...
and Public Health88. Relatório de Avaliação 2010-2012 - Trienal 2013. Área de avaliação: Saúde Coletiva. Disponivel em: < https://drive.google.com/viewerng/viewer?a=v&pid=sites&srcid=Y2FwZXMuZ292LmJyfHRyaWVuYWwtMjAxM3xneDo3ZWRlMmUwNzBhOGJlMWNl> Acesso em: 20/11/2014.
https://drive.google.com/viewerng/viewer...
, these types of publication are very representative in scientific production. Inspired or not in periodic classification, these programs sought in the last evaluation qualify them within the bibliographic production of area document with the values ​​shown in the figures here reported.

While recognized within the medical field that these works do not constitute important bibliographic production, these authors believe that one should value these publications, if not with the same intensity as in referred programs55. Relatório de Avaliação 2010-2012 - Trienal 2013. Área de avaliação: Ciência da Computação. Disponivel em: < https://drive.google.com/viewerng/viewer?a=v&pid=sites&srcid=Y2FwZXMuZ292LmJyfHRyaWVuYWwtMjAxM3xneDoxMTdkMTgxYjVjN2M4OTk0> Acesso em: 20/11/2014.
https://drive.google.com/viewerng/viewer...
,66. Relatório de Avaliação 2010-2012 - Trienal 2013. Área de avaliação: Biotecnologia. Disponivel em: < https://drive.google.com/viewerng/viewer?a=v&pid=sites&srcid=Y2FwZXMuZ292LmJyfHRyaWVuYWwtMjAxM3xneDo2NjQwNDI1YWExNTg2OWZi> Acesso em: 20/11/2014.
https://drive.google.com/viewerng/viewer...
,77. Relatório de Avaliação 2010-2012 - Trienal 2013. Área de avaliação: Ciências Biológicas I. Disponivel em: < https://drive.google.com/viewerng/viewer?a=v&pid=sites&srcid=Y2FwZXMuZ292LmJyfHRyaWVuYWwtMjAxM3xneDo1YWQzYjdhM2YxODFhZWMx> Acesso em: 20/11/2014.
https://drive.google.com/viewerng/viewer...
,88. Relatório de Avaliação 2010-2012 - Trienal 2013. Área de avaliação: Saúde Coletiva. Disponivel em: < https://drive.google.com/viewerng/viewer?a=v&pid=sites&srcid=Y2FwZXMuZ292LmJyfHRyaWVuYWwtMjAxM3xneDo3ZWRlMmUwNzBhOGJlMWNl> Acesso em: 20/11/2014.
https://drive.google.com/viewerng/viewer...
,99. Relatório de Avaliação 2010-2012 - Trienal 2013. Área de avaliação: Medicina I. Disponivel em: < https://drive.google.com/viewerng/viewer?a=v&pid=sites&srcid=Y2FwZXMuZ292LmJyfHRyaWVuYWwtMjAxM3xneDoyNjNjZmQ3OWQ5ZjE5ZTEw> Acesso em: 20/11/2014.
https://drive.google.com/viewerng/viewer...
, but in a specific way to score them. To count on evaluation of Medicine III, they should have original content and good potential to provide significant contribution to the national surgical area.

Is also the understanding of these authors that the metric of books and their chapters is still in embryonic construction process, similar to what happened with the Qualis Periodicals more than a decade ago and only nowadays has achieved significant recognition, although still requiring ongoing changes. Motivated by the believe that many of these works contribute to the growth of Brazil's surgery, it is important to create a metric system for publications on books and chapters in the intellectual production (Item 4) contained in the document area, as do other CAPES areas 55. Relatório de Avaliação 2010-2012 - Trienal 2013. Área de avaliação: Ciência da Computação. Disponivel em: < https://drive.google.com/viewerng/viewer?a=v&pid=sites&srcid=Y2FwZXMuZ292LmJyfHRyaWVuYWwtMjAxM3xneDoxMTdkMTgxYjVjN2M4OTk0> Acesso em: 20/11/2014.
https://drive.google.com/viewerng/viewer...
,66. Relatório de Avaliação 2010-2012 - Trienal 2013. Área de avaliação: Biotecnologia. Disponivel em: < https://drive.google.com/viewerng/viewer?a=v&pid=sites&srcid=Y2FwZXMuZ292LmJyfHRyaWVuYWwtMjAxM3xneDo2NjQwNDI1YWExNTg2OWZi> Acesso em: 20/11/2014.
https://drive.google.com/viewerng/viewer...
,7 8.9.

It should be noted that item 4 includes, in addition to scientific production, technical and artistic production. Thus, is appropriate include it in the evaluation of these topics, but with lower percentage (5%) than the other sub-items. It is suggested to evaluate this production, differently to what was held by the Medicine I, which dismissed the last three years the publication of books like new knowledge dissemination vehicle99. Relatório de Avaliação 2010-2012 - Trienal 2013. Área de avaliação: Medicina I. Disponivel em: < https://drive.google.com/viewerng/viewer?a=v&pid=sites&srcid=Y2FwZXMuZ292LmJyfHRyaWVuYWwtMjAxM3xneDoyNjNjZmQ3OWQ5ZjE5ZTEw> Acesso em: 20/11/2014.
https://drive.google.com/viewerng/viewer...
.

The authors propose that the production of books and chapters may have better evaluation and representativeness in the Professional Master's programs, in which, of course, the placement of these works by faculty and students is more important.

Regardless of numerous discussions, establish the metrics of these works is a great and necessary challenge. It is also important to make the evaluation process more transparent and uniform among the areas.

CONCLUSION

It is proposed to insert metric for books and book chapters in the intellectual production of Medicine III Area Document giving to it the percentage of 5% among the eligible publications in the area.

REFERENCES

Publication Dates

  • Publication in this collection
    2015

History

  • Received
    19 Feb 2015
  • Accepted
    12 Sept 2015
Colégio Brasileiro de Cirurgiões Rua Visconde de Silva, 52 - 3º andar, 22271- 090 Rio de Janeiro - RJ, Tel.: +55 21 2138-0659, Fax: (55 21) 2286-2595 - Rio de Janeiro - RJ - Brazil
E-mail: revista@cbc.org.br