Brazilian knowledge production in the field of child and adolescent health

Abstracts

OBJECTIVES: To assess (a) the trend of MEDLINE citation of pediatrics articles associated with Brazilian institutions from 1990 through 2004; (b) the number of Brazilian pediatrics articles published in journals with the highest impact factor; and (c) the regional distribution of institutions. METHODS: PubMed search limited to ages 0 to 18 years, English language, MEDLINE and humans subsets, Brazilian affiliation. For each year, we compared the articles retrieved to the whole of MEDLINE citations with the same search limits, except for affiliation, as well as to the total Brazilian scientific production cited in MEDLINE, without age limits. We made a descriptive analysis, and used the chi-square test for trend. Data concerning publication in journals with the highest impact factor were aggregated into three-year periods. RESULTS: A total of 7,222 Brazilian pediatrics articles were listed in MEDLINE from 1990 through 2004, corresponding to 0.95% of all articles concerning the age group from 0 to 18 years. There was a fivefold increase in the absolute number of Brazilian articles along the study period. The ratio of Brazilian to total articles increased from 0.51 to 1.60% (p < 0.01). Scientific knowledge production remains strongly concentrated in the Southeast of Brazil. CONCLUSION: Brazilian research activities in pediatrics have had a steady upward trend, which relates to the proportional growth of the Brazilian scientific production as a whole.

Knowledge production; pediatric research; bibliometrics; impact fator

OBJETIVOS: Verificar (a) a tendência das citações de artigos de pediatria oriundos de instituições brasileiras, no MEDLINE, de 1990 até 2004; (b) o número de artigos publicados em periódicos com os maiores fatores de impacto; e (c) a distribuição regional das instituições. MÉTODOS: Busca no PubMed com os seguintes limites: idade de 0 a 18 anos, língua inglesa, subconjuntos MEDLINE e humanos, afiliação brasileira. Para cada ano, comparamos os artigos encontrados com o total de citações no MEDLINE, obedecendo aos mesmos limites de busca, com exceção da afiliação; o mesmo foi feito com o total da produção científica brasileira citada no MEDLINE, sem limites de idade. Realizamos uma análise descritiva, usando o teste qui-quadrado para a tendência. Agregamos os dados relativos à publicação em periódicos com os maiores fatores de impacto em períodos de 3 anos. RESULTADOS: Entre 1990 e 2004, o MEDLINE citou 7.222 artigos de pediatria brasileiros, correspondendo a 0,95% de todos os artigos tratando da faixa etária de 0 a 18. O número absoluto de artigos brasileiros aumentou cinco vezes durante esse período. A fração de artigos brasileiros em relação ao número total de artigos aumentou de 0,51 a 1,60% (p < 0,01). A produção científica permanece fortemente concentrada na Região Sudeste do Brasil. CONCLUSÃO: As atividades brasileiras de pesquisa em pediatria demonstram uma tendência ascendente constante, proporcional ao crescimento de toda a produção científica brasileira.

Produção científica; pesquisa em pediatria; bibliometria; fator de impacto

ORIGINAL ARTICLE

Brazilian knowledge production in the field of child and adolescent health

Danilo Blank^I; Luciana O. Rosa^II; Ricardo Q. Gurgel^III; Marcelo Z. Goldani^IV

^IProfessor, Departamento de Pediatria, Faculdade de Medicina, Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, RS, Brasil

^IIResearch fellow, Faculdade de Medicina, Pontifícia Universidade Católica do Rio Grande do Sul (PUCRS), Porto Alegre, RS, Brasil

^IIIProfessor, Núcleo de Pós-Graduação em Medicina, Universidade Federal de Sergipe (UFS), São Cristóvão, SE, Brasil

^IVProfessor, Departamento de Pediatria, Faculdade de Medicina, UFRGS, Porto Alegre, RS, Brasil

^{Correspondence}

ABSTRACT

OBJECTIVE: To assess (a) the trend of MEDLINE citation of pediatrics articles associated with Brazilian institutions from 1990 through 2004; (b) the number of Brazilian pediatrics articles published in journals with the highest impact factor; and (c) the regional distribution of institutions.

METHODS: PubMed search limited to ages 0 to 18 years, English language, MEDLINE and humans subsets, Brazilian affiliation. For each year, we compared the articles retrieved to the whole of MEDLINE citations with the same search limits, except for affiliation, as well as to the total Brazilian scientific production cited in MEDLINE, without age limits. We made a descriptive analysis, and used the chi-square test for trend. Data concerning publication in journals with the highest impact factor were aggregated into three-year periods.

RESULTS: A total of 7,222 Brazilian pediatrics articles were listed in MEDLINE from 1990 through 2004, corresponding to 0.95% of all articles concerning the age group from 0 to 18 years. There was a fivefold increase in the absolute number of Brazilian articles along the study period. The ratio of Brazilian to total articles increased from 0.51 to 1.60% (p < 0.01). Scientific knowledge production remains strongly concentrated in the Southeast of Brazil.

CONCLUSION: Brazilian research activities in pediatrics have had a steady upward trend, which relates to the proportional growth of the Brazilian scientific production as a whole.

Key words: Knowledge production, pediatric research, bibliometrics, impact fator.

Introduction

The increase in knowledge production - and hence in the number of periodicals - during the second half of the past century led professionals and researchers to face the challenge of developing quality criteria that could assist readers in selecting the best scientific evidence.^1,2 Nevertheless, the lack of a consensus framework to measure scientific and technical outcomes under a fair conceptual scheme led to a disparate array of indicators. Thus, in practice, the so-called scientometrics is often done solely by using bibliometrics, that is, measurement of scientific publications, even though beyond it there are no systematic linkages with other categories of metrics of science and technology.³ Bibliometrics is based upon the concept that the quality of periodicals and articles can be inferred by the hierarchization of scientific production, constructed through quantitative techniques that yield certain indexes.⁴

The impact factor, introduced in the early 1960s by Eugene Garfield and Irving Sherby, is the most prominent among the many proposed journal citation measures.^5,6 A journal's impact factor is calculated by dividing the number of citations in the current year to any items published in the journal in the previous two years by the number of substantive articles (source items) published in the journal in the same two years.⁷ Thus, the impact factor, which is considered an indicator of visibility and diffusion of scientific knowledge, is basically a ratio between cited and citable items. Its strength lies above all in the comprehensibility, stability and seeming reproducibility; on the other hand, some obvious flaws, along with its uninformed use, have resulted in a flood of critical and controversial discussions about its correctness.^6,8-17 Thomson Scientific, formerly known as the Institute for Scientific Information (ISI) calculates and publishes in the Journal Citation Reports the impact factor of 7,500 of the world's most cited peer-reviewed journals in approximately 200 disciplines. Among those, there are 17 Brazilian journals, seven of which from the health field.⁹

With the development of practical applications for bibliometric indexes, they became the technical foundation for systematic evaluation and strategic decision-making within the political arena, and accordingly also in the entire social domain. In Brazil, bibliometrics is widely used to determine the allocation of subsidies and technical resources, by means of institutional schemes of bibliographic hierarchization. For instance, Qualis, the System of Classification of Periodicals, Annals and Journals of the Federal Coordinating Agency for the Improvement of Higher Education (CAPES), employs the Journal Citation Reports as a reference to classify periodicals of international circulation.¹⁸ CAPES regulates and orients the national scientific production through the continual evaluation of research institutes and graduate programs, aiming at broadening the Brazilian participation in the international scene.

The degree of international insertion of the Brazilian scientific production can be gauged through the analysis of bibliographic citation databases, such as Biological Abstracts, FSTA-Food Science, Web of Science and OVID. Two databases stand out in Latin America: the Scientific Library Online (SciELO) and the Latin American and Caribbean Center on Health Sciences Information (LILACS). The world's largest database in the health field is the U.S. National Library of Medicine's Medical Literature Analysis and Retrieval System Online (MEDLINE), that contains about 13 million references to articles from 4,800 journals worldwide.¹⁹

Although bibliometrics is thoroughly incorporated into the day-to-day work of researchers as a tool to monitor institutional and personal performance, there are only a few studies focusing on the trend and degree of international insertion of Brazil's scientific production according to specific knowledge areas. Thus, we decided to carry out the present study with the following objectives: (a) to evaluate the international insertion of the Brazilian scientific production in the field of child and adolescent health, as well as its time trend, by assessing MEDLINE citations and publication of articles in journals with the highest impact factor; (b) to evaluate the distribution of such scientific knowledge production, according to geographic region and type of research center.

Methods

We performed a PubMed search in order to retrieve all MEDLINE citations of articles of Brazilian affiliation concerning child and adolescent health published in English from 1990 through 2004. To do so, we applied the following limit parameters: affiliation [Brazil OR Brasil]; age [All Child: 0-18 years]; language [English]; subjects [Humans]; database subset [MEDLINE]; publication date [1990-2004]. The affiliation parameter means that the search included articles whose first author's listed address (including e-mail address) or institution was in Brazil, irrespective of the journal's country of publication. With such limits, PubMed retrieval is limited to MEDLINE citations, also excluding "in process" and "supplied by publisher" citations, because those have not yet completed the indexing process and do not carry the required data. We calculated the yearly rate of participation of Brazilian affiliation articles in relation to the database of all affiliations.

We ascertained the location that defined the article's affiliation, either the author's or the institution's address, and particularly the institution's nature (research center, university, attending hospital).

We also looked into the number of Brazilian articles that had been published in journals of higher impact factor. Considering that about 66% of citations within the field of child and adolescent health are derived from four North-American (Pediatrics, Journal of Pediatrics, JAMA, and The New England Journal of Medicine) and three British journals (Archives of Disease in Childhood, Lancet, and British Medical Journal),20 we performed a selective search of Brazilian articles in those seven periodicals, and aggregated the data into three year periods.

We made a descriptive analysis of the data and used the chi-square test for trend analysis of proportions.

Results

A total of 7,222 Brazilian pediatrics articles, published in English, were cited in MEDLINE along the past 15 years, just short of 1% of the entire database of papers concerning this field of study. However, while the growth of the latter was about 61%, the absolute number of Brazilian articles had an overall 404% increase. There was a significant increment of 213% (p < 0.01) in the fraction of Brazilian pediatrics articles in MEDLINE, while the fraction of all Brazilian articles published in English, irrespective of age limits, increased by 264% (p < 0.01) in the same period (Table 1).

The number of pediatrics articles from Brazil published in the journals of highest impact factor remained unchangingly scarce along the study period. Figure 1 depicts the publication of Brazilian articles on child and adolescent health in all the periodicals analyzed, except JAMA and The New England Journal of Medicine, which did not publish any Brazilian pediatrics article from 1990 through 2004. Considering the significant growth in the overall number of Brazilian articles (both on pediatrics and all subjects) that were indexed in MEDLINE, there was a proportional reduction in the number of papers published in those high impact periodicals.

As to the state of origin of the studies, there was a great concentration of scientific knowledge production in São Paulo throughout the study period. On the other hand, there was a considerable increase in the participation of the state of Rio Grande do Sul, and a slight increase for Minas Gerais and Paraná, while the share of Rio de Janeiro diminished (Figure 2).

During this period, there was a reduction in the contribution of pediatric centers to the total number of articles in child and adolescent health, whereas centers concerned with other areas, such as public health, internal medicine, psychiatry, physiology, and molecular biology, expanded their participation (Figure 3).

Discussion

This study showed a significant increase in the international visibility of Brazilian articles focused on child and adolescent health, albeit they were not satisfactorily introduced into the realm of journals with the highest impact factors. The results also show that research in the field of child and adolescent health has been following the same ascending trend as the overall production of medical knowledge in Brazil.

Another study indicates a 44% growth in the participation of Brazilian papers among the international scientific publications, and a 39% increase in general citations from 1997 to 2001. However, the number of citations to Brazilian authors' articles decreased from 7.5 citations per paper to 3.5 citations per paper in the same period.²¹ At least four hypotheses can be brought up to explain such phenomenon: first, Brazilian articles have restricted access to periodicals with high impact factors, as shown in this study, and so they are less likely to be cited. Second, whereas many Brazilian authors succeed in getting published in high impact journals, their work is infrequently cited. It has been shown that only 15% of the articles in any given journal are responsible for its impact factor,17 and that an author's institutional affiliation is among the factors that lead to citation of his work. This means that belonging to outside institutions reduces one's chances of being cited. Third, in spite of the fact that one is most probably cited by peers, the low impact factor of national journals along with the little value attributed to them in terms of the evaluation criteria used by funding agencies drive authors to submit their best papers to foreign periodicals. This results in a vicious circle: the more an author takes his research reports away from his equals by publishing abroad, even with less chances of being cited elsewhere, the more the national journals are devalued, and thus tend to attract fewer quality papers, which brings even fewer citations. Fourth, we can hypothesize that the path to a wider recognition of the Brazilian scientific offspring is in the midst of a transition stage, which can be characterized by an initial growth of the publication of articles in international journals, which should be followed by greater visibility and later on by more citations.

The results of this study confirm the great concentration of scientific knowledge production in the state of São Paulo, as previously shown.22 The growth in papers from other states, such as Rio Grande do Sul and Minas Gerais, correlates to the assessment of graduate programs carried out by the Coordination for the Improvement of Higher Education from the Ministry of Education, which indicates the increasingly higher excellence of the academic research institutions in those regions.¹⁸

On the other hand, the significant participation of non-pediatric specialties such as basic sciences, public health, epidemiology, and internal medicine demonstrates that child and adolescent health as a research subject is not a monopoly of pediatric research centers. This finding may also be reflecting the historical origin and insertion of pediatric researchers in non-pediatric graduate programs. Or it may be the result of an institutional strategy adopted by many research centers, which tend to concentrate their production in hegemonic graduate programs, generally focused on basic science or adult health. This vast diversity of approaches leads us to rethink the composition of this study field, particularly concerning graduate programs with a restricted clinical pediatrics scope, so as to promote a future reorganization that favors a multidisciplinary character.

Some methodological limitations of this study must be pointed out. The first and most obvious one is that a PubMed search restricts the retrieval to MEDLINE citations of articles. Since only 32 of the 283 Brazilian journals indexed in the LILACS database are part of 4,250 journals indexed in MEDLINE , and only 12 are index in ISI it is evident that many quality studies have been missed.²³ We did not attempt to manually retrieve the non-indexed journals because it would be out of the scope of this study, but one should bear in mind that the few Brazilian journals that are indexed in MEDLINE tend to undergo a great increase in submissions. That induces a rise in rejection rates and sends a growing number of good papers to the less accessible domain of gray publications.²⁴

Second, MEDLINE data are constantly being revised and updated; thus, the figures we retrieved may still change in the next three years. However, we have been monitoring this process in the past year using the same descriptors and limits employed in this study, and the modest variation we observed did not affect the overall trends we identified.

Third, the decision to include other productive fields in addition to those dedicated to clinical research may have led to an overestimation of the true position of Brazilian pediatricians' knowledge production. However, we kept uniform criteria in analyzing the time trend, exactly to show the overall international insertion of Brazilian pediatrics research.

In conclusion, the increase in the number of articles on child and adolescent health and their broader insertion into the international scene have significantly contributed toward fostering the upward trend of Brazil's scientific knowledge production. Nevertheless, in spite of the positive results, very few Brazilian articles - in plain disproportion to the significant growth in the number of published articles - make into high impact journals. This poses a great challenge to Brazilian researchers to keep up the pace of increasing international visibility, chiefly through the quest for quality, in order to beat the odds against discrimination.

References

23. BIREME - Centro Latino-Americano e do Caribe de Informação em Ciências da Saúde. http://www.bireme.br. Acesso: 31/10/2005.

Manuscript received Nov 03 2005, accepted for publication Jan 22 2006.

• 1. Garfield E. The impact factor: Institute for Scientific Information. http://www.isinet.com Acesso: 14/03/2005.
• 2. Aksnes DW. Citations and their use as indicators in science policy. Studies of validity and applicability issues with a particular focus on highly cited papers [ thesis] . Enschede-West, Nehterlands: University of Twente; 2005.
• 3. Geisler E. The measurement of scientific activity: research directions in linking philosophy of science and metrics of science and technology outputs. Scientometrics. 2005;62:269-84.
• 4. Bordons M, Zulueta A. Evaluación de la actividad científica a través de indicadores bibliométricos. Rev Esp Cardiol. 1999;52:790-800.
• 5. Garfield E. Journal impact factor: a brief review. CMAJ. 1999;161:979-800.
• 6. Glanzel W, Moed HF. Journal impact measures in bibliometric research. Scientometrics. 2002;53:171-93
• 7. Garfield E. The agony and the ecstasy - The history and meaning of the journal impact factor. Conference at the 5th International Congress on Peer Review and Biomedical Publication; 2005 Sep 16-18; Chicago, USA. Chicago: JAMA/BMJ; 2005. http://garfield.library.upenn.edu/papers/jifchicago2005.pdf Acesso: 08/10/2005.
• 8. Coelho PM, Antunes CM, Costa HM, Kroon EG, Sousa Lima MC, Linardi PM. The use and misuse of the "impact factor" as a parameter for evaluation of scientific publication quality: a proposal to rationalize its application. Braz J Med Biol Res. 2003;36:1605-12.
• 9. Coura JR, Willcox LC. Impact factor, scientific production and quality of Brazilian medical journals. Mem Inst Oswaldo Cruz. 2003;98:293-7.
• 10. Leff D. Making an impact: the rise of the impact factor as a measure of journal quality. J Am Diet Assoc. 2005;105:29-30.
• 11. Lundberg G. The "omnipotent" Science Citation Index impact factor. Med J Aust. 2003;178:253-4.
• 12. Nakayama T, Fukui T, Fukuhara S, Tsutani K, Yamazaki S. Comparison between impact factors and citations in evidence-based practice guidelines. JAMA. 2003;290:755-6.
• 13. Porta M, Copete JL, Fernandez E, Alguacil J, Murillo J. Mixing journal, article, and author citations, and other pitfalls in the bibliographic impact factor. Cad Saude Publica. 2003;19:1847-62.
• 14. Saha S, Saint S, Christakis DA. Impact factor: a valid measure of journal quality? J Med Libr Assoc. 2003;91:42-6.
• 15. Walter G, Bloch S, Hunt G, Fisher K. Counting on citations: a flawed way to measure quality. Med J Aust. 2003;178:280-1.
• 16. Sevinc A. Manipulating impact factor: an unethical issue or an Editor's choice? Swiss Med Wkly. 2004;134:410.
• 17. Weale AR, Bailey M, Lear PA. The level of non-citation of articles within a journal as a measure of quality: a comparison to the impact factor. BMC Med Res Methodol. 2004;4:1-8.
• ¹⁸
  CAPES. Critérios Qualis e de conceitos relacionados à publicação. http://www.capes.gov.br/capes/portal/conteudo/10/QuemQuem.htm Acesso: 10/10/2005.
• 19. National Library of Medicine. MEDLINE Factsheet. http://www.nlm.nih.gov/pubs/factsheets/medline.html Acesso: 10/10/2005.
• 20. Birken CS, Parkin PC. In which journals will pediatricians find the best evidence for clinical practice? Pediatrics. 1999;103(5 Pt 1):941-7.
• 21. King D. The scientific impact of nations. Nature. 2004;430:311-6.
• 22. Rummler G, Spinola. A identificaçăo e procedęncia de periódicos nacionais referentes ŕ temática de saúde pública ou saúde coletiva, editados entre 1998 e 2003. Cad Saude Publica. 2004;20:1041-9.
• 24. Blank D, Buchweitz C, Procianoy R. Impact of SciELO and MEDLINE indexing on the submission of articles to a "non-English" journal. J Pediatr (Rio J). 2005;81:431-4.

Publication Dates

• Publication in this collection
  12 June 2006
• Date of issue
  Apr 2006

History

• Received
  03 Nov 2005
• Accepted
  22 Jan 2006