SciELO - Scientific Electronic Library Online

vol.51Night work and BMI: is it related to on-shift napping?Aedes aegypti breeding site in an underground rainwater reservoir: a warning índice de autoresíndice de materiabúsqueda de artículos
Home Pagelista alfabética de revistas  

Servicios Personalizados




Links relacionados


Revista de Saúde Pública

versión On-line ISSN 1518-8787

Rev. Saúde Pública vol.51  São Paulo  2017  Epub 04-Dic-2017 

Brief Communication

Uncited articles in Brazilian public health journals

Angela Maria Belloni CuencaI 

Milena Maria de Araújo Lima BarbosaII 

Karoline de OliveiraIII 

Fernanda Paranhos QuintaIII 

Maria do Carmo Avamilano AlvarezIV 

Ivan França JuniorI 

IUniversidade de São Paulo. Faculdade de Saúde Pública. Departamento de Saúde, Ciclos de Vida e Sociedade. São Paulo, SP, Brasil

IIUniversidade de São Paulo. Faculdade de Saúde Pública. Programa de Pós-Graduação em Saúde Pública. São Paulo, SP, Brasil

IIIUniversidade de São Paulo. Pró-Reitoria de Pesquisa. Bolsista do Programa Institucional de Bolsas de Iniciação Científica. São Paulo, SP, Brasil

IVUniversidade de São Paulo. Faculdade de Saúde Pública. Biblioteca: Centro de Informação e Referência. São Paulo, SP, Brasil


Here, we describe the percentage of non-citation in Brazilian public health journals, a field that, until now, had not been investigated nationally or internationally. We analyzed articles, published between 2008 and 2012, of eight public health journals indexed in the scopus database. The percentage of non-citation differs between journals (from 5.7% to 58.1%). We identified four statistically distinct groups: História, Ciência, Saúde – Manguinhos (58% uncited articles); Physis: Revista de Saúde Coletiva, Interface, and Saúde e Sociedade (32% to 37%); Ciência & Saúde Coletiva and Revista Brasileira de Epidemiologia (16% to 17%); and Cadernos de Saúde Pública and Revista de Saúde Pública (6%). The non-citation in the first three years post-publication also varies according to journal. Four journals have shown a clear decline of non-citation: Cadernos de Saúde Pública, Ciência & Saúde Coletiva, Revista Brasileira de Epidemiologia, and Physis. Another three (Revista de Saúde Pública, Saúde e Sociedade, and Interface) presented an oscillation in non-citation, but the rates of 2008 and 2012 are similar, with different magnitudes. In turn, the journal História, Ciência, Saúde – Manguinhos maintains high rates of non-citation. Multidisciplinary journals attract more citation, but a comprehensive citation model still needs to be formulated and tested.

Key words: Journal Article; Scientific and Technical Publications; Citation Databases; Systems for Evaluation of Publications


Aqui, descrevemos o percentual de não citação em revistas brasileiras de saúde pública, campo até agora não investigado nacional ou internacionalmente. Analisamos artigos, publicados entre 2008 e 2012, de oito revistas de saúde pública indexadas na base Scopus. O percentual de não citação difere entre as revistas (de 5,7% a 58,1%). Identificamos quatro grupos estatisticamente distintos: História, Ciência, Saúde – Manguinhos (58% de artigos não citados); Physis: Revista de Saúde Coletiva, Interface, e Saúde e Sociedade (32% a 37%); Ciência & Saúde Coletiva e Revista Brasileira de Epidemiologia (16% a 17%); e Cadernos de Saúde Pública e Revista de Saúde Pública (6%). A não citação nos primeiros três anos pós-publicação também varia segundo revista. Quatro revistas mostraram claro declínio da não citação: Cadernos de Saúde Pública, Ciência & Saúde Coletiva, Revista Brasileira de Epidemiologia e Physis. Outras três (Revista de Saúde Pública, Saúde e Sociedade e Interface) apresentaram oscilação na não citação, mas as taxas de 2008 e 2012 são similares, com magnitudes diferentes. Por sua vez, a revista História, Ciência, Saúde – Manguinhos mantém taxas elevadas de não citação. Revistas multidisciplinares atraem mais citação, mas um modelo compreensivo de citações ainda precisa ser formulado e testado.

Palavras-Chave: Artigo de Revista; Publicações Científicas e Técnicas; Bases de Dados de Citações; Sistemas de Avaliação das Publicações


From the beginning of the citation analysis, Garfield1 already showed concern about articles that were not cited. Hamilton2, in a citation analysis in the Web of Science, found 55% of uncited articles over the first five years post-publication. This proportion, however, varied according to the area of knowledge: 47.4% in the exact and biological sciences, 74.7% in the social sciences, and 98.0% in the arts and humanities3. A reanalysis of Hamilton’s data, restricted to original articles, estimated lower rates of non-citation: 22.4% (exact and biological sciences); 48% (social sciences); and 93% (arts and humanities)4. Although the non-citation rates estimated by Hamilton have been criticized2-6, the idea remains that, in fact, most of the articles were not cited or will not be cited7.

However, there are clear indications that non-citation has been continuously declining since the 1980s in the fields of natural sciences, engineering, medicine, and social sciences, but not in the humanities7.

The field of public health, multidisciplinary in nature, may be a relevant case for the evaluation of non-citation. However, it has not yet been investigated. Here, we describe the phenomenon of non-citation in Brazilian public health journals.


The selected journals had to obey two criteria: a) to be part of the Collective Health Editors Forum of the Brazilian Association of Collective Health (created in 2014, access at, which represents the Brazilian public health journals; and b) be indexed in the Scopus database, chosen by the multidisciplinary scope (more than 20 thousand journals indexed), relevance in the scientometric evaluation, and availability of the metadata for analysis. Thus, we selected eight journals (abbreviations by the National Library of Medicine Catalog - NLM): Cadernos de Saúde Pública (Cad Saude Publica); Ciência & Saúde Coletiva (Cien Saude Colet); História, Ciência, Saúde – Manguinhos (Hist Cienc Saude); Interface: Comunicação, Saúde, Educação (Interface); Physis: Revista de Saúde Coletiva (Physis); Revista Brasileira de Epidemiologia (Rev Bras Epidemiol); Revista de Saúde Pública (Rev Saude Publica); and Saúde e Sociedade (Saude Soc).

From these, data were extracted (in November 2015) referring to two types of articles: the original articles and the reviews, published between 2008 and 2012. We chose 2008 as the initial date because all selected journals were indexed in Scopus. The end date (2015) ensured a minimum of three years for the most recent (published in 2012) to receive citations.

We retrieved 5,736 article records, of which 66 were deleted for duplicity, with a final sample of 5,670 articles. From each, the following information was obtained: article title, authorship, publication year, pagination, journal title, and number of citations of each article (per year and total).

To validate the consistency of the database, a subsample with an arbitrary value of 10% of the articles (567/5.670) was analyzed. We found some inconsistencies. The most frequent was the delay in indexing, i.e., new records of citations of articles appeared between the period of the sample extraction and that of the evaluation of the subsample. Thus, 60 articles presented late citations, equivalent to 10.6% of the 567. Of these 60 articles, four (0.7%) would no longer be classified as uncited. Other, less frequent, problems were: year of citation divergent from the date indicated in Scopus (0.5%) and a citation counted twice by Scopus (0.5%). The non-citation rate of the eight journals remained: 18.6% in the sample and 18.5% in the subsample. Given the small magnitude of problems and their insignificant effect on the estimates, the data studied were those extracted from the Scopus base, without corrections.

We calculated, for each article and in each journal, the proportion of non-citation in the whole period and the proportion of non-citation in the first three years after publication.


We observed a major difference in non-citation rates between the journals, ranging from 5.7% to 58.1%, a difference of 10 times between the lowest and the highest rates (Figure 1). Also noteworthy is the distribution in four statistically distinct groups: in the group with the highest non-citation, is the journal Hist Cienc Saude. Another group, comprised of social science health journals, included Physis, Interface, and Saude Soc. In an intermediate situation, we have Cienc Saude Colet and Rev Bras Epidemiol. The lowest non-citation indexes were identified in the Cad Saude Publica and Rev Saude Publica journals.

a p > 0.05

b p < 0.001

Figure 1 The proportion of uncited articles in Brazilian public health journals, published from 2008 to 2012. Scopus Database, 2015. 

The non-citation behavior in the first years after publication also varies according to the journal (Figure 2). It is interesting to note that the non-citation dynamics in the last three years differ from the groups observed in Figure 1. There are four journals with a clear decline in non-citation: Cad Saude Publica, Cienc Saude Colet, Rev Bras Epidemiol, and Physis. In other three (Rev Saude Publica, Saude Soc, and Interface), there is an oscillation in non-citation, but rates in 2012 are similar to those of 2008, albeit with different magnitudes. Finally, the journal Hist Cienc Saude maintains high rates of non-citation.

Figure 2 Evolution of non-citation rates in the three years following the publication of articles in Brazilian public health journals. Scopus Database, 2015. 


Brazilian journals have marked differences between them in terms of non-citation but behave similarly to what has already been described in the literature regarding the peculiarities of each area. In fact, Pendlebury4, in an analysis of ISI-indexed journals between 1984 and 1988, already found these disparities between areas of knowledge, in which natural sciences, engineering, and medicine had lower rates, followed by the social sciences, and with the greatest non-citation in the humanities. Larivière et al.7 found similar groupings decades later.

The journal of humanities in the field of public health, which prioritizes the publication of articles related to documentation, research, and museum study in the history of science and health, has the highest overall non-citation rate but the rate has been decreasing in the first three years after publication. This pattern is comparable to the humanities journals analyzed in the Web of Science database7; that is, in the Brazilian case, 79% of articles not cited in Hist Cienc Saude versus 90% in the humanities journals mentioned by Larivière et al.7 Likewise, our health social science journals have similar behavior to those in the same field of knowledge indexed in the Web of Science, both in rates and in the reduction trend of non-citation.

On the other hand, it is noteworthy that our multidisciplinary public health journals (Rev Saude Publica and Cad Saude Publica) have comparatively lower levels of non-citation than those found by Larivière et al.7 These authors found a rate of uncited articles for medicine in general, and not their specialties, of around 11%. These differences can hardly be attributed to the different bibliometric sources, Scopus in our study and Web of Science in Larivière et al.7, because there is a high correlation between the impact indicators of these two bases8. The reasons why Rev Saude Publica and Cad Saude Publica are able to attract more citations still need to be studied, but they seem to follow the relationship in which the more interdisciplinary, the greater the ability to attract citations9.

Similarly, the journals Cienc Saude Colet and Rev Bras Epidemiol, both recent in the field of publishing in the area (created in 1996 and 1998, respectively), have similar characteristics regarding the overall rate of non-citation and the reduction of articles without citation in first three years. Cienc Saude Colet approaches multidisciplinary public health journals, with fewer articles not cited, probably due to its equally multidisciplinary nature. This journal undeniably has a strong tradition of disseminating production in the social sciences in health. Rev Bras Epidemiol, which publishes in an area that has the article as one of its main means of dissemination, has been attracting more and more citations. The monitoring of non-citations of these two journals, henceforth, may elucidate if they will be in the group of multidisciplinary journals.

Our analysis privileged non-citation, as we intend to bring up the proportion of articles that seem to have had no impact or recognition in the scientific community. Of course, this may be a limitation, since the analyses focused on the metrics of the citations (indicators such as index H, SJR, FI, cites per documents, among others) may reach different bibliometric classifications.

Pendlebury4 evaluates that a “certain level of non-citation in a journal is probably more an expression of the process of creation and dissemination of knowledge than any kind of performance measure,” recognizing that books are more important in the delivery of scientific communication among social and humanities scientists than in the exact and biological sciences. Thus, non-citation rates would represent more the forms of citing than the acceptance or rejection of the knowledge produced. The reasons for these differences have been speculated, but there are no comprehensive models to explain them. This gap opens room for further investigation.


1. Garfield E. Uncitedness III – the importance of not being cited. Essays Inf Sci. 1973;1:413-4. Available from: ]

2. Hamilton DP. Publishing by– and for?– the numbers. Science. 1990;250(4986):1331-2. ]

3. Hamilton DP. Research papers: who’s uncited now? Science. 1991;251(4989):25. ]

4. Pendlebury DA. Science, citation and funding [letter to the editor]. Science. 1991;251(5000):1410-1. ]

5. Abt HA. Science, citation, and funding. Science. 1991;251(5000):1408-9. ]

6. Garfield E. I had a dream... about uncitedness. The Scientist. 1998 [cited 2017 Aug 6];12(14):10. Available from: ]

7. Larivière V, Archambault É, Gingras Y, Wallace ML. The fall of uncitedness. In: Book of abstracts of the 10th International Conference on Science and Technology Indicators; 2008 Sept 17-29; Vienna, Austria. p.279-82. [ Links ]

8. Archambault E, Campbell D, Gingras Y, Larivière V. Comparing bibliometric statistics obtained from the Web of Science and Scopus. J Assoc Inf Sci Technol. 2009;60(7):1320-6. ]

9. Chen S, Arsenault C, Larivière V. Are top-cited papers more interdisciplinary? J Informetr. 2015;9(4):1034-46. ]

Funding: Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq – Process 303961/2013-9 and 149594/2015-1).

Received: December 31, 2016; Accepted: March 23, 2017

Correspondence: Ivan França Junior. Departamento de Saúde, Ciclos de Vida e Sociedade – FSP/USP. Av. Dr. Arnaldo, 715. 01246-904 São Paulo, SP, Brasil. E-mail:

Authors’ Contribution: Study conception and planning: IFJ, AMBC, MMALB. Data collection, analysis, and interpretation: MMALB, MCAA, KO, FPQ, AMBC, IFJ. Preparation and review of the manuscript: IFJ, AMBC, MMALB, MCAA, KO. Approval of the final version: IFJ, AMBC, MMALB. Manuscript critical review: MMALB. Public responsibility for the article’s content: AMBC, MMALB, MCAA, KO, FPQ, AMBC, IFJ.

Conflict of Interest: The authors declare no conflict of interest.

Creative Commons License  This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.