Assessment of the different citation systems in the scientific publication of nursing authors from Spanish-speaking countries

Avaliação dos diferentes sistemas de citação na publicação científica de autores de enfermagem em espanhol

Joan De Pedro-Gómez Miquel Bennasar-Veny Francisco Guillem-Conejeros Marcos Falque-Vidal José Miguel Morales-Asencio About the authors

ABSTRACT

Objective:

To assess the distribution of citations of nursing authors in Spanish in Google Scholar as well as to compare the possible differences between this source and Web of Science and Scopus.

Method:

This is a descriptive cross-sectional study based on the citation systems offered by Google Scholar, Web of Science, and Scopus.

Results:

Nursing researchers present a verified mean h-index of 7.82 in Academic Google. 74% of researchers belong to the academic field, compared to 26%, who are in health services. Most of them live in Spain (83%), followed by Colombia (12%), Mexico (4%), and Chile (1%). In Spain, the community with the largest number of researchers is Andalusia (41.5%), followed by Valencia (14.6%), and Madrid (7.3%).

Conclusion:

The Google Scholar citation system requires adjustments in its algorithm for selecting works and citations, and it should also allow some system of confirmation by authors. Nursing can have relatively low h-index values compared to other courses due to short research development.

Nursing Research; Bibliometrics; Scientific Communication and Diffusion; Scholarly Communication

RESUMO

Objetivo:

Avaliar a distribuição de citações de autores de enfermagem em espanhol no Google Scholar, bem como comparar as possíveis diferenças entre esta fonte e Web of Science e Scopus.

Método:

Estudo descritivo transversal baseado nos sistemas de citação oferecidos pelo Google Scholar, Web of Science e Scopus.

Resultados:

Pesquisadores da área de enfermagem apresentam índice h médio verificado no Google Acadêmico de 7,82. 74% dos pesquisadores pertencem à área acadêmica, contra 26% que estão agrupados nos serviços de saúde. A maioria deles está localizada na Espanha (83%), seguida pela Colômbia (12%), México (4%) e Chile (1%). Na Espanha, a comunidade com maior número de pesquisadores é a Andaluzia (41,5%), seguida da Comunidade Valenciana (14,6%) e Madrid (7,3%).

Conclusão:

O sistema de citações do Google Scholar requer ajustes em seu algoritmo de seleção de obras e citações, devendo também permitir algum sistema de confirmação por parte dos autores. A enfermagem pode ter valores relativamente baixos do índice h em comparação com outras disciplinas devido ao curto desenvolvimento da pesquisa.

Pesquisa em Enfermagem; Bibliometria; Comunicação e Divulgação Científica; Comunicação Acadêmica

RESUMEN

Objetivo:

Evaluar la distribución de citas de autores enfermeros en español en Google Académico, así como comparar las posibles diferencias entre esta fuente y Web of Science y Scopus.

Método:

Estudio descriptivo transversal basado en los sistemas de citas ofrecidos por Google Académico, Web of Science y Scopus.

Resultados:

Los investigadores del área de enfermería presentan un índice h verificado medio de 7.82 en Google Académico. El 74% de los investigadores pertenece al ámbito académico, frente a un 26% que se aglutina en los servicios de salud. La mayoría de ellos se ubican en España (83%), seguido de Colombia (12%), Méjico (4%) y Chile (1%). En España, la comunidad que mayor número de investigadores aglutina es Andalucía (41,5%), seguida de la Comunidad Valencia (14,6%) y Madrid (7,3%).

Conclusión:

El sistema de citación de Google Académico precisa de ajustes en su algoritmo de selección de trabajos y citas, además debería permitir algún sistema de confirmación por parte de los autores. Enfermería puede tener valores relativamente bajos del índice h frente a otras disciplinas debido al breve desarrollo investigador.

Investigación en Enfermería; Bibliometría; Comunicación y Divulgación Científica; Comunicación Académica

INTRODUCTION

Publishing research results in specialized journals is the main way of disseminating knowledge in all academic areas. Scientific articles are the main instrument for measuring researcher productivity11 Avena MJ, Barbosa DA. Bibliometric indicators of the nursing journals according to the index databases. Rev Esc Enferm USP. 2017;51:e03262. doi: http://dx.doi.org/10.1590/S1980-220X2017014603262
http://dx.doi.org/10.1590/S1980-220X2017...
. Analysis of scientific production has become an essential element in academic institutions, and increasingly in health institutions, assuming not only a method of approach to the progress of courses or research groups, but also a factor of accessibility to research funding, individual economic incentives (six-year research)22 España. Ministerio de Educación, Cultura y Deporte. Resolución de 30 de noviembre de 2016 de la Secretaría de Estado de Educación, Formación Profesional y Universidades, por la que se fija el procedimiento y plazo de presentación de solicitudes de evaluación de la actividad investigadora a la Comisión Nacional Evaluadora de la Actividad Investigadora [Internet]. Madrid; 2016 [citado 2019 mar. 15]. Disponible en: https://www.boe.es/diario_boe/txt.php?id=BOE-A-2016-11464
https://www.boe.es/diario_boe/txt.php?id...
, or a resource for institutional or corporate visibility; although it is true that, in courses with an essential social component, such as nursing, an increase in visibility does not represent an increase in impact factor.

Among the various methods available for this purpose, assessing the impact of publications has been one of the fastest-growing. The impact factor, idealized in the 1970s, went through a radically different use from its initial conception, which aimed at selecting journals by academic institutions and documentation services. Thus, an assumption that journals with great impact factor would be the ones that would have the greatest relevance in research quickly permeated the academic and research community until it became the bargaining chip in personnel selection processes, competitive financing, or assessment of scientific activity33 Cabezas-Clavijo A, Delgado-López-Cózar E. Google Scholar and the h-index in biomedicine: the popularization of bibliometric assessment. Med Intensiva. 2013;37(5):343-54. doi: 10.1016/j.medine.2013.01.008
https://doi.org/10.1016/j.medine.2013.01...
.

Criticisms of this hegemonic assessment system that are based on an assumption that does not necessarily reflect the reality of scientific production quality44 Amezcua M. Controversias en la evaluación del conocimiento: alegatos a propósito de una ciencia aplicada. Index Enferm. 2011;20(1-2):7-11. doi: https://dx.doi.org/10.4321/S1132-12962011000100002
https://dx.doi.org/10.4321/S1132-1296201...

5 Amezcua M ¿Para qué sirve el Indice de Impacto de una revista? Index Enferm [Internet]. 2010 [citado 2019 mar.15];19(2-3):83-7. Disponible en: http://scielo.isciii.es/scielo.php?script=sci_arttext&pid=S1132-12962010000200002
http://scielo.isciii.es/scielo.php?scrip...
-66 Ravenscroft J, Liakata M, Clare A, Duma D. Measuring scientific impact beyond academia: an assessment of existing impact metrics and proposed improvements. PLoS One. 2017;12(3):e0173152. doi: https://doi.org/10.1371/journal.pone.0173152
https://doi.org/10.1371/journal.pone.017...
, led to the emergence of other formal assessment systems such as h-index77 Hirsch JE. An index to quantify an individual's scientific research output. Proc Natl Acad Sci U S A. 2005;102(46):16569-72. doi: https://doi.org/10.1073/pnas.0507655102
https://doi.org/10.1073/pnas.0507655102...
. This measure seeks to estimate the relevance of a researcher's contributions cumulatively over time, in order to avoid the classical limitations of the impact factor. Although this index is applied only to publications, the practice among academic institutions and research agencies of using it as an indicator of individual scientific production is widespread88 Génova G, Astudillo H, Fraga A. The scientometric bubble considered harmful. Sci Eng Ethics. 2016;22(1):227-35. doi:10.1007/s11948-015-9632-6
https://doi.org/10.1007/s11948-015-9632-...
.

A researcher obtains an h-index if they have published "h" papers with at least "h" citations. Thus, a high h-index means that researchers had had significant productivity and that the rest of the scientific community authors have cited them relevantly. However, this index penalizes young researchers in the early stages of their professional careers.

For some years, the systematic measurement of this index and citations by author has been available in several sources such as Google Scholar, Scopus and Web of Science (WoS). The accessibility of Google Scholar has caused the popularization of access to this resource, and its use in academic environments is increasing33 Cabezas-Clavijo A, Delgado-López-Cózar E. Google Scholar and the h-index in biomedicine: the popularization of bibliometric assessment. Med Intensiva. 2013;37(5):343-54. doi: 10.1016/j.medine.2013.01.008
https://doi.org/10.1016/j.medine.2013.01...
. In the field of nursing knowledge, there is the possibility of immediately knowing the volume of citations of an author who is registered in the system as well as his or her h-index and the generation of author rankings by areas. However, it is necessary to determine the extent to which these measurements provide an approximate picture of the reality and the possible differences in results depending on the source used for measurement. Possible errors in the attribution of citations, or lack of attention by authors in updating their suitability, produces the phenomenon identified by Amezcua as "bastard authorship"99 Amezcua M. Autorías bastardas en Google Scholar, otra expresión de impostura científica. Index Enf [Internet]. 2018 [citado 2019 mar. 15];27(3):115-7. Disponible en: http://scielo.isciii.es/scielo.php?script=sci_arttext&pid=S1132-12962018000200002
http://scielo.isciii.es/scielo.php?scrip...
, which offers a distorted image of production and citation and, therefore, distorts these indexes' purpose. Thus, if an author does not refine citations automatically assigned by the system, citations will appear that may not be the result of their scientific activity, but which, spuriously, will artificially increase their h-index and total citation count.

Therefore, this study aims to assess the distribution of citations by Spanish and Latin American nursing authors included in the Google Academic Nursing descriptor, as well as to compare the possible differences between this source and WoS and Scopus.

METHOD

STUDY DESIGN

This cross-sectional descriptive study was carried out using the Google Academic, WoS and Scopus citation systems.

SELECTION CRITERIA

A search was conducted at the consultation service of this provider under the labels in Spanish "Enfermería" and "Enfemeria" (nursing) (without a tilde) between November 2018 and March 2019. For data collection, the closing date was used (number of citations obtained by authors) on March 4, 2019. A search was also carried out with the label "nursing". It was found that the authors who appeared under this search were included in the previous searches, although some authors who were only labeled with the English term were also identified.

DATA COLLECTION

To make the visibility and interpretation of data more manageable, the first 100 authors with the highest number of citations in Google Scholar were selected among the three different labels alluding to Enfermería (nursing) (with a tilde, without a tilde and in English). A list was made on the same day of which articles appeared in the database of each author and how many citations each article had, in order to know in the future to which article could be attributed a possible increase in the number of citations and to obtain reference information of the citation status on that date.

DATA ANALYSIS AND TREATMENT

A search was carried out in each author's profile, article by article, comparing the name that appeared in an author's profile with the names that appeared at the top of all articles, with the following conditions:

a) In articles in which an author's name did not appear at the top as authors, the assigned citations were discounted and articles and citations were removed in order to obtain a new h-index.

b) In articles in which an author's name appeared at the top in full as it appeared in their profile, the assigned citations were counted; however, in cases which an article's theme had nothing to do with nursing or health sciences (e.g. architecture, engineering, geology, etc.), although there was a coincidence of affiliation, meetings were not attributed.

c) In articles in which an author's name appeared partially at the top (not exactly or in a similar way) as it appeared in their profiles, a more exhaustive search was carried out in different databases; this could result in a removal of citations if, as a result of a search, a name did not match any of other names at the top of an article and did not resemble any of them in any way (the surname matches, but the name does not, the name matches, but the surname does not match, neither the name nor the surname matched), or if the article's subject had nothing to do with nursing or health sciences.

If there was any doubt whether the author object of the research could be one of the authors that appear at the top of the article, but there were elements, in addition to the name, that indicated that it could be the author (such as the nature of the article, usual co-authors, etc.), the citations were computed.

A new classification was made, from the data obtained, in order of number of citations, obtaining a new h-index.

The country, the autonomous community where they live (in the case of Spanish authors) and their field of work were determined (in those who developed their activity both in the clinical field and in that of teaching, the criterion was established belonging to the clinical environment). Finally, of the first 25 authors, a search was made on Scopus and WoS for their citations and h-index, to compare the results with those of Google Scholar.

ETHICAL ASPECTS

Approval by a research ethics committee is not required, since it is a bibliometric study on the citations of scientific nursing articles in Spanish.

RESULTS

The first 100 authors accumulated a total of 60,919 verified citations (compared to 137,612 attributed), which represents only 44.26% of the correct citations, with a verified mean h-index of 10.95 (compared to an attributed h-index of 13.72). The difference between attributed and verified citations exceeded 10,000 citations in authors such as Martinez Riera (-19,828 attributed citations that did not correspond to him), Calero García (-21,427) or Sánchez García (-12,132). Table 1 shows the original and corrected citations and h-index.

Table 1
List of authors analyzed in Google Scholar in Spanish-speaking countries - Spain, Colombia, Mexico, Chile, Cuba, and Argentina; 2019.

Most authors are located in Spain (n=73; 73%), followed by Colombia (n=16; 16%), Mexico (n=7; 7%), Chile (n=2; 2%), Cuba (n=1; 1%) and Argentina (n=1; 1%).

Most authors (85%) belong to the academic field, compared to 15%, who are grouped in health services. The citation analysis of Google Scholar in the first 25 authors regarding the other two large sources such as Scopus and WoS showed notable differences (Table 2). The mean h-index for WOS was 9.28, 8.76 on Scopus and 18.92 (verified) on Google Scholar.

Table 2
Differences in citations and h-index between sources - Spain, Colombia, and Chile; 2019.

DISCUSSION

This study aims to assess the impact of researchers and nursing students from Google Scholar in Spain. The results show a classification that collects interesting aspects from a point of view of the distribution of the impact of citations among authors, autonomous communities and work environments. One of the first signs that Google Scholar requires essential adjustments in its selection and works algorithms is the important difference found in some authors, which added up to thousands of erroneous citations and should be taken into account from an institutional point of view when assessing academic and research merits. However, although Google Scholar allows some adjustments, such as review and editing the citations that are suggested or even combine or delete publications, the problem is in the system automation without prior verification by authors, which generates the incorporation of citations and incorrect publications.

On the other hand, the geographic distribution pattern found is related to some of the findings of nursing production in Spain developed by the Spanish Foundation for Science and Technology (FECYT - Fundación Española para la Ciencia y la Tecnología)1010 Fundación Española para la Ciencia y la Tecnología. Indicadores bibliométricos de la actividad científica española 2005-2014 [Internet]. Madrid: FECYT; 2017 [citado 2019 ene. 19]. Disponible en: https://www.fecyt.es/es/publicacion/indicadores-bibliometricos-de-la-actividad-cientifica-espanola-2005-2014
https://www.fecyt.es/es/publicacion/indi...
. It should be noted that the unequal relationship between the theoretical potential based on the territorial dimension and its scientific production is scarce both at the level of authors and citations. This relationship must be deepened, seeking to find the causes for which certain geographical areas do not respond to their potential for scientific production1111 Morales Asencio JM, Hueso Montoro C, de Pedro-Gómez JE, Bennasar-Veny M. 1977-2017: la investigación enfermera en España tras 40 años en la universidad. Enferm Clin. 2017;27(5):314-26. doi: https://doi.org/10.1016/j.enfcli.2017.08.003
https://doi.org/10.1016/j.enfcli.2017.08...
.

H-index, as a rule, is usually linked to an author's specific area; therefore, it can contribute to provide an indication of citation patterns for that area1212 Thompson DR, Watson R. h-indices and the performance of professors of nursing in the UK. J Clin Nurs. 2010;19:2957-8. doi: 10.1111/j.1365-2702.2010.03267.x
https://doi.org/10.1111/j.1365-2702.2010...
. However, an article can be cited for many reasons, which can go beyond the merely scientific, even for revocation purposes. Citations do not always imply the quality of authorship or relevance of a publication in an author's scientific field; he or she may have obtained multiple citations about an article that does not necessarily belong to his area of knowledge, having been integrated into a multidisciplinary group that may have a more extensive citation profile than the original author. In nursing, this phenomenon may be especially relevant due to an asymmetry that still exists in research development of nurses within multidisciplinary teams in health sciences, so that they are integrated into consolidated groups in other fields that serve as a means to obtain publications and impact, but with little impact on nursing science; young researchers are the most likely to explore this path. In the case of Brazil, with the presence of only three national nursing journals in WoS, Brazilian citations will always be minimized. Currently, only Scopus comes close to covering the same value as SciELO, which is today the most comprehensive assessment indicator for nursing journals in the Ibero-American universe1313 Mugnaini R. The Impact Factor: its popularity and impacts, and the need to preserve the scientific knowledge generation process [editorial]. Rev Esc Enferm USP. 2016;50(5):722-3. doi: http://dx.doi.org/10.1590/s0080-623420160000600002
http://dx.doi.org/10.1590/s0080-62342016...
. This phenomenon is essential within the academic context. Curricular assessment of candidates for teaching vacancies in Departments of Nursing can generate competitive advantages for professionals who have not developed their career as a nursing researcher, and may even determine the progress of the lines of research and supervision of the nursing course in the departments.

Another essential element is obtaining citations from works in which the person analyzed is not a key author, but benefits from the citations obtained. Citation count does not adequately discriminate this aspect, which can occur in multicentric studies at the national or international level, or in research networks, in which the fundamental contribution of the study does not correspond to the analyzed person in question, but to other members of the body of authors. The creation of research networks and collaborative studies has a notable effect on researchers' performance since it favors the exchange of resources, knowledge, and experiences1414 Contandriopoulos D, Duhoux A, Larouche C, Perroux M. The impact of a researcher's structural position on scientific performance: an empirical analysis. PLoS One. 2016;11(8):e0161281. doi: 10.1371/journal.pone.0161281
https://doi.org/10.1371/journal.pone.016...
. Concerning bibliometric impact, current methods do not discriminate well in this type of situation88 Génova G, Astudillo H, Fraga A. The scientometric bubble considered harmful. Sci Eng Ethics. 2016;22(1):227-35. doi:10.1007/s11948-015-9632-6
https://doi.org/10.1007/s11948-015-9632-...
. In the current state of care research in Spain, with a notable expansion of research groups, but minimal possibilities of financing care research networks1111 Morales Asencio JM, Hueso Montoro C, de Pedro-Gómez JE, Bennasar-Veny M. 1977-2017: la investigación enfermera en España tras 40 años en la universidad. Enferm Clin. 2017;27(5):314-26. doi: https://doi.org/10.1016/j.enfcli.2017.08.003
https://doi.org/10.1016/j.enfcli.2017.08...
, it is difficult to estimate the extent of this impact.

When comparing the h index of English articles with other areas, relatively low values for nursing are observed, with mean values around 111212 Thompson DR, Watson R. h-indices and the performance of professors of nursing in the UK. J Clin Nurs. 2010;19:2957-8. doi: 10.1111/j.1365-2702.2010.03267.x
https://doi.org/10.1111/j.1365-2702.2010...
. In our study, the mean value of the h index of Spanish authors analyzed in the same source they used is close to 7. Taking into account that a value of 20 is established as an adequate threshold for a consolidated researcher, it is evident that nursing has a citation pattern possibly different from that of applied science or other fields of knowledge1212 Thompson DR, Watson R. h-indices and the performance of professors of nursing in the UK. J Clin Nurs. 2010;19:2957-8. doi: 10.1111/j.1365-2702.2010.03267.x
https://doi.org/10.1111/j.1365-2702.2010...
. In other words, it is possible that the social nature (in part) of the nursing area, among other factors, is negatively affecting the citation of authors in this scientific area33 Cabezas-Clavijo A, Delgado-López-Cózar E. Google Scholar and the h-index in biomedicine: the popularization of bibliometric assessment. Med Intensiva. 2013;37(5):343-54. doi: 10.1016/j.medine.2013.01.008
https://doi.org/10.1016/j.medine.2013.01...
.

Other authors, after analyzing the academic scientific production of Canadian nurses, propose that an article cited in the field of nursing ten times or more is a "sound article"; 50 or more citations, as a "very sound" article; 100 citations or more, as an "excellent article"; over 150 citations, as an "exceptional article". Likewise, they propose that h-indexes between 5 and 9 show authors with a well-established record of nursing publications, and values between 10 and 14 as an excellent record, placing the threshold of 15 as an exceptional record for a nursing author1515 Hack TF, Crooks D, Plohman J, Kepron E. Research citation analysis of nursing academics in Canada: identifying success indicators. J Adv Nurs. 2010;66(11):2542-9. doi: 10.1111/j.1365-2648.2010.05429.x
https://doi.org/10.1111/j.1365-2648.2010...
. It is necessary to contrast these types of proposals, trying to control the essential language bias that occurs at the expense of non-Anglo-Saxon authors. The influence of where the results of nursing research are published is inevitable, taking into account the context of expansion and scientific consolidation of nursing1616 Lima-Serrano M, Lima-Rodríguez JS, Porcel-Gálvez AM, Gil-García E ¿Cómo mejorar la visibilidad de la investigación enfermera española? Publicaciones de referencia e índices de calidad. Enferm Clin. 2015;25(1):27-32. doi: 10.1016/j.enfcli.2014.11.003
https://doi.org/10.1016/j.enfcli.2014.11...
.

With its successes and imperfections, the analysis of the impact of nursing academics and scientists is inevitable. As methods and criteria are improved, the visibility of nursing research will reach higher levels; however, it should not be forgotten that the final key of research is the contribution to society and that efforts in assessing the social impact must be redoubled1717 Amezcua M, Amezcua González A, Pozuelo Zurera S, Maldonado Hidalgo AM. También las publicaciones deben comprometerse con el impacto social. Index Enferm. 2015;24(3):186-90. doi: https://dx.doi.org/10.4321/S1132-12962015000200015
https://dx.doi.org/10.4321/S1132-1296201...
. Moreover, the fact that assessment methods should not focus exclusively on the bibliometric impact, as is the case today1818 Davidson PM, Newton PJ, Ferguson C, Daly J, Elliott D, Homer C, et al. Rating and ranking the role of bibliometrics and webometrics in nursing and midwifery. ScientificWorldJournal. 2014;2014:135812. doi: 10.1155/2014/135812
https://doi.org/10.1155/2014/135812...
.

The fact that a research result is widely cited does not necessarily imply that it is implemented in health services or that it reaches patients, caregivers, or the general public. Also, it does not imply that this research transcends the professional or conceptual educational field, as is the case with multiple results of research on human responses that later do not to transcend the Diagnostic Taxonomy of the North America Nursing Diagnosis Association (NANDA)1919 Morales Asencio J, Morilla Herrera J. Investigación sobre resultados enfermeros. Precisión y elección del criterio de resultado. Investigación en metodología y lenguajes enfermeros. Barcelona: Elsevier; 2016. p. 231-42.. It should not be forgotten that bibliometric impact is a surrogate variable of the real impact of an investigation.

Some authors suggest the need for a bibliometric indicator that incorporates the social and clinical impact of the research findings according to the area of knowledge and compensates for the bias of the work's publication language2020 Powell KR, Peterson SR. Coverage and quality: A comparison of Web of Science and Scopus databases for reporting faculty nursing publication metrics. Nurs Outlook. 2017;65(5):572-8. doi:10.1016/j.outlook.2017.03.004
https://doi.org/10.1016/j.outlook.2017.0...
. In the same vein, other authors highlight the difference in the number of citations in favor of open access publications at the expense of classic paid publications, in addition to language bias2121 Avena MJ, Barbosa DA. Brazilian nursing journals: strengths, weaknesses and challenges. Rev Bras Enferm [Internet]. 2018;71(5):2489-95. doi: https://doi.org/10.1590/0034-7167-2017-0828
https://doi.org/10.1590/0034-7167-2017-0...
.

The limitations of this type of analysis are related to the opacity of inclusion and selection criteria of sources used by Google Scholar, which prevents one from knowing in detail the adequacy and extent of the citations. The differences between Google Scholar, Scopus, and WoS only highlight the importance of this aspect and the necessary transparency in measurement methods, in addition to the fact that Google Scholar can also compute works of a non-scientific nature2222 Torres-Salinas D, Ruiz-Pérez R, Delgado-López-Cozar E. Google Scholar como herramienta para la evaluación científica. El Prof Inf [Internet]. 2009 [citado 2019 ene.19];18(5):501-10. Disponible en: http://www.elprofesionaldelainformacion.com/contenidos/2009/septiembre/03.pdf
http://www.elprofesionaldelainformacion....
. Moreover, not only are the data sources of each platform different, but WoS only collects citations from indexed journals from its collection, and Scopus, despite having more journals than WoS, remains a more limited universe of sources compared to the heterogeneous spectrum covered by Google Scholar. On the other hand, with the development and expansion of the social media, it is difficult to assess to what extent the use of these media as alternative ways of disseminating authors' works can increase their visibility to a greater or lesser degree; therefore, the possibility of citation by other authors2323 Smith DR, Watson R. Career development tips for today's nursing academic: bibliometrics, altmetrics and social media. J Adv Nurs. 2016;72(11):2654-61. doi: 10.1111/jan.13067
https://doi.org/10.1111/jan.13067...
is an aspect that must be assessed in the future, in addition to the growing importance that researchers must give to it in their dissemination strategies. As the social media expanded, the limits of the metric of services exceeded the conventional ones, and methods such as webometrics or altimetrics will certainly transform the impact analysis in a not too distant future1818 Davidson PM, Newton PJ, Ferguson C, Daly J, Elliott D, Homer C, et al. Rating and ranking the role of bibliometrics and webometrics in nursing and midwifery. ScientificWorldJournal. 2014;2014:135812. doi: 10.1155/2014/135812
https://doi.org/10.1155/2014/135812...
. Furthermore, one cannot rule out that there are some authors from the disciplinary field who have not been covered by the term "Enfermería" in its different variants.

CONCLUSION

The Google Scholar citation system requires essential adjustments in its algorithm for selecting works and citations. Moreover, there should be a system that forces authors to verify them. H-index, with its limitations, can be a sound method for assessing the quality of scientific production. However, the nursing citation pattern is possibly different and, together with its more recent research path, could generate relatively low h-index values compared to other areas. This issue should be taken into account when establishing limits for the h-index assessment using references from other areas.

REFERENCES

  • 1
    Avena MJ, Barbosa DA. Bibliometric indicators of the nursing journals according to the index databases. Rev Esc Enferm USP. 2017;51:e03262. doi: http://dx.doi.org/10.1590/S1980-220X2017014603262
    » http://dx.doi.org/10.1590/S1980-220X2017014603262
  • 2
    España. Ministerio de Educación, Cultura y Deporte. Resolución de 30 de noviembre de 2016 de la Secretaría de Estado de Educación, Formación Profesional y Universidades, por la que se fija el procedimiento y plazo de presentación de solicitudes de evaluación de la actividad investigadora a la Comisión Nacional Evaluadora de la Actividad Investigadora [Internet]. Madrid; 2016 [citado 2019 mar. 15]. Disponible en: https://www.boe.es/diario_boe/txt.php?id=BOE-A-2016-11464
    » https://www.boe.es/diario_boe/txt.php?id=BOE-A-2016-11464
  • 3
    Cabezas-Clavijo A, Delgado-López-Cózar E. Google Scholar and the h-index in biomedicine: the popularization of bibliometric assessment. Med Intensiva. 2013;37(5):343-54. doi: 10.1016/j.medine.2013.01.008
    » https://doi.org/10.1016/j.medine.2013.01.008
  • 4
    Amezcua M. Controversias en la evaluación del conocimiento: alegatos a propósito de una ciencia aplicada. Index Enferm. 2011;20(1-2):7-11. doi: https://dx.doi.org/10.4321/S1132-12962011000100002
    » https://dx.doi.org/10.4321/S1132-12962011000100002
  • 5
    Amezcua M ¿Para qué sirve el Indice de Impacto de una revista? Index Enferm [Internet]. 2010 [citado 2019 mar.15];19(2-3):83-7. Disponible en: http://scielo.isciii.es/scielo.php?script=sci_arttext&pid=S1132-12962010000200002
    » http://scielo.isciii.es/scielo.php?script=sci_arttext&pid=S1132-12962010000200002
  • 6
    Ravenscroft J, Liakata M, Clare A, Duma D. Measuring scientific impact beyond academia: an assessment of existing impact metrics and proposed improvements. PLoS One. 2017;12(3):e0173152. doi: https://doi.org/10.1371/journal.pone.0173152
    » https://doi.org/10.1371/journal.pone.0173152
  • 7
    Hirsch JE. An index to quantify an individual's scientific research output. Proc Natl Acad Sci U S A. 2005;102(46):16569-72. doi: https://doi.org/10.1073/pnas.0507655102
    » https://doi.org/10.1073/pnas.0507655102
  • 8
    Génova G, Astudillo H, Fraga A. The scientometric bubble considered harmful. Sci Eng Ethics. 2016;22(1):227-35. doi:10.1007/s11948-015-9632-6
    » https://doi.org/10.1007/s11948-015-9632-6
  • 9
    Amezcua M. Autorías bastardas en Google Scholar, otra expresión de impostura científica. Index Enf [Internet]. 2018 [citado 2019 mar. 15];27(3):115-7. Disponible en: http://scielo.isciii.es/scielo.php?script=sci_arttext&pid=S1132-12962018000200002
    » http://scielo.isciii.es/scielo.php?script=sci_arttext&pid=S1132-12962018000200002
  • 10
    Fundación Española para la Ciencia y la Tecnología. Indicadores bibliométricos de la actividad científica española 2005-2014 [Internet]. Madrid: FECYT; 2017 [citado 2019 ene. 19]. Disponible en: https://www.fecyt.es/es/publicacion/indicadores-bibliometricos-de-la-actividad-cientifica-espanola-2005-2014
    » https://www.fecyt.es/es/publicacion/indicadores-bibliometricos-de-la-actividad-cientifica-espanola-2005-2014
  • 11
    Morales Asencio JM, Hueso Montoro C, de Pedro-Gómez JE, Bennasar-Veny M. 1977-2017: la investigación enfermera en España tras 40 años en la universidad. Enferm Clin. 2017;27(5):314-26. doi: https://doi.org/10.1016/j.enfcli.2017.08.003
    » https://doi.org/10.1016/j.enfcli.2017.08.003
  • 12
    Thompson DR, Watson R. h-indices and the performance of professors of nursing in the UK. J Clin Nurs. 2010;19:2957-8. doi: 10.1111/j.1365-2702.2010.03267.x
    » https://doi.org/10.1111/j.1365-2702.2010.03267.x
  • 13
    Mugnaini R. The Impact Factor: its popularity and impacts, and the need to preserve the scientific knowledge generation process [editorial]. Rev Esc Enferm USP. 2016;50(5):722-3. doi: http://dx.doi.org/10.1590/s0080-623420160000600002
    » http://dx.doi.org/10.1590/s0080-623420160000600002
  • 14
    Contandriopoulos D, Duhoux A, Larouche C, Perroux M. The impact of a researcher's structural position on scientific performance: an empirical analysis. PLoS One. 2016;11(8):e0161281. doi: 10.1371/journal.pone.0161281
    » https://doi.org/10.1371/journal.pone.0161281
  • 15
    Hack TF, Crooks D, Plohman J, Kepron E. Research citation analysis of nursing academics in Canada: identifying success indicators. J Adv Nurs. 2010;66(11):2542-9. doi: 10.1111/j.1365-2648.2010.05429.x
    » https://doi.org/10.1111/j.1365-2648.2010.05429.x
  • 16
    Lima-Serrano M, Lima-Rodríguez JS, Porcel-Gálvez AM, Gil-García E ¿Cómo mejorar la visibilidad de la investigación enfermera española? Publicaciones de referencia e índices de calidad. Enferm Clin. 2015;25(1):27-32. doi: 10.1016/j.enfcli.2014.11.003
    » https://doi.org/10.1016/j.enfcli.2014.11.003
  • 17
    Amezcua M, Amezcua González A, Pozuelo Zurera S, Maldonado Hidalgo AM. También las publicaciones deben comprometerse con el impacto social. Index Enferm. 2015;24(3):186-90. doi: https://dx.doi.org/10.4321/S1132-12962015000200015
    » https://dx.doi.org/10.4321/S1132-12962015000200015
  • 18
    Davidson PM, Newton PJ, Ferguson C, Daly J, Elliott D, Homer C, et al. Rating and ranking the role of bibliometrics and webometrics in nursing and midwifery. ScientificWorldJournal. 2014;2014:135812. doi: 10.1155/2014/135812
    » https://doi.org/10.1155/2014/135812
  • 19
    Morales Asencio J, Morilla Herrera J. Investigación sobre resultados enfermeros. Precisión y elección del criterio de resultado. Investigación en metodología y lenguajes enfermeros. Barcelona: Elsevier; 2016. p. 231-42.
  • 20
    Powell KR, Peterson SR. Coverage and quality: A comparison of Web of Science and Scopus databases for reporting faculty nursing publication metrics. Nurs Outlook. 2017;65(5):572-8. doi:10.1016/j.outlook.2017.03.004
    » https://doi.org/10.1016/j.outlook.2017.03.004
  • 21
    Avena MJ, Barbosa DA. Brazilian nursing journals: strengths, weaknesses and challenges. Rev Bras Enferm [Internet]. 2018;71(5):2489-95. doi: https://doi.org/10.1590/0034-7167-2017-0828
    » https://doi.org/10.1590/0034-7167-2017-0828
  • 22
    Torres-Salinas D, Ruiz-Pérez R, Delgado-López-Cozar E. Google Scholar como herramienta para la evaluación científica. El Prof Inf [Internet]. 2009 [citado 2019 ene.19];18(5):501-10. Disponible en: http://www.elprofesionaldelainformacion.com/contenidos/2009/septiembre/03.pdf
    » http://www.elprofesionaldelainformacion.com/contenidos/2009/septiembre/03.pdf
  • 23
    Smith DR, Watson R. Career development tips for today's nursing academic: bibliometrics, altmetrics and social media. J Adv Nurs. 2016;72(11):2654-61. doi: 10.1111/jan.13067
    » https://doi.org/10.1111/jan.13067

Publication Dates

  • Publication in this collection
    16 Apr 2021
  • Date of issue
    2021

History

  • Received
    10 June 2019
  • Accepted
    23 May 2020
Universidade de São Paulo, Escola de Enfermagem Av. Dr. Enéas de Carvalho Aguiar, 419 , 05403-000 São Paulo - SP/ Brasil, Tel./Fax: (55 11) 3061-7553, - São Paulo - SP - Brazil
E-mail: reeusp@usp.br