Assessment of the different citation systems in the scientific publication of nursing authors from Spanish-speaking countries

Avaliação dos diferentes sistemas de citação na publicação científica de autores de enfermagem em espanhol

ABSTRACT

Objective:

To assess the distribution of citations of nursing authors in Spanish in Google Scholar as well as to compare the possible differences between this source and Web of Science and Scopus.

Method:

This is a descriptive cross-sectional study based on the citation systems offered by Google Scholar, Web of Science, and Scopus.

Results:

Nursing researchers present a verified mean h-index of 7.82 in Academic Google. 74% of researchers belong to the academic field, compared to 26%, who are in health services. Most of them live in Spain (83%), followed by Colombia (12%), Mexico (4%), and Chile (1%). In Spain, the community with the largest number of researchers is Andalusia (41.5%), followed by Valencia (14.6%), and Madrid (7.3%).

Conclusion:

The Google Scholar citation system requires adjustments in its algorithm for selecting works and citations, and it should also allow some system of confirmation by authors. Nursing can have relatively low h-index values compared to other courses due to short research development.

Nursing Research; Bibliometrics; Scientific Communication and Diffusion; Scholarly Communication

RESUMO

Objetivo:

Avaliar a distribuição de citações de autores de enfermagem em espanhol no Google Scholar, bem como comparar as possíveis diferenças entre esta fonte e Web of Science e Scopus.

Método:

Estudo descritivo transversal baseado nos sistemas de citação oferecidos pelo Google Scholar, Web of Science e Scopus.

Resultados:

Pesquisadores da área de enfermagem apresentam índice h médio verificado no Google Acadêmico de 7,82. 74% dos pesquisadores pertencem à área acadêmica, contra 26% que estão agrupados nos serviços de saúde. A maioria deles está localizada na Espanha (83%), seguida pela Colômbia (12%), México (4%) e Chile (1%). Na Espanha, a comunidade com maior número de pesquisadores é a Andaluzia (41,5%), seguida da Comunidade Valenciana (14,6%) e Madrid (7,3%).

Conclusão:

O sistema de citações do Google Scholar requer ajustes em seu algoritmo de seleção de obras e citações, devendo também permitir algum sistema de confirmação por parte dos autores. A enfermagem pode ter valores relativamente baixos do índice h em comparação com outras disciplinas devido ao curto desenvolvimento da pesquisa.

Pesquisa em Enfermagem; Bibliometria; Comunicação e Divulgação Científica; Comunicação Acadêmica

RESUMEN

Objetivo:

Evaluar la distribución de citas de autores enfermeros en español en Google Académico, así como comparar las posibles diferencias entre esta fuente y Web of Science y Scopus.

Método:

Estudio descriptivo transversal basado en los sistemas de citas ofrecidos por Google Académico, Web of Science y Scopus.

Resultados:

Los investigadores del área de enfermería presentan un índice h verificado medio de 7.82 en Google Académico. El 74% de los investigadores pertenece al ámbito académico, frente a un 26% que se aglutina en los servicios de salud. La mayoría de ellos se ubican en España (83%), seguido de Colombia (12%), Méjico (4%) y Chile (1%). En España, la comunidad que mayor número de investigadores aglutina es Andalucía (41,5%), seguida de la Comunidad Valencia (14,6%) y Madrid (7,3%).

Conclusión:

El sistema de citación de Google Académico precisa de ajustes en su algoritmo de selección de trabajos y citas, además debería permitir algún sistema de confirmación por parte de los autores. Enfermería puede tener valores relativamente bajos del índice h frente a otras disciplinas debido al breve desarrollo investigador.

Investigación en Enfermería; Bibliometría; Comunicación y Divulgación Científica; Comunicación Académica

INTRODUCTION

Publishing research results in specialized journals is the main way of disseminating knowledge in all academic areas. Scientific articles are the main instrument for measuring researcher productivity¹. Analysis of scientific production has become an essential element in academic institutions, and increasingly in health institutions, assuming not only a method of approach to the progress of courses or research groups, but also a factor of accessibility to research funding, individual economic incentives (six-year research)², or a resource for institutional or corporate visibility; although it is true that, in courses with an essential social component, such as nursing, an increase in visibility does not represent an increase in impact factor.

Among the various methods available for this purpose, assessing the impact of publications has been one of the fastest-growing. The impact factor, idealized in the 1970s, went through a radically different use from its initial conception, which aimed at selecting journals by academic institutions and documentation services. Thus, an assumption that journals with great impact factor would be the ones that would have the greatest relevance in research quickly permeated the academic and research community until it became the bargaining chip in personnel selection processes, competitive financing, or assessment of scientific activity³.

Criticisms of this hegemonic assessment system that are based on an assumption that does not necessarily reflect the reality of scientific production quality^4-6, led to the emergence of other formal assessment systems such as h-index⁷. This measure seeks to estimate the relevance of a researcher's contributions cumulatively over time, in order to avoid the classical limitations of the impact factor. Although this index is applied only to publications, the practice among academic institutions and research agencies of using it as an indicator of individual scientific production is widespread⁸.

A researcher obtains an h-index if they have published "h" papers with at least "h" citations. Thus, a high h-index means that researchers had had significant productivity and that the rest of the scientific community authors have cited them relevantly. However, this index penalizes young researchers in the early stages of their professional careers.

For some years, the systematic measurement of this index and citations by author has been available in several sources such as Google Scholar, Scopus and Web of Science (WoS). The accessibility of Google Scholar has caused the popularization of access to this resource, and its use in academic environments is increasing³. In the field of nursing knowledge, there is the possibility of immediately knowing the volume of citations of an author who is registered in the system as well as his or her h-index and the generation of author rankings by areas. However, it is necessary to determine the extent to which these measurements provide an approximate picture of the reality and the possible differences in results depending on the source used for measurement. Possible errors in the attribution of citations, or lack of attention by authors in updating their suitability, produces the phenomenon identified by Amezcua as "bastard authorship"⁹, which offers a distorted image of production and citation and, therefore, distorts these indexes' purpose. Thus, if an author does not refine citations automatically assigned by the system, citations will appear that may not be the result of their scientific activity, but which, spuriously, will artificially increase their h-index and total citation count.

Therefore, this study aims to assess the distribution of citations by Spanish and Latin American nursing authors included in the Google Academic Nursing descriptor, as well as to compare the possible differences between this source and WoS and Scopus.

METHOD

STUDY DESIGN

This cross-sectional descriptive study was carried out using the Google Academic, WoS and Scopus citation systems.

SELECTION CRITERIA

A search was conducted at the consultation service of this provider under the labels in Spanish "Enfermería" and "Enfemeria" (nursing) (without a tilde) between November 2018 and March 2019. For data collection, the closing date was used (number of citations obtained by authors) on March 4, 2019. A search was also carried out with the label "nursing". It was found that the authors who appeared under this search were included in the previous searches, although some authors who were only labeled with the English term were also identified.

DATA COLLECTION

To make the visibility and interpretation of data more manageable, the first 100 authors with the highest number of citations in Google Scholar were selected among the three different labels alluding to Enfermería (nursing) (with a tilde, without a tilde and in English). A list was made on the same day of which articles appeared in the database of each author and how many citations each article had, in order to know in the future to which article could be attributed a possible increase in the number of citations and to obtain reference information of the citation status on that date.

DATA ANALYSIS AND TREATMENT

A search was carried out in each author's profile, article by article, comparing the name that appeared in an author's profile with the names that appeared at the top of all articles, with the following conditions:

a) In articles in which an author's name did not appear at the top as authors, the assigned citations were discounted and articles and citations were removed in order to obtain a new h-index.

b) In articles in which an author's name appeared at the top in full as it appeared in their profile, the assigned citations were counted; however, in cases which an article's theme had nothing to do with nursing or health sciences (e.g. architecture, engineering, geology, etc.), although there was a coincidence of affiliation, meetings were not attributed.

c) In articles in which an author's name appeared partially at the top (not exactly or in a similar way) as it appeared in their profiles, a more exhaustive search was carried out in different databases; this could result in a removal of citations if, as a result of a search, a name did not match any of other names at the top of an article and did not resemble any of them in any way (the surname matches, but the name does not, the name matches, but the surname does not match, neither the name nor the surname matched), or if the article's subject had nothing to do with nursing or health sciences.

If there was any doubt whether the author object of the research could be one of the authors that appear at the top of the article, but there were elements, in addition to the name, that indicated that it could be the author (such as the nature of the article, usual co-authors, etc.), the citations were computed.

A new classification was made, from the data obtained, in order of number of citations, obtaining a new h-index.

The country, the autonomous community where they live (in the case of Spanish authors) and their field of work were determined (in those who developed their activity both in the clinical field and in that of teaching, the criterion was established belonging to the clinical environment). Finally, of the first 25 authors, a search was made on Scopus and WoS for their citations and h-index, to compare the results with those of Google Scholar.

ETHICAL ASPECTS

Approval by a research ethics committee is not required, since it is a bibliometric study on the citations of scientific nursing articles in Spanish.

RESULTS

The first 100 authors accumulated a total of 60,919 verified citations (compared to 137,612 attributed), which represents only 44.26% of the correct citations, with a verified mean h-index of 10.95 (compared to an attributed h-index of 13.72). The difference between attributed and verified citations exceeded 10,000 citations in authors such as Martinez Riera (-19,828 attributed citations that did not correspond to him), Calero García (-21,427) or Sánchez García (-12,132). Table 1 shows the original and corrected citations and h-index.

Table 1
List of authors analyzed in Google Scholar in Spanish-speaking countries - Spain, Colombia, Mexico, Chile, Cuba, and Argentina; 2019.

	COUNTRY	Community	Main institution	AUTHOR	Attributed citations	Verified citations	Attributed h-index	Verified h-index
1	Spain	C. Valenciana	University	CARMEN DE LA CUESTA	2052	2049	27	27
2	Spain	C. Valenciana	University	JOSE VERDU SORIANO	2346	2346	24	24
3	Spain	Andalucía	University	JOSE MIGUEL MORALES ASENCIO	2220	2220	24	24
4	Spain	Andalucía	University	EMILIO GONZALEZ JIMENEZ	2154	2075	25	24
5	Spain	Andalucía	University	LOURDES DIAZ RODRIGUEZ	1602	1358	25	23
6	Spain	La Rioja	University	JAVIER SOLDEVILLA AGREDA	1921	1854	23	23
7	Spain	C. Valenciana	University	FRANCISCO PEDRO GARCIA-FERNANDEZ	2110	2110	21	21
8	Spain	Andalucía	University	PEDRO L PANCORBO-HIDALGO	2078	2078	21	21
9	Spain	C. Valenciana	University	JOSE SILES	1944	1944	21	21
10	Spain	Madrid	Research Institute	TERESA MORENO CASBAS	4680	3979	23	20
11	Colombia	-	University	FRED MANRIQUE ABRIL	4829	1219	25	19
12	Spain	Andalucía	University	MANUEL AMEZCUA	2123	2123	19	19
13	Spain	Baleares	University	MIGUEL BENASSAR VENY	1070	1070	19	19
14	Colombia	-	University	ZULEIMA COGOLLO	936	884	19	19
15	Spain	Andalucía	University	JACQUELINE SCHMITDT RIO VALLE	1649	1010	20	17
16	Spain	Baleares	University	JORDI PERICAS	1519	1519	17	17
17	Spain	C. Valenciana	University	MCARMEN SOLANO RUIZ	823	807	17	17
18	Chile	-	University	SANDRA VALENZUELA SUAZO	845	845	16	16
19	Spain	Baleares	University	JOAN DE PEDRO	820	820	16	16
20	Colombia	-	University	BEATRIZ SANCHEZ HERRERA	1283	1240	17	15
21	Spain	Andalucía	University	CESAR HUESO MONTORO	865	865	15	15
22	Spain	Andalucía	University	INMACULADA GARCIA GARCIA	5426	905	37	14
23	Spain	Andalucía	University	EUGENIA GIL GARCIA	599	579	14	14
24	Spain	Andalucía	University	RAFAEL DEL PINO CASADO	641	641	14	14
25	Spain	Andalucía	University	JAVIER RAMOS TORRECILLAS	432	432	14	14
26	Spain	C. Valenciana	University	MARIA ISABEL ORTS-CORTES	841	703	15	13
27	Spain	C. Valenciana	Health Services	PABLO LOPEZ CASANOVA	812	612	15	13
28	Colombia	-	University	CONSUELO VELEZ ALVAREZ	655	529	14	13
29	Spain	Navarra	University	MARIA ARANTZAMAENDI	568	568	13	13
30	Spain	Navarra	University	ANA CARVAJAL VALCARCEL	680	569	13	12
31	Spain	Andalucía	Health Services	SERGIO R. LOPEZ ALONSO	417	417	12	12
32	Colombia	-	University	FABIO ALBERTO CAMARGO FIGUERA	392	392	12	12
33	Spain	C. Valenciana	University	LORETO MACIA SOLER	522	522	11	11
34	Spain	C. Valenciana	University	JOSE RAMON MARTINEZ RIERA	20306	478	59	11
35	Spain	Baleares	Health Services	CONCHA ZAFORTEZA	428	428	11	11
36	Mexico	-	University	KARLA SELENE LOPEZ GARCIA	306	306	11	11
37	Spain	Andalucía	University	MARIA ISABEL LOPEZ MEDINA *	997	971	10	10
38	Spain	Andalucía	University	ISABEL M LOPEZ MEDINA *	1558	927	14	10
39	Mexico	-	University	VICTORIA FERNANDEZ GARCIA	631	631	10	10
40	Spain	Andalucía	Health Services	ADOLFO ROMERO	419	419	10	10
41	Spain	Andalucía	University	RAFA MONTOYA	355	355	10	10
42	Spain	Andalucía	University	MANUEL RODRIGIEZ PALMA	342	342	10	10
43	Spain	Andalucía	University	MARTA LIMA SERRANO	320	320	10	10
44	Spain	Andalucía	Health Services	MARIA DEL CARMEN RODRIGUEZ TORRES	280	274	10	10
45	Colombia	-	University	CLAUDIA PATRICIA VALENCIA MORA	242	242	10	10
46	Mexico	-	University	FRANCISCO RAFAEL GUZMAN FACUNDO	464	464	9	9
47	Spain	Navarra	University	BLANCA MARIN	500	362	12	9
48	Spain	C. Valenciana	University	JOSEP ADOLF GUIRAO GORIS	450	450	9	9
49	Colombia	-	University	IRMA YOLANDA CASTILLO AVILA	393	188	11	9
50	Spain	Andalucía	University	LIMA-RODRIGUEZ JS	349	268	10	9
51	Spain	C. Valenciana	University	MT ROMA FERRI	342	342	9	9
52	Mexico	-	University	REY ARTURO SALCEDO ALVAREZ	292	287	9	9
53	Spain	Murcia	University	CESAR CARRILLO GARCIA	280	264	9	9
54	Spain	Andalucía	University	MARIA DOLORES GUERRA-MARTIN	275	214	9	9
55	Spain	Andalucía	University	JOSE LUIS GOMEZ URQUIZA	233	233	9	9
56	Spain	Navarra	University	AGURTZANE MUJIKA	209	209	9	9
57	Spain	C. Valenciana	Health Services	ENRIQUE J. VERA REMATINEZ	201	201	9	9
58	Spain	Castilla y León	Health Services	DANIEL FERNANDEZ	181	181	9	9
59	Colombia	-	University	JUAN GUILLERMO ROJAS	439	416	8	8
60	Spain	Cantabria	University	ROSARIO FERNANDEZ PEÑA	345	223	8	8
61	Spain	Navarra	University	LETICIA SAN MARTIN RODRIGUEZ	2327	2327	8	8
62	Spain	Andalucía	Health Services	MARIA DE LA PAZ SALIDO MORENO	2113	112	19	8
63	Colombia	-	University	LUZ MARINA BAUTISTA ROGIGUEZ	548	296	14	8
64	Spain	Murcia	University	ANTONIO JESUS RAMOS MORCILLO	238	176	9	8
65	Spain	Castilla y León	University	M ELENA FERNANDEZ MARTINEZ	230	230	8	8
66	Spain	Andalucía	Health Services	JOSE CARLOS CANCA SANCHEZ	226	226	8	8
67	Spain	Cataluña	University	LEYVA MORAL J M	222	222	8	8
68	Colombia	-	University	GIOMAR M HERRERA-AMAYA	219	203	8	8
69	Spain	Andalucía	Health Services	SERAFIN FERNANDEZ-SALAZAR	198	198	8	8
70	Spain	Andalucía	University	MIGUEL ANGEL CALVO CALVO	195	138	8	8
71	Spain	Andalucía	Health Services	JOAQUIN JESUS BLANCA GUTIERREZ	193	193	8	8
72	Spain	Castilla La Mancha	University	JOSE MANUEL FERNANDEZ	191	191	8	8
73	Mexico	-	University	ROSA M OSTIGUIN MELENDEZ	266	266	7	7
74	Colombia	-	University	BARRAGAN-BECERRA JULIAN ANDRES	150	150	7	7
75	Spain	Galicia	Health Services	EVA TIZON-BOUZA	132	132	7	7
76	Mexico	-	University	NORA PATRICIA SANCHEZ CHAVEZ	111	111	7	7
77	Spain	Castilla y León	University	ANA BELEN SANCHEZ GARCIA	12264	132	50	7
78	Spain	Andalucía	University	CARMEN ALVAREZ NIETO	773	773	7	7
79	Spain	Murcia	University	EDNA GOMEZ BUSTAMENTE	384	250	9	7
80	Colombia	-	University	MARIA EUGENIA PICO MERCHÁN	258	258	7	7
81	Argentina	-	University	JOSE VELA	239	239	7	7
82	Spain	Asturias	Health Services	PATRICIO SUAREZ-GIL	211	198	7	7
83	Spain	Andalucía	University	ANA MARIA PORCEL GALVEZ	190	190	7	7
84	Cuba	-	University	NISLENI MARTINEZ	70	70	6	6
85	Spain	Andalucía	University	CRISTINA O'FERRAL	7600	60	40	6
86	Spain	Andalucía	University	JUAN GOMEZ SALGADO	950	127	13	6
87	Colombia	-	University	LUZ NELLY RIVERA ALVAREZ	264	262	6	6
88	Spain	Galicia	Health Services	ANGEL ALFREDO MARTINEZ QUES	234	234	6	6
89	Spain	Castilla y León	University	NORA ANGELICA ARMENDARIZ GARCIA	226	151	8	6
90	Spain	Canarias	Health Services	GONZALO DUARTE CLIMENTS	209	166	7	6
91	Colombia	-	University	YEIS MIGUEL BORRE ORTIZ	208	208	6	6
92	Spain	La Rioja	University	RAUL JUAREZ VELA	177	177	6	6
93	Chile	-	University	VERONICA GUERRA	176	176	5	5
94	Colombia	-	University	YADIRA PABON	48	48	5	5
95	Spain	Andalucía	University	MJ CALERO-GARCIA	21493	66	66	5
96	Colombia	-	University	CAROLINA LUCERO ENRIQUEZ CEREZO	288	152	8	5
97	Spain	Cataluña	University	EMILIA CHIRVECHES PEREZ	217	210	6	5
98	Spain	Extremadura	University	SERGIO RICO MARTIN	206	45	5	5
99	Mexico	-	University	Mº DEL CARMEN PEREZ RODRIGUEZ	190	90	6	5
100	Spain	Andalucía	University	LUIS JAVIER MORENO CORRAL	187	187	5	5

• ^*
  Duplicate author with different affiliation in Google Scholar

Most authors are located in Spain (n=73; 73%), followed by Colombia (n=16; 16%), Mexico (n=7; 7%), Chile (n=2; 2%), Cuba (n=1; 1%) and Argentina (n=1; 1%).

Most authors (85%) belong to the academic field, compared to 15%, who are grouped in health services. The citation analysis of Google Scholar in the first 25 authors regarding the other two large sources such as Scopus and WoS showed notable differences (Table 2). The mean h-index for WOS was 9.28, 8.76 on Scopus and 18.92 (verified) on Google Scholar.

Table 2
Differences in citations and h-index between sources - Spain, Colombia, and Chile; 2019.

	COUNTRY	Community	AUTHOR	GS citations	GS h-index	WoS citations	WoS h-index	Scopus citations	Scopus h-index
1	Spain	C. Valenciana	CARMEN DE LA CUESTA	2049	27	173	7	300	11
2	Spain	C. Valenciana	JOSE VERDU SORIANO	2346	24	258	8	798	13
3	Spain	Andalucía	JOSE MIGUEL MORALES ASENCIO	2220	24	580	12	559	12
4	Spain	Andalucía	EMILIO GONZALEZ JIMENEZ	2075	24	758	16	793	17
5	Spain	Andalucía	LOURDES DIAZ RODRIGUEZ	1358	23	817	18	821	18
6	Spain	La Rioja	JAVIER SOLDEVILLA AGREDA	1854	23	277	8	386	11
7	Spain	C. Valenciana	FRANCISCO P. GARCIA-FERNANDEZ	2110	21	634	8	593	9
8	Spain	Andalucía	PEDRO L PANCORBO-HIDALGO	2078	21	659	9	691	10
9	Spain	C. Valenciana	JOSE SILES	1944	21	159	7	99	6
10	Spain	Madrid	TERESA MORENO CASBAS	3979	20	1792	17	2024	15
11	Colombia	-	FRED MANRIQUE ABRIL	1219	19	184	8	91	6
12	Spain	Andalucía	MANUEL AMEZCUA	2123	19	695	12	222	6
13	Spain	Baleares	MIGUEL BENASSAR VENY	1070	19	368	11	430	12
14	Colombia	-	ZULEIMA COGOLLO	884	19	179	8	89	5
15	Spain	Andalucía	JACQUELINE SCHMITDT RIO VALLE	1010	17	331	11	250	5
16	Spain	Baleares	JORDI PERICAS	1519	17	114	6	542	7
17	Spain	C. Valenciana	MCARMEN SOLANO RUIZ	807	17	161	6	119	6
18	Chile	-	SANDRA VALENZUELA SUAZO	845	16	158	7	41	3
19	Spain	Baleares	JOAN DE PEDRO	820	16	227	9	173	8
20	Colombia	-	BEATRIZ SANCHEZ HERRERA	1240	15	238	9	51	4
21	Spain	Andalucía	CESAR HUESO MONTORO	865	15	112	6	59	4
22	Spain	Andalucía	INMACULADA GARCIA GARCIA	905	14	308	7	278	7
23	Spain	Andalucía	EUGENIA GIL GARCIA	579	14	83	5	131	6
24	Spain	Andalucía	RAFAEL DEL PINO CASADO	641	14	172	6	191	7
25	Spain	Andalucía	JAVIER RAMOS TORRECILLAS	432	14	280	11	294	11

DISCUSSION

This study aims to assess the impact of researchers and nursing students from Google Scholar in Spain. The results show a classification that collects interesting aspects from a point of view of the distribution of the impact of citations among authors, autonomous communities and work environments. One of the first signs that Google Scholar requires essential adjustments in its selection and works algorithms is the important difference found in some authors, which added up to thousands of erroneous citations and should be taken into account from an institutional point of view when assessing academic and research merits. However, although Google Scholar allows some adjustments, such as review and editing the citations that are suggested or even combine or delete publications, the problem is in the system automation without prior verification by authors, which generates the incorporation of citations and incorrect publications.

On the other hand, the geographic distribution pattern found is related to some of the findings of nursing production in Spain developed by the Spanish Foundation for Science and Technology (FECYT - Fundación Española para la Ciencia y la Tecnología)¹⁰. It should be noted that the unequal relationship between the theoretical potential based on the territorial dimension and its scientific production is scarce both at the level of authors and citations. This relationship must be deepened, seeking to find the causes for which certain geographical areas do not respond to their potential for scientific production¹¹.

H-index, as a rule, is usually linked to an author's specific area; therefore, it can contribute to provide an indication of citation patterns for that area¹². However, an article can be cited for many reasons, which can go beyond the merely scientific, even for revocation purposes. Citations do not always imply the quality of authorship or relevance of a publication in an author's scientific field; he or she may have obtained multiple citations about an article that does not necessarily belong to his area of knowledge, having been integrated into a multidisciplinary group that may have a more extensive citation profile than the original author. In nursing, this phenomenon may be especially relevant due to an asymmetry that still exists in research development of nurses within multidisciplinary teams in health sciences, so that they are integrated into consolidated groups in other fields that serve as a means to obtain publications and impact, but with little impact on nursing science; young researchers are the most likely to explore this path. In the case of Brazil, with the presence of only three national nursing journals in WoS, Brazilian citations will always be minimized. Currently, only Scopus comes close to covering the same value as SciELO, which is today the most comprehensive assessment indicator for nursing journals in the Ibero-American universe¹³. This phenomenon is essential within the academic context. Curricular assessment of candidates for teaching vacancies in Departments of Nursing can generate competitive advantages for professionals who have not developed their career as a nursing researcher, and may even determine the progress of the lines of research and supervision of the nursing course in the departments.

Another essential element is obtaining citations from works in which the person analyzed is not a key author, but benefits from the citations obtained. Citation count does not adequately discriminate this aspect, which can occur in multicentric studies at the national or international level, or in research networks, in which the fundamental contribution of the study does not correspond to the analyzed person in question, but to other members of the body of authors. The creation of research networks and collaborative studies has a notable effect on researchers' performance since it favors the exchange of resources, knowledge, and experiences¹⁴. Concerning bibliometric impact, current methods do not discriminate well in this type of situation⁸. In the current state of care research in Spain, with a notable expansion of research groups, but minimal possibilities of financing care research networks¹¹, it is difficult to estimate the extent of this impact.

When comparing the h index of English articles with other areas, relatively low values for nursing are observed, with mean values around 11¹². In our study, the mean value of the h index of Spanish authors analyzed in the same source they used is close to 7. Taking into account that a value of 20 is established as an adequate threshold for a consolidated researcher, it is evident that nursing has a citation pattern possibly different from that of applied science or other fields of knowledge¹². In other words, it is possible that the social nature (in part) of the nursing area, among other factors, is negatively affecting the citation of authors in this scientific area³.

Other authors, after analyzing the academic scientific production of Canadian nurses, propose that an article cited in the field of nursing ten times or more is a "sound article"; 50 or more citations, as a "very sound" article; 100 citations or more, as an "excellent article"; over 150 citations, as an "exceptional article". Likewise, they propose that h-indexes between 5 and 9 show authors with a well-established record of nursing publications, and values between 10 and 14 as an excellent record, placing the threshold of 15 as an exceptional record for a nursing author¹⁵. It is necessary to contrast these types of proposals, trying to control the essential language bias that occurs at the expense of non-Anglo-Saxon authors. The influence of where the results of nursing research are published is inevitable, taking into account the context of expansion and scientific consolidation of nursing¹⁶.

With its successes and imperfections, the analysis of the impact of nursing academics and scientists is inevitable. As methods and criteria are improved, the visibility of nursing research will reach higher levels; however, it should not be forgotten that the final key of research is the contribution to society and that efforts in assessing the social impact must be redoubled¹⁷. Moreover, the fact that assessment methods should not focus exclusively on the bibliometric impact, as is the case today¹⁸.

The fact that a research result is widely cited does not necessarily imply that it is implemented in health services or that it reaches patients, caregivers, or the general public. Also, it does not imply that this research transcends the professional or conceptual educational field, as is the case with multiple results of research on human responses that later do not to transcend the Diagnostic Taxonomy of the North America Nursing Diagnosis Association (NANDA)¹⁹. It should not be forgotten that bibliometric impact is a surrogate variable of the real impact of an investigation.

Some authors suggest the need for a bibliometric indicator that incorporates the social and clinical impact of the research findings according to the area of knowledge and compensates for the bias of the work's publication language²⁰. In the same vein, other authors highlight the difference in the number of citations in favor of open access publications at the expense of classic paid publications, in addition to language bias²¹.

The limitations of this type of analysis are related to the opacity of inclusion and selection criteria of sources used by Google Scholar, which prevents one from knowing in detail the adequacy and extent of the citations. The differences between Google Scholar, Scopus, and WoS only highlight the importance of this aspect and the necessary transparency in measurement methods, in addition to the fact that Google Scholar can also compute works of a non-scientific nature²². Moreover, not only are the data sources of each platform different, but WoS only collects citations from indexed journals from its collection, and Scopus, despite having more journals than WoS, remains a more limited universe of sources compared to the heterogeneous spectrum covered by Google Scholar. On the other hand, with the development and expansion of the social media, it is difficult to assess to what extent the use of these media as alternative ways of disseminating authors' works can increase their visibility to a greater or lesser degree; therefore, the possibility of citation by other authors²³ is an aspect that must be assessed in the future, in addition to the growing importance that researchers must give to it in their dissemination strategies. As the social media expanded, the limits of the metric of services exceeded the conventional ones, and methods such as webometrics or altimetrics will certainly transform the impact analysis in a not too distant future¹⁸. Furthermore, one cannot rule out that there are some authors from the disciplinary field who have not been covered by the term "Enfermería" in its different variants.

CONCLUSION

The Google Scholar citation system requires essential adjustments in its algorithm for selecting works and citations. Moreover, there should be a system that forces authors to verify them. H-index, with its limitations, can be a sound method for assessing the quality of scientific production. However, the nursing citation pattern is possibly different and, together with its more recent research path, could generate relatively low h-index values compared to other areas. This issue should be taken into account when establishing limits for the h-index assessment using references from other areas.

REFERENCES

• ¹
  Avena MJ, Barbosa DA. Bibliometric indicators of the nursing journals according to the index databases. Rev Esc Enferm USP. 2017;51:e03262. doi: http://dx.doi.org/10.1590/S1980-220X2017014603262
  » http://dx.doi.org/10.1590/S1980-220X2017014603262
• ²
  España. Ministerio de Educación, Cultura y Deporte. Resolución de 30 de noviembre de 2016 de la Secretaría de Estado de Educación, Formación Profesional y Universidades, por la que se fija el procedimiento y plazo de presentación de solicitudes de evaluación de la actividad investigadora a la Comisión Nacional Evaluadora de la Actividad Investigadora [Internet]. Madrid; 2016 [citado 2019 mar. 15]. Disponible en: https://www.boe.es/diario_boe/txt.php?id=BOE-A-2016-11464
  » https://www.boe.es/diario_boe/txt.php?id=BOE-A-2016-11464
• ³
  Cabezas-Clavijo A, Delgado-López-Cózar E. Google Scholar and the h-index in biomedicine: the popularization of bibliometric assessment. Med Intensiva. 2013;37(5):343-54. doi: 10.1016/j.medine.2013.01.008
  » https://doi.org/10.1016/j.medine.2013.01.008
• ⁴
  Amezcua M. Controversias en la evaluación del conocimiento: alegatos a propósito de una ciencia aplicada. Index Enferm. 2011;20(1-2):7-11. doi: https://dx.doi.org/10.4321/S1132-12962011000100002
  » https://dx.doi.org/10.4321/S1132-12962011000100002
• ⁵
  Amezcua M ¿Para qué sirve el Indice de Impacto de una revista? Index Enferm [Internet]. 2010 [citado 2019 mar.15];19(2-3):83-7. Disponible en: http://scielo.isciii.es/scielo.php?script=sci_arttext&pid=S1132-12962010000200002
  » http://scielo.isciii.es/scielo.php?script=sci_arttext&pid=S1132-12962010000200002
• ⁶
  Ravenscroft J, Liakata M, Clare A, Duma D. Measuring scientific impact beyond academia: an assessment of existing impact metrics and proposed improvements. PLoS One. 2017;12(3):e0173152. doi: https://doi.org/10.1371/journal.pone.0173152
  » https://doi.org/10.1371/journal.pone.0173152
• ⁷
  Hirsch JE. An index to quantify an individual's scientific research output. Proc Natl Acad Sci U S A. 2005;102(46):16569-72. doi: https://doi.org/10.1073/pnas.0507655102
  » https://doi.org/10.1073/pnas.0507655102
• ⁸
  Génova G, Astudillo H, Fraga A. The scientometric bubble considered harmful. Sci Eng Ethics. 2016;22(1):227-35. doi:10.1007/s11948-015-9632-6
  » https://doi.org/10.1007/s11948-015-9632-6
• ⁹
  Amezcua M. Autorías bastardas en Google Scholar, otra expresión de impostura científica. Index Enf [Internet]. 2018 [citado 2019 mar. 15];27(3):115-7. Disponible en: http://scielo.isciii.es/scielo.php?script=sci_arttext&pid=S1132-12962018000200002
  » http://scielo.isciii.es/scielo.php?script=sci_arttext&pid=S1132-12962018000200002
• ¹⁰
  Fundación Española para la Ciencia y la Tecnología. Indicadores bibliométricos de la actividad científica española 2005-2014 [Internet]. Madrid: FECYT; 2017 [citado 2019 ene. 19]. Disponible en: https://www.fecyt.es/es/publicacion/indicadores-bibliometricos-de-la-actividad-cientifica-espanola-2005-2014
  » https://www.fecyt.es/es/publicacion/indicadores-bibliometricos-de-la-actividad-cientifica-espanola-2005-2014
• ¹¹
  Morales Asencio JM, Hueso Montoro C, de Pedro-Gómez JE, Bennasar-Veny M. 1977-2017: la investigación enfermera en España tras 40 años en la universidad. Enferm Clin. 2017;27(5):314-26. doi: https://doi.org/10.1016/j.enfcli.2017.08.003
  » https://doi.org/10.1016/j.enfcli.2017.08.003
• ¹²
  Thompson DR, Watson R. h-indices and the performance of professors of nursing in the UK. J Clin Nurs. 2010;19:2957-8. doi: 10.1111/j.1365-2702.2010.03267.x
  » https://doi.org/10.1111/j.1365-2702.2010.03267.x
• ¹³
  Mugnaini R. The Impact Factor: its popularity and impacts, and the need to preserve the scientific knowledge generation process [editorial]. Rev Esc Enferm USP. 2016;50(5):722-3. doi: http://dx.doi.org/10.1590/s0080-623420160000600002
  » http://dx.doi.org/10.1590/s0080-623420160000600002
• ¹⁴
  Contandriopoulos D, Duhoux A, Larouche C, Perroux M. The impact of a researcher's structural position on scientific performance: an empirical analysis. PLoS One. 2016;11(8):e0161281. doi: 10.1371/journal.pone.0161281
  » https://doi.org/10.1371/journal.pone.0161281
• ¹⁵
  Hack TF, Crooks D, Plohman J, Kepron E. Research citation analysis of nursing academics in Canada: identifying success indicators. J Adv Nurs. 2010;66(11):2542-9. doi: 10.1111/j.1365-2648.2010.05429.x
  » https://doi.org/10.1111/j.1365-2648.2010.05429.x
• ¹⁶
  Lima-Serrano M, Lima-Rodríguez JS, Porcel-Gálvez AM, Gil-García E ¿Cómo mejorar la visibilidad de la investigación enfermera española? Publicaciones de referencia e índices de calidad. Enferm Clin. 2015;25(1):27-32. doi: 10.1016/j.enfcli.2014.11.003
  » https://doi.org/10.1016/j.enfcli.2014.11.003
• ¹⁷
  Amezcua M, Amezcua González A, Pozuelo Zurera S, Maldonado Hidalgo AM. También las publicaciones deben comprometerse con el impacto social. Index Enferm. 2015;24(3):186-90. doi: https://dx.doi.org/10.4321/S1132-12962015000200015
  » https://dx.doi.org/10.4321/S1132-12962015000200015
• ¹⁸
  Davidson PM, Newton PJ, Ferguson C, Daly J, Elliott D, Homer C, et al. Rating and ranking the role of bibliometrics and webometrics in nursing and midwifery. ScientificWorldJournal. 2014;2014:135812. doi: 10.1155/2014/135812
  » https://doi.org/10.1155/2014/135812
• ¹⁹
  Morales Asencio J, Morilla Herrera J. Investigación sobre resultados enfermeros. Precisión y elección del criterio de resultado. Investigación en metodología y lenguajes enfermeros. Barcelona: Elsevier; 2016. p. 231-42.
• ²⁰
  Powell KR, Peterson SR. Coverage and quality: A comparison of Web of Science and Scopus databases for reporting faculty nursing publication metrics. Nurs Outlook. 2017;65(5):572-8. doi:10.1016/j.outlook.2017.03.004
  » https://doi.org/10.1016/j.outlook.2017.03.004
• ²¹
  Avena MJ, Barbosa DA. Brazilian nursing journals: strengths, weaknesses and challenges. Rev Bras Enferm [Internet]. 2018;71(5):2489-95. doi: https://doi.org/10.1590/0034-7167-2017-0828
  » https://doi.org/10.1590/0034-7167-2017-0828
• ²²
  Torres-Salinas D, Ruiz-Pérez R, Delgado-López-Cozar E. Google Scholar como herramienta para la evaluación científica. El Prof Inf [Internet]. 2009 [citado 2019 ene.19];18(5):501-10. Disponible en: http://www.elprofesionaldelainformacion.com/contenidos/2009/septiembre/03.pdf
  » http://www.elprofesionaldelainformacion.com/contenidos/2009/septiembre/03.pdf
• ²³
  Smith DR, Watson R. Career development tips for today's nursing academic: bibliometrics, altmetrics and social media. J Adv Nurs. 2016;72(11):2654-61. doi: 10.1111/jan.13067
  » https://doi.org/10.1111/jan.13067

Publication Dates

• Publication in this collection
  16 Apr 2021
• Date of issue
  2021

History

• Received
  10 June 2019
• Accepted
  23 May 2020