The development of an instrument to assess nursing care responsiveness at a university hospital

Abstracts

Um indicador de qualidade introduzido pela Organização Mundial da Saúde no campo da avaliação em saúde é a responsividade. Assim, o objetivo do trabalho foi elaborar um instrumento para avaliar a opinião do cliente sobre a responsividade do serviço de enfermagem de um hospital universitário público. A pesquisa desenvolveu-se nas seguintes etapas: construção do instrumento com base em levantamento bibliográfico, validação aparente, teste de aplicabilidade e teste piloto. O instrumento permite identificar as Expectativas e as Percepções do cliente, subdivididas em aspectos relacionados à Estrutura, Processo e Resultados, mensurados por meio da escala Likert com valores de 0 a 7. Com base nos resultados do teste piloto foi possível estabelecer uma indicação numérica da Taxa de Satisfação Relativa e do seu intervalo. O trabalho proporcionou a construção de uma escala de percentual de aceitabilidade para o intervalo, que pode indicar quanto o serviço de enfermagem responde às expectativas dos clientes.

Serviços de enfermagem; Qualidade da assistência à saúde; Indicadores de qualidade; Avaliação; Estudos de validação

ORIGINAL ARTICLE

The development of an instrument to assess nursing care responsiveness at a university hospital

Elaboración de un instrumento para evaluar la responsividad del Servicio de Enfermería de un Hospital Universitario

Ana Vanessa Deffaccio Rodrigues^I; Dagmar Willamowius Vituri^II; Maria do Carmo Lourenço Haddad^III; Marli Terezinha Oliveira Vannuchi^IV; William Tiago de Oliveira^V

^IRN. Resident in Nursing Service Management, University Hospital of the State University of Londrina. Londrina, PR, Brazil. [e-mail]

^IIRN. MS. Advisor in Nursing Service Quality, University Hospital of Londrina. Londrina, PR, Brazil. [e-mail]

^IIIRN. PhD. Professor, State University of Londrina, Nursing Department. Londrina, PR, Brazil. [e-mail]

^IVRN. PhD. Professor, State University of Londrina, Nursing Department. Londrina, PR, Brazil. [e-mail]

^VRN. Resident in Nursing Service Management, University Hospital of the State University of Londrina. Londrina, PR, Brazil. [e-mail]

ABSTRACT

Responsiveness is a quality indicator presented by the World Health Organization in the field of health evaluation. Therefore, the objective of this study was to develop an instrument to evaluate the clients' opinions regarding the responsiveness of the nursing service of a public university hospital. The research was developed in the following stages: developing the instrument based on a literature review, apparent validation, applicability test and pilot test. The instrument allows the identification of the clients' expectations and perceptions, subdivided into aspects related to the Structure Process and Outcomes, measured using a 7-point Likert scale. Based on the results of the pilot test it was possible to establish a numerical indication of the Relative Satisfaction Rate and its interval. The study promoted the development of an acceptability percentage scale for the interval, which indicates the extent to which the nursing service meets the clients' expectations.

Descriptors: Nursing services; Quality of health care; Quality indicators; Evaluation; Validation studies

RESUMEN

Responsividad es un indicador de calidad presentado por la Organización Mundial de la Salud en el campo de evaluación en salud. El objetivo del trabajo fue elaborar un instrumento de evaluación de opinión del paciente respecto a la responsividad del servicio de enfermería de hospital universitario público. Investigación desarrollada en las etapas: construcción del instrumento basado en reseña bibliográfica, validación aparente, prueba de aplicabilidad y prueba piloto. El instrumento permite identificar las Expectativas y Percepciones del paciente, subdivididas en aspectos relativos a Estructura, Proceso y Resultados, medido mediante escala Likert en valores de 0 a 7. En base a los resultados de prueba piloto, fue posible establecer una indicación numérica de Tasa de Satisfacción Relativa y de su intervalo. El trabajo proporcionó la construcción de una escala de porcentual de aceptabilidad para el intervalo, que puede indicar en cuanto responde el servicio de enfermería a las expectativas del paciente.

Descriptores: Servicios de enfermería; Calidad de la atención de salud; Indicadores de calidad; Evaluación; Estudios de validación

INTRODUCTION

Quality of a service is a result of the commitment and responsibility of those involved, who are willing to provide good service while not hurting the social, ethical and religious beliefs of clients and successfully fulfill their tasks. Based on this assumption, the evaluation of the quality of health services becomes essential to satisfying service users, permitting the detection of problems and failures in the process of delivering care to patients.

All those involved in the care process should participate in the development and implementation of evaluative processes to legitimate and ensure greater involvement in the search for quality. Quality indicators are another tool available to evaluate quality of care⁽¹⁾.

Evaluation of the quality of nursing care through indicators can be used to reinforce the natural desire of health workers to improve care at the same time as it helps to understand the quality of care. Evaluating health services is not a simple procedure and it is essential that this process be based on criteria predetermined by the service to achieve pre-established quality standards developed according to the local context⁽¹⁾.

A quality indicator introduced in the field of evaluation in healthcare is responsiveness. It emerged as an alternative to the concept 'satisfaction' given the subjective nature of the latter⁽²⁾. Responsiveness is related to the way health system design recognizes and responds to expectations of individuals universally recognized in relation to non-medical aspects of care⁽³⁾.

Responsiveness is translated into two dimensions: respect for people and client-orientation. Respect for people is related to the ethics involved in the interaction of patients with health services and is composed of the following: dignity, confidentiality, autonomy and communication. The second dimension, client-orientation, includes categories that influence the satisfaction of patients and are directly related to care: fast service, social support, facilities and choice⁽⁴⁾.

Research addressing responsiveness⁽³⁾ considers two elements of operation: measuring what happens when a client interacts with the system and measuring how the client perceives and evaluates what happened.

This study was conducted considering the relevance of the concept of 'responsiveness' and the importance of measuring how a public university hospital nursing service responds to the perceptions and expectations of clients. We believe that, based on the development and validation of an instrument to evaluate the responsiveness of a nursing service according to the opinion of clients, we can collect data and transform the data into information that is essential for implementing effective and efficacious management as well as sensitizing the nursing staff to consider care needs beyond the technical dimension of care.

METHOD

This quantitative study was conducted in a public university hospital located in the North of the state of Paraná, Brazil. The hospital is linked to the Brazilian Unified Health System (SUS) and has 317 beds distributed among female and male medical-surgical units, infectious and contagious diseases, tuberculosis treatment, burn treatment center, maternity, nursery, pediatrics, emergency department, neonatal, pediatric and adult intensive and semi-intensive care units. Additionally, it maintains differentiated sectors such as an eye bank, a bone marrow transplant and blood center, and also provides outpatient care.

The process of the instrument's development took place in 2009 after it was approved by the Nursing Board of the institution where the study was conducted and by the Research Ethics Committee at the State University of Londrina and registered in the Research Ethics National Information System (CAAE nº 0124.0.268.000-9). All those involved in the study, experts and patients, signed free and informed consent forms⁽⁵⁾.

The methodological procedures were adapted from an existing study⁽¹⁾, which allowed the implementation of the following stages: development of the instrument of evaluation of responsiveness of nursing services, face validity, applicability test, pilot test and computation of relative satisfaction.

The development of the instrument to evaluate responsiveness in nursing services was based on an extensive bibliographic search concerning quality of care^(1,3,6-9). Priorities were established in the first stage to measure the responsiveness of the nursing service at the hospital where the study was conducted. These priorities were based on three criteria: the importance of the care activity to be measured; the potential to improve its quality; and degree of control professionals performing care hold over the mechanisms that allow the desired improvement⁽¹⁰⁾.

The instrument was developed so that its measures can be applied in various sectors and facilities with the most varied context for adult patients with safety and precision of results. Considering the level of information and culture of the interviewees, the instrument can be considered a questionnaire when self-applied or a form if a researcher fills out the instrument⁽¹¹⁾.

Patients can evaluate the quality of service of a hospital in different ways by comparing how hospitals should provide services and their perceptions concerning how activities are performed in these facilities. The perception of the quality of a service is therefore the direction and extent to which the perceptions and expectations of consumers diverge⁽¹²⁾.

The term expectations is used in the literature and refers to the quality of services as opposed to the way the term consumer's satisfaction is used. 'Patient satisfaction' can be defined as positive individual evaluation of distinct dimensions of health care⁽¹³⁾. Expectation is considered a preview the consumer makes about something according to his/her desires or needs⁽¹²⁾, which is influenced by the individual's set of negative and positive experiences that influence one's behavior⁽¹⁴⁾.

Hence, based on these concepts, an instrument was developed to evaluate the responsiveness of a nursing service with two distinct purposes: one focused on the expectations of patients and another focused on their perceptions, which was subdivided into the categories Structure, Process and Results⁽⁷⁾.

Structure corresponds to the resources necessary to provide care, including physical area, human, material and financial resources, in addition to information systems. Process involves a great part of the complexity of the treatment and directly depends on user access to the service, ethical/interpersonal aspects, user autonomy, social support and communication/information the individual received and the work process of the nursing team. Results refer to the user's analysis of final products in terms of health.

A Likert scale, ranging from 1 to 7⁽¹²⁾, was used for patients to choose according to how strongly they agree or disagree with whether the service provides a given resource or condition: (1) strongly disagree and (7) strongly agree. The numbers between the two poles should be assigned according to how strong a feeling/impression is concerning the availability of a resource or condition. Zero is assigned when the patient has no opinion concerning the statement or never experienced the situation.

Face validity was tested. It is a subjective evaluation aimed to verify the superficial evidence of the integrity of what the instrument is supposed to measure. Thus, it is a superficial evaluation performed by those who will use it⁽¹⁵⁾.

The instrument was validated through a written and objective evaluation with all ten nurses from the Burn Treatment Center at the institution where the study was conducted.

At this point, the experts evaluated each statement concerning Expectations and Perceptions according to the requirements Attributable, Accessible, Communicable, Contextual, Effective/Precise, Feasible, Objective and assigned a score from 1 to 4 to each^(1,16). They also left written comments and suggestions in relation to each set of these components of indicators.

Each expert received an invitation letter to participate in the study, a free and informed consent form to sign, the instrument to evaluate, and instructions on how to fill it out. The results of the experts' evaluation were plotted and analyzed according to agreement among them. Changes in the instrument were implemented according to the relevance of the expert opinions.

After the instrument was reformulated, it was submitted to an applicability test in a randomized sample of five patients hospitalized in the female medical-surgical unit at the studied hospital. These patients had to have been hospitalized for more than four hours and be able to provide answers so the level of understanding in relation to the evaluation items could be verified. Data were plotted again and analyzed considering the difficulties the patients faced and suggestions were presented. The instrument was then reformulated.

A pilot test was applied after the applicability test and second reformulation. The pilot test was applied to an intentional sample of 20% of inpatients of the female medical-surgical unit of the studied hospital who were admitted at least two days before in the unit. The purpose was to verify the responsiveness of the service of the unit's nursing team in the four work shifts. After data were plotted and analyzed, the authors considered the construction of the instrument to evaluate the responsiveness of nursing services was concluded.

To validate the methodology and present a score for responsiveness in the nursing services, a formula used in another study was employed⁽¹⁷⁾. It computes the Relative Satisfaction (RS) rate where:

RS = Perception/Expectation X 100

The GAP between the distance of users' RS and full satisfaction was also computed⁽¹⁷⁾:

GAP = 100% - RS

Data were plotted in the Microsoft Office Excel program, 2003.

RESULTS

Of the ten nurses invited to participate in the study, only two did not evaluate the instrument: one was on maternity leave and another on vacation during the data collection period. The age of professionals ranged from 26 to 49 years old. In relation to time working in the studied institution: 37.5% of the individuals worked from 1 to 5 years, 25% from 6 to 15 years, 37.5% from 16 to 25 years, and only two (25%) individuals had another job.

The average agreement ranged from 85% to 100% for the total of 34 statements included in the evaluation instrument handed to experts, according to the required evaluative elements of Attribution, Accessibility, Communication, Context, Effective/Precise, Feasible and Objective. The comments and suggestions presented by the experts were analyzed and contributed to the instrument's reformulations.

Only one statement was removed from the instrument and another was changed based on the suggestions; four comments were disregarded in the reformulation. The other statements remained the same since, according to the participating nurses, they were clear and objective.

DISCUSSION

The only statement removed from the instrument, which concerned Expectation, was As a patient with no possibility of therapy, my family and I received support (emotional, social, psychological) from the institution. The reason is that it only received 77% of agreement and received comments and suggestions such as Item not applicable to patients since when patients are at a terminal phase they are not able to score this objective; I suggest changing patient with no possibility of therapy; and it does not measure the quality of nursing care. It measures the institution's quality of service because it involves teams in psychology and social work (multidisciplinary teams), not only nursing.

The statement The unit's employees should have Knowledge, Competence, Technique, Technical Ability and Organization to provide safe care and not expose patients to evitable risks received the comment The attribute may be confused with trust and empathy, which was disregarded since the authors believed that related aspects were clear.

The statement The time passed since admission and accommodation in the unit should be brief to avoid causing patients discomfort received the comment It did not take long is difficult to measure. How long is too long (two, three or seven days)? was also disregarded because the objective is to evaluate how convenient or inconvenient the time spent between admission and accommodation in the unit was; opinions may vary depend on what one considers too long or not too long.

The statement The unit's nursing staff always wears a visible and legible identification badge with name, profession and a picture received the following comment The use of name tags in closed areas may not be possible; thus this issue should be verified. Hence, the authors decided to consider this item only in units where the use of nametags is not restricted.

The statement The patient should have the opportunity to talk with nurses about his/her treatment and diagnosis received the following comment It is applicable but the patient does not seek out nurses to clarify doubts. They seek the physician. It's a cultural matter. Nurses only reinforce it. I suggest The nurses reinforced the importance of medical instructions in my treatment and diagnosis. We opted for disregarding the comment because it shows the extent to which nurses occupy their space as professionals within the institution and sector they work and is an important requirement to evaluate the quality of nursing.

The statement The patient should have permission to bring in food/articles not provided by the hospital and which do not interfere in the treatment was not changed according to the suggestion Redact permission because it is granted by the nutritionist. The reason it was disregarded is that nurses should also have control over the patient's food.

The comment Attribute not directly linked to nursing concerning the statement My family and I had the right to access information contained in the medical file guaranteed whenever asked was disregarded because nurses are the ones technically responsible for the patient's medical file and the mediary in the release of the medical file in case the patient or family ask for it.

The test of applicability of the instrument performed with patients revealed they had difficulties understanding and interpreting some statements. Thus, some sentences had to be changed to make them clearer. The pilot test enabled a first visualization of the application of the instrument.

After all the methodological stages were complied with, the Instrument of Evaluation of Responsiveness of Nursing Services - Expectations and Perceptions was considered concluded (^{Appendix A}).

Data presented in the pilot test permitted the computation of RS and GAP⁽¹⁷⁾. We opted for using the medians so that the central measure would not be much influenced by extreme values and also because the perception of some users exceeded expectations, which could alter the average concerning perception⁽¹⁸⁾.

Given the results, a scale of percentage of acceptability for the GAP was developed, which allowed specific analysis of the results. According to data presented in Table 1, a positive value means the service falls short of users' expectations, zero means the service is adequate and satisfies the user's expectations, while a negative value means the services exceeded the expectations of users.

Scores of zero concerning Expectations and Perceptions were removed from data analysis because zero indicates the user did not experience the situation and therefore does not have an opinion concerning the statement.

From the 1990s on, both public and private Brazilian organizations in different sectors in the fields of management and administration began to contribute with instruments and research methodologies to qualify managerial processes and the monitoring of the quality of services provided to citizens/users/clients⁽¹⁹⁾.

However, many criticized research addressing 'satisfaction' due to the construct's subjective aspect with many determinants, such as degree of individual expectations and requirements in relation to services and the patient's individual characteristics such as gender, age, social status and psychological status⁽²⁰⁾.

One of the challenges faced is the construction of instruments to evaluate the satisfaction of service users that takes into consideration educational and cultural differences and different ways of using the services, especially in relation to the process's cognitive aspects⁽¹⁹⁾.

A search for greater objectivity was one of the reasons for developing an instrument to measure how clients perceive the responsiveness of nursing services, considering the different approaches, types of questions, and how they were formulated to differ from research addressing the satisfaction of patients.

The need to develop an instrument emerged from the lack of standardization of instruments found in the literature that measure satisfaction and especially the lack of a specific instrument evaluating the responsiveness of nursing services from the perspective of users. The purpose was to provide a reference for comparative studies addressing responsiveness and, consequently, improve the routines of nursing services, enabling significant advancements in the area of health services management.

CONCLUSION

The evaluation of nursing care quality can encourage health workers to improve the quality of care provided to clients and also help understanding and the measuring of the dynamics of such care. Satisfaction is a broad and subjective construct, thus it cannot clearly evaluate care, while responsiveness indicates how the client perceives the services based on previous expectations.

This study was conducted taking these issues into account since the objective was to develop an instrument to evaluate the opinion of clients concerning the responsiveness of the nursing service of a university public hospital and identify their expectations and perceptions.

After the Evaluation of Responsiveness of Nursing Services instrument was developed, there were the following stages: Face validation; Applicability test; Pilot test; and Computation of Relative Satisfaction.

Average agreement reached among experts during the validation phase ranged from 85% to 100% per statement, some of which required changes for the study to proceed. The applicability test confirmed the target population would understand the instrument, that the changes were coherent and demonstrated it can be applied either as a questionnaire or a form. The pilot test provided data that enabled the computation of Relative Satisfaction Rate (RS) and interval (GAP).

This computation in turn enabled the development of a percentage scale concerning GAP acceptability, which may indicate the extent to which the nursing service meets the expectations of clients.

The objective of this study was achieved but further studies are required to determine the instrument's reliability and the standardization of evaluation of responsiveness so that comparative studies can lead to the improvement of nursing services.

REFERENCES

• 1. Vituri DW, Matsuda LM. Content validation of quality indicators for nursing care evaluation. Rev Esc Enferm USP [Internet]. 2009 [cited 2010 Aug 12];43(2):429-37. Available from: http://www.scielo.br/pdf/reeusp/v43n2/en_a24v43n2.pdf
• ²
  World Health Organization (WHO). The world health report 2000: Health Systems: improving performance [Internet]. Geneva; 2000 [cited 2010 Ago 15]. Available from: http://www.who.int/whr/2000/en/
• 3. Vaitsman J, Andrade GRB. Satisfação e responsividade: formas de medir a qualidade e a humanização da assistência à saúde. Ciênc Saúde Coletiva. 2005;10(3):599-613.
• 4. Silva A. A framework for measuring responsiveness [Internet]. Geneva: World Health Organization; 2000. (GPE Discussion Paper Series, 32). [cited 2010 Aug 15]. Available from: http://www.who.int/healthinfo/paper32.pdf
• 5. Conselho Nacional de Saúde. Resolução n. 196, de 10 de outubro de 1996. Dispõe sobre diretrizes e normas regulamentares de pesquisas envolvendo seres humanos. Bioética. 1996;4(2 Supl):15-25.
• ⁶
  Brasil. Ministério da Saúde; Secretaria de Atenção à Saúde. Avaliação para melhoria da qualidade da Estratégia de Saúde da Família: guia de implantação municipal AMQ [Internet]. Brasília; 2006 [citado 2010 set. 9]. Disponível em: http://dtr2002.saude.gov.br/proesf/autoavaliacaoesf/home/pdf/Guia%20de%20Implanta%C3%A7%C3%A3o%20AMQ.pdf
• 7. Donabedian A. The definition of quality and approaches to its assessment: explorations in quality: explorations in quality assessment and monitoring. Chicago: Health Administration Press; 1980.
• 8. Yuri NE, Tronchin DMR. Quality of maternal-child health care at a university hospital, according to the nurses' perspective. Rev Esc Enferm USP [Internet]. 2010 [cited 2010 Aug 12];44(2):331-8. Available from: http://www.scielo.br/pdf/reeusp/v44n2/en_13.pdf
• 9. Afanador NP. Indicadores de cuidado. Actual Enferm (Bogotá). 2001;4(1):13-8.
• 10. Mcglynn EA, Asch SM. Developing a clinical performance measure. Am J Prev Med. 1998;14(3):14-21.
• 11. Cervo AL, Bervian PA, Silva R. Metodologia científica. 6ª ed. São Paulo: Pearson Prentice Hall; 2007.
• 12. Parasuraman A, Zeithaml VA. Servqual: a multiple-item scale for measuring consumer perceptions of service quality. J Retailing. 1988;64(1):12-40.
• 13. Linder-Pelz S. Toward a theory of patient satisfaction. Soc Sci Med. 1982;16(5):577-82.
• 14. Oliver R. Measurement and evaluation of satisfaction process in retail settings. J Retailing. 1981;57(1):25-48.
• 15. Queijo AF, Padilha KG. Nursing Activities Score (NAS): cross-cultural adaptation and validation to portuguese language. Rev Esc Enferm USP [Internet]. 2009 [cited 2010 Aug 12];43(n.esp):1018-25. Available from: http://www.scielo.br/pdf/reeusp/v43nspe/en_a04v43ns.pdf
• 16. Fernandes MVL, Hallage NM. Construção e validação de indicadores de avaliação de práticas de controle e prevenção de infecção do trato urinário associada a cateter. Acta Paul Enferm. 2006;19(2):174-89.
• 17. Berry LL, Parasuraman A. Serviços de marketing: competindo através da qualidade. São Paulo: Maltese-Norma; 1992.
• 8. Cozin SK, Turrini RNT. Avaliação dos produtos e serviços de uma biblioteca de enfermagem: satisfação do usuário. Rev Latino Am Enferm. 2008;16(4):758-64.
• 19. Andrade GRB, Vaitsman J, Farias LOT. Metodologia de elaboração do Índice da Responsividade do Serviço (IRS). Cad Saúde Pública. 2010;26(3):523-534
• 20. Sitzia J, Wood N. Patient satisfaction: a review of issues and concepts. Soc Sci Med. 1997;45(12):1829-43.

Appendix A

Correspondence addressed to:

Publication Dates

• Publication in this collection
  20 Mar 2012
• Date of issue
  Feb 2012

History

• Received
  11 Nov 2010
• Accepted
  13 July 2011