Acessibilidade / Reportar erro

Instrument for assessing the quality of mobile emergency pre-hospital care: content validation* ( Extracted from the thesis "Avaliação da qualidade da assistência prestada pelo Serviço de Atendimento Móvel de Urgência do Rio Grande do Norte", Universidade Federal do Rio Grande do Norte, 2014.

Instrumento Para Evaluación De La Calidad De La Asistencia Pre Hospitalaria Móvil De Urgencia: Validación De Contenido

Abstracts

OBJECTIVES

To validate an instrument to assess quality of mobile emergency pre-hospital care.

METHOD

A methodological study where 20 professionals gave their opinions on the items of the proposed instrument. The analysis was performed using Kappa test (K) and Content Validity Index (CVI), considering K> 0.80 and CVI ≥ 0.80.

RESULTS

Three items were excluded from the instrument: Professional Compensation; Job Satisfaction and Services Performed. Items that obtained adequate K and CVI indexes and remained in the instrument were: ambulance conservation status; physical structure; comfort in the ambulance; availability of material resources; user/staff safety; continuous learning; safety demonstrated by the team; access; welcoming; humanization; response time; costumer privacy; guidelines on care; relationship between professionals and costumers; opportunity for costumers to make complaints and multiprofessional conjunction/actuation.

CONCLUSION

The instrument to assess quality of care has been validated and may contribute to the evaluation of pre-hospital care in mobile emergency services.

Emergency Nursing; Prehospital Care; Validation Studies


OBJETIVOS

Validar un instrumento de evaluación de la calidad de la asistencia pre hospitalaria móvil de urgencia. El análisis fue llevado a cabo por medio de la prueba Kappa (K) e Índice de Validez de Contenido (IVC), considerando K > 0,80 e IVC ≥ 0,80.

RESULTADOS

Tres puntos fueron excluidos del instrumento: remuneración, satisfacción profesional y atención realizada. Los puntos que obtuvieron índices K e IVC óptimos y permanecieron en el instrumento fueron: estado de conservación de las ambulancias; estructura física; confort de la ambulancia; recursos materiales; seguridad del usuario/profesional; educación permanente; seguridad demostrada por el equipo; acceso; acogida; humanización; tiempo respuesta; privacidad al usuario; orientaciones acerca de la atención; relación entre el profesional y el usuario; oportunidad del usuario de hacer quejas y articulación multiprofesional.

CONCLUSIÓN

El instrumento de evaluación de la calidad de la asistencia fue validado y podrá contribuir con la evaluación de los servicios pre hospitalarios móviles de urgencia.

Enfermería de Urgencia; Atención Prehospitalaria; Estudios de Validación


OBJETIVOS

Validar um instrumento de avaliação da qualidade da assistência pré-hospitalar móvel de urgência.

MÉTODO

Estudo metodológico, com 20 profissionais que opinaram sobre os itens do instrumento proposto. A análise foi realizada por meio do teste Kappa (K) e Índice de Validade do Conteúdo (IVC), considerando K > 0,80 e IVC ≥ 0,80.

RESULTADOS

Três itens foram excluídos do instrumento: remuneração, satisfação profissional e atendimento realizado. Os itens que obtiveram índices K e IVC ótimos e permaneceram no instrumento foram: estado de conservação das ambulâncias; estrutura física; conforto da ambulância; recursos materiais; segurança do usuário/profissional; educação permanente; segurança demonstrada pela equipe; acesso; acolhimento; humanização; tempo resposta; privacidade ao usuário; orientações sobre o atendimento; relacionamento entre o profissional e usuário; oportunidade do usuário realizar reclamações e articulação multiprofissional.

CONCLUSÃO

O instrumento de avaliação da qualidade da assistência foi validado e poderá contribuir para a avaliação de serviços pré-hospitalares móveis de urgência.

Enfermagem em Emergência; Assistência Pré-Hospitalar; Estudos de Validação


Introduction

In Brazil the urgency and emergency sectors are identified as incapacitated and problematic areas within the health system, in which the decentralization, regionalization and hierarchy guidelines are insufficiently implemented(1Silva NC, Nogueira LT. Avaliação de indicadores operacionais de um serviço de atendimento móvel de urgência. Cogitare Enferm. 2012;17(3):471-7.).

In this context, the National Emergency Care Policy and consequently, the Mobile Emergency Care Services, were created in order to manage the health system, from the perspective of Pre-Hospital Care (PHC), organizing the network attention and structuring the national emergency services.

Considering that the Mobile Emergency Care Service is a regulatory complex health system that interferes with the organization of the care network and the structuring of emergency services, it is necessary to evaluate the quality of provision of these services, in order to propose solutions and improve their quality(2Brasil. Ministério da Saúde. Portaria n. 1864, de 29 de setembro de 2003. Institui o componente pré-hospitalar móvel da Política Nacional de Atenção às Urgências, por intermédio da implantação do Serviço de Atendimento Móvel de Urgências em municípios e regiões de todo o território brasileiro: SAMU 192 [Internet]. Brasília; 2003 [citado 2013 jul. 11]. Disponível em: http://dtr2001.saude.gov.br/sas/PORTARIAS/Port2003/GM/GM-1864.htm.
http://dtr2001.saude.gov.br/sas/PORTARIA...
).

Evaluating something means to give it value regarding something specific, based on a judgment, through a specific method. On the other hand, the result evaluation intends to analyze the production of services provided, or to evaluate the actions provided to the population(3Donabedian A. La dimensión internacional de la evaluación y garantía de la calidad. Salud Publ Mex. 1990;32(2):113-7.-4Bosi MLM, Pontes RJS, Vasconcelos SM. Quality dimensions in health evaluation: manager's conceptions. Rev Saúde Pública [Internet]. 2010 [cited 2012 July 5];44(2):318-24. Available from: http://www.scielo.br/pdf/rsp/v44n2/en_12.pdf
http://www.scielo.br/pdf/rsp/v44n2/en_12...
).

Regarding the assessment of health care, many services have been using evaluation methods based on the triad proposed by Donabedian: structure, process and outcome. In this approach, the structure relates to the human resources (staff), hospital buildings, financing, and equipment; the process comprises activities between patients and providers, based on standards; and the result is the final product of the care provided, in order to assess the health status and the satisfaction of standards and expectations(5Donabedian A. The quality of care: how can it be assessed? J Am Med Assoc [Internet]. 1988 [cited 2011 Nov 28];260(12):1743-8. Available from: http://post.queensu.ca/~hh11/assets/applets/The_Quality_of_Care__How_Can_it_Be_Assessed_-_Donabedian.pdf
http://post.queensu.ca/~hh11/assets/appl...
-6O'Dwyer G, Matta IEA, Pepe VLE. Avaliação dos serviços hospitalares de emergência do estado do Rio de Janeiro. Ciênc Saúde Coletiva. 2008;13(5):1637-48.).

In this study, only the evaluation of structure and process indicators were considered, since there are some difficulties in evaluating outcomes in the PHC area, as assessments should be performed by service users. The patient's stay inside the ambulance is short and it varies depending on the route taken to the health unit. Patient needs and maintenance of hemodynamic stability are priority, although patients are often unconscious and without a companion, compromising the collection of outcome data.

In Donabedian's theoretical framework, the opinion of health professionals working in the PHC mobile emergency allows for a systematic structure and process analysis of health care in these services, resulting in the assessment of quality of care from the view of these professionals(5Donabedian A. The quality of care: how can it be assessed? J Am Med Assoc [Internet]. 1988 [cited 2011 Nov 28];260(12):1743-8. Available from: http://post.queensu.ca/~hh11/assets/applets/The_Quality_of_Care__How_Can_it_Be_Assessed_-_Donabedian.pdf
http://post.queensu.ca/~hh11/assets/appl...
).

The achievement of high quality work in the emergency area is dependent on interdisciplinary action, protocol adoption, specific knowledge, technical skill, coordination between different levels of the Unified Brazilian Health System and implementation of an internal quality policy(7Silva LG, Matsuda LM, Waidman MAP. The structure of a public emergency care servisse from the workers' view: perspectives on quality. Texto Contexto Enferm [Internet]. 2012 [cited 2013 Oct 25];21(2):320-8. Available from: http://www.scielo.br/pdf/tce/v21n2/en_a09v21n2.pdf
http://www.scielo.br/pdf/tce/v21n2/en_a0...
-8Pena MM, Silva EMS, Tronchin DMR, Melleiro MM. The use of the quality model of Parasuraman Zeithaml and Berry in Health Services. Rev Esc Enferm USP [Internet]. 2013 [cited 2014 June 20];47(5):1227-32. Available from: http://www.scielo.br/pdf/reeusp/v47n5/0080-6234-reeusp-47-05-1227.pdf
http://www.scielo.br/pdf/reeusp/v47n5/00...
).

Despite advances in emergency care in Brazil, services in this area, and particularly those of a public nature, remain burdened in precarious situations with human resources facing many obstacles in their work process. Studies on the pre-hospital quality care are scarce in Brazil, and there is no valid instruments for such evaluations(9O'Dwyer GO, Oliveira SP, Seta MH. Avaliação dos serviços hospitalares de emergência do programa QualiSUS. Ciênc Saúde Coletiva. 2009;14(5):1881-90.).

The instrument of Quality Assessment on Pre-hospital Care (QA-PHC) was developed based on previous integrative review which sought to identify quality of care indicators in pre-hospital mobile emergency services. The integrative review study that formed the basis for the construction of the QA-PHC instrument is under consideration for publication in a scientific journal.

The objective of this study was to validate an instrument to evaluate the quality of mobile emergency pre-hospital care.

Method

This is a methodological study for content validation, carried out between the months of July and August 2012. Validation studies are performed to verify the quality of instruments, being the fundamental aspect for the legitimacy and credibility of search results. Content validation is the methodology that encompasses two distinct phases; the conceptual analysis that is based on the literature and the evaluation by experts or judges. This methodological approach indicates that the instrument actually reflects the purpose for which it is being used(1010 Pilatti LA, Pedroso B, Gutierrez GL. Propriedades Psicométricas de Instrumentos de Avaliação: um debate necessário. Rev Bras Ensino Ciên Tecnol [Internet]. 2010 [citado 2013 out. 23];3(1):81-91. Disponível em: http://revistas.utfpr.edu.br/pg/index.php/rbect/article/view/619/469
http://revistas.utfpr.edu.br/pg/index.ph...
).

Content validity is accomplished by analyzing the items that compose the instrument. It is the determination of representativeness and extent to which each item of the measure proves the phenomenon of interest and the dimension of each item in what it proposes to investigate, performed by expert judges in the subject being investigated. This method assesses the extent to which the selected items measure a theoretical construct representing well all the important facets of the concept to be measured(1010 Pilatti LA, Pedroso B, Gutierrez GL. Propriedades Psicométricas de Instrumentos de Avaliação: um debate necessário. Rev Bras Ensino Ciên Tecnol [Internet]. 2010 [citado 2013 out. 23];3(1):81-91. Disponível em: http://revistas.utfpr.edu.br/pg/index.php/rbect/article/view/619/469
http://revistas.utfpr.edu.br/pg/index.ph...
).

The study population was comprised of professionals working in PHC of Rio Grande do Norte, counted as 11 nurses and 24 doctors.

To calculate the sample and considering a sampling error of 10%, the following formula was used(1111 Barbetta PA. Estatística aplicada às ciências sociais. 7ª ed. Florianópolis: UFSC; 2007.):

Thus, the study sample should include 26 professionals in PHC of Rio Grande do Norte, hereinafter referred to as judges. The criteria for inclusion were: graduation in nursing or medicine, practice experience in PHC of at least one year, having published a study or a have a title (specialization, masters or doctorate) related to mobile emergency pre-hospital care services.

The selection of study participants was performed by accessibility, at the headquarters of the Mobile Emergency Care of Rio Grande do Norte. Among the 35 professionals of that service, only 20 met the inclusion criteria. They were then informed about the objectives of the study and invited to participate. All potential participants in the study agreed to participate and signed duplicates of the Informed Consent and answered the questionnaires in a quiet and private place.

The study sample consisted of 20 professionals from the PHC mobile emergency care area in Rio Grande do Norte, who acted as judges, counting 10 nurses and 10 doctors. An instrument with two parts (personal and professional characteristics) was used for data collection; and another with the 20 proposed items to assess the quality of care in the pre-hospital mobile emergency service, called the instrument of Quality Assessment on Pre-hospital Care (QA-PHC).

The QA-PHC instrument was developed based on emergency pre-hospital care quality indicators identified by studies in the area(9O'Dwyer GO, Oliveira SP, Seta MH. Avaliação dos serviços hospitalares de emergência do programa QualiSUS. Ciênc Saúde Coletiva. 2009;14(5):1881-90.,1212 Al-Shaqsi S. Current challenges in the provision of ambulance services in New Zealand. Int J Emerg Med [Internet]. 2010 [cited 2011 Jan 18];3(4):213-7. Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3047887/.
http://www.ncbi.nlm.nih.gov/pmc/articles...

13 Siriwardena AN, Shaw D, Donohoe R, Black S, Stephenson J. Development and pilot of clinical performance indicators for English Ambulance Services. Emerg Med J. 2010;27(4):327-31.

14 Stelfox HT, Bobranska-Artiuch B, Nathens A, Straus SE. A systematic review of quality indicators for evaluating pediatric trauma care. Crit Care Med [Internet]. 2010 [cited 2011 Jan 18];38(4):1187-96. Available from: http://www.ncbi.nlm.nih.gov/pubmed/20154596
http://www.ncbi.nlm.nih.gov/pubmed/20154...
-1515 El Sayed MJ. Measuring quality in emergency medical services: a review of clinical performance indicators. Emerg Med Inter [Internet]. 2012 [cited 2013 Jan 18]; 161630:1-7. Available from: http://www.hindawi.com/journals/emi/2012/161630/
http://www.hindawi.com/journals/emi/2012...
). The identification of quality indicators was carried out through an integrative literature review, guided by the following question: What are the quality healthcare indicators used in studies analyzing mobile emergency pre-hospital services?

The search of the studies took place through the following databases: LILACS (Index of Scientific And Technical Literature Of Latin America And The Caribbean) and MEDLINE (Medical Literature Analysis and Retrieval System Online), through the Virtual Health Library (VHL); Web of Science (Thomson Scientific/ISI Web Services); CINAHL (Cumulative Index to Nursing and Allied Health); PubMed; COCHRANE LIBRARY and EMBASE. Searches were also carried out in the Electronic Library SciELO (Scientific Electronic Library Online) and in the Digital Library of Theses and Dissertations, Universidade de São Paulo (USP).

The search resulted in the identification of 16 studies that met the following inclusion criteria: articles that used indicators of care evaluation in pre-hospital emergency services and articles related to the evaluation of healthcare assistance, available in Portuguese, English and/or Spanish in free full-versions published between January 2007 and June 2012 in the databases above; publications with only abstract available or those published in two or more databases were excluded.

Based on the literature search described above, it was possible to build the proposed items for the quality evaluation instrument of PHC emergency mobile services, which was then submitted to judges (Chart 1). Scale items were constructed to be answered in the form of the Likert scale, ranging from 1 to 5. The initial instructions informed the respondent that they must evaluate aspects of the structure and process of Mobile Emergency Care of Rio Grande do Norte, according to the qualification of the service, as to the following parameters: 1 = terrible, 2 = poor, 3 = regular, 4 = good, and 5 = excellent.

However, in this step of content validation the judges who participated in the study also assessed the items of the proposed instrument on relevance/permanence of each item through the following options: Agree (A) or Disagree (D), and if they agreed, they had to characterize it as an indicator of Structure (S) or Process (P).

Chart 1
Initial composition of Quality Assessment on Pre-hospital Care (QA-PHC) instrument evaluated by specialists - Natal, RN, Brazil, 2012.

The evaluation of the instrument items occurred from the agreement or disagreement of the judges about the permanency/importance of the items. In addition, judges could propose suggestions to the items. Kappa index (Κ) was used to check the level of agreement and consistency (reliability) of judges' opinions; Content Validity Index (CVI) was used to assess agreement of the judges as to the representativeness of the measure in relation to the analyzed content.

Kappa index measures the proportion of agreement and varies from "minus 1" to "plus 1," with the closer to 1 being the better the level of agreement between observers. As acceptance criteria, a match > 0.80 was established which is considered as excellent/perfect between the judges, and with a 95% confidence interval. It is noteworthy that the classification of the Kappa index ranges as follows: 0.00 - poor; 0.00 to 0.20 - weak/slight; 0.21 to 0.40 - fair; 0.41 to 0.60 - moderate; 0.61 to 0.80 - substantial; 0.81 to 0.99 - excellent; and 1.00 - perfect(1616 Pereira MG. Epidemiologia: teoria e prática. Rio de Janeiro: Guanabara Koogan; 1995.

17 Fleiss JL. Reliability of measurement. In:. Fleiss JLThe design and analysis of clinical experiments. New York: John Wiley; 1999. p. 1-31.
-1818 Perroca MG, Gaidzinki RR. Avaliando a confiabilidade interavaliadores de um instrumento para classificação de pacientes: coeficiente Kappa. Rev Esc Enferm USP. 2003;37(1):72-80.).

CVI evaluates the analysis of the judges as to the representativeness of the instrument in relation to the analyzed content, and it is calculated by dividing the number of judges who agreed with the item by the total number of judges (CVI for each item). CVI ≥ 0.80 was considered consensus for a valid indicator item to be kept in the instrument(1010 Pilatti LA, Pedroso B, Gutierrez GL. Propriedades Psicométricas de Instrumentos de Avaliação: um debate necessário. Rev Bras Ensino Ciên Tecnol [Internet]. 2010 [citado 2013 out. 23];3(1):81-91. Disponível em: http://revistas.utfpr.edu.br/pg/index.php/rbect/article/view/619/469
http://revistas.utfpr.edu.br/pg/index.ph...
).

The study followed the ethical principles contained in Resolution No. 466/12, and it was approved by the Research Ethics Committee of the Hospital Universitário Onofre Lopes, Universidade Federal do Rio Grande do Norte (HUOL/UFRN), under protocol N. 437/2010 and CAAE: 0025.0.294.051-10.

The collected data was organized in an electronic data sheet and then exported to statistical software. After coding and tabulation, data were analyzed using reflective reading and using descriptive statistics with absolute and relative frequencies, and application of the Kappa test via the Online Kappa Calculator and CVI(1919 Randolph JJ. Online Kappa calculator [Internet]. 2008 [cited 2013 Jan 22]. Available from: http://justus.randolph.name/kappa
http://justus.randolph.name/kappa...
).

Results

The presentation of results is organized as follows: personal and professional characteristics of judges in the study; followed by the levels of agreement, consistency and content validation of the proposed instrument according to the judges' opinions (Chart 2), and finally, the categorization of the instrument's items as structure or process according to the judges' evaluation (Charts 3 and 4).

The study judges were compose of 10 nurses (50.0%) and 10 physicians (50%). Most participants were 31-40 years old (60%), female (60%), with expertise in emergency care (80%), and 15% had master's degrees. As for the time of experience in pre-hospital emergency mobile services, 70% had between 5 and 9 years of experience.

Kappa and CVI values obtained for the items of the instrument for the quality of care in mobile emergency PHC are shown in Chart 2.

Chart 2
Kappa indexes and CVI for the items of the Quality Assessment on Pre-Hospital Care (QA-PHC) instrument - Natal, RN, Brazil, 2012.

According to Chart 2, most of the Kappa values and/or CVI obtained were greater than the established cut-off points, except for Professional compensation (K = 0.62), Job satisfaction (K = 0.62 ) and Service provided (K = 0.80), which had considerably substantial Kappa index but were below the limit (> 0.80) and therefore removed.

Kappa values obtained in the other items of the instrument indicate excellent agreement, consistency and validity of items. Analysis of CVI values obtained showed that all values were greater than 0.80, indicating excellent agreement among the judges as to the instrument items.

Charts 3 and 4 show the items that reached Kappa values and CVI above the regulatory limit and remained in the instrument, categorized by the judges as either related to structure or process.

Chart 3
Kappa indexes and CVI for items related to structure - Natal, RN, Brazil, 2012.

As to the items of the instrument rated by judges as Process, involving the process of care in emergency between professionals and costumers, perfect indexes were obtained in the judges' evaluation through Kappa and CVI values obtained (Chart 4). Only the item Services provided (K = 0.80) was removed, so that nine items remained in the process category, as shown on Chart 4.

Chart 4
Kappa indexes and CVI for items related to process - Natal, RN, Brazil, 2012.

Discussion

A study(2020 Ebben RHA, Vloet LCM, Schalk DMJ, Mintjes-de Groot JAJ, Achterberg T. An exploration of factors influencing ambulance and emergency nurses' protocol adherence in the Netherlands. J Emerg Nurs. 2014;40(2):124-30.) conducted with 103 mobile emergency PHC services in the Netherlands showed that among physical characteristics, mechanical maintenance of the ambulance and availability of material resources, as well as safe accommodation of these items in the internal space provide greater safety to the service process, confirming the importance of some items proposed in the instrument.

The need to evaluate structural aspects in a pre-hospital service is also reinforced by studies that claim that comfort inside the ambulances and overall physical structure are important features that should be constantly evaluated, since they influence the health stabilization dynamics of the patient who is in the ambulance(2121 Dawson S, King L, Grantham H. Review article: improving the hospital clinical handover between paramedics and emergency department staff in the deteriorating patient. Emerg Med Australa. 2013;25(5):393-405.).

The arrangement of spaces in an ambulance has a direct impact on the quality of care and patient safety. In addition, the safe storage of supplies and internal equipment affect the efficiency and the safety of providing primary care, as they may fall on the professionals, generating possible work injuries(2222 Hignett S, Crumpton E, Coleman R. Designing emergency ambulances for the 21st Century. Emerg Med J. 2009;26(2):135-40.).

In the United Kingdom, research funded by the National Patient Safety Agency and Ambulance Service Association indicated that comfort/space/layout, and patient and professional safety are some indicators that should be part of an evaluation of satisfaction research, in order to raise levels of patient safety and the quality of care provided by ambulance services that treat patients with emergency needs, thereby confirming the importance of the structural items of the proposed instrument(2222 Hignett S, Crumpton E, Coleman R. Designing emergency ambulances for the 21st Century. Emerg Med J. 2009;26(2):135-40.).

Keeping the item Safety for the professional in the instrument is related to urban violence rates in some cities. In this context, a study conducted in Brazil describes the insecurity of decentralized bases where ambulances are often found at several strategic points. Research has shown that one of the most stressful aspects of the service is patient care outside of a health facility, since working on public roads may implies risks(2323 Dantas RAN, Costa IKF, Nóbrega WG, Dantas DV, Costa IKF, Torres GV. Ocorrências realizadas pelo serviço de atendimento móvel de urgência metropolitano. Rev Enferm UFPE On line [Internet]. 2014 [citado 2014 jun. 10];8(4):842-9. Disponível em: http://www.revista.ufpe.br/revistaenfermagem/index.php/revista/article/view/4077.
http://www.revista.ufpe.br/revistaenferm...
).

There are reports that in the UK, 2-3% of incidents with PHC professionals were associated with violence and aggression, so teams should be prepared to contain violent patients and for the risk of armed robbery(2222 Hignett S, Crumpton E, Coleman R. Designing emergency ambulances for the 21st Century. Emerg Med J. 2009;26(2):135-40.).

The item Continuous education is reinforced by an American study that recommends the development of a safety culture prepared for emergencies, confirming the need of planning specific internal education centers for continuing education of professionals working in emergency services(2424 Farris D. Leveraging organizational structure in safety and emergency preparedness programs. J Chem Health Saf [Internet]. 2012 [cited 2013 Jan 19];19(4):9-14. Available from: http://www.sciencedirect.com/science/article/pii/S1871553212000266
http://www.sciencedirect.com/science/art...
).

These education centers should be responsible for the dissemination of information and updates in the emergency area, since the protocols are dynamic and are constantly reviewed, constructed and validated(2424 Farris D. Leveraging organizational structure in safety and emergency preparedness programs. J Chem Health Saf [Internet]. 2012 [cited 2013 Jan 19];19(4):9-14. Available from: http://www.sciencedirect.com/science/article/pii/S1871553212000266
http://www.sciencedirect.com/science/art...

25 Andrade MAC, Artmann E, Trindade ZA. Humanização da saúde em um serviço de emergência de um hospital público: comparação sobre representações sociais dos profissionais antes e após a capacitação. Ciênc Saúde Coletiva. 2011;16 Supl 1:1115-24.
-2626 Gerdtz MC, Daniel C, Dearie V, Prematunga R, Bamert M, Duxbury J. The outcome of a rapid training program on nurses' attitudes regarding the prevention of aggression in emergency departments: a multi-site evaluation. Int J Nurs Stud. 2013;50(11):1434-45.).

The item of Safety demonstrated by professional staff in providing care for the victim was also considered important by a Dutch study that indicated that safety of staff is directly related to the use of emergency care protocols(2020 Ebben RHA, Vloet LCM, Schalk DMJ, Mintjes-de Groot JAJ, Achterberg T. An exploration of factors influencing ambulance and emergency nurses' protocol adherence in the Netherlands. J Emerg Nurs. 2014;40(2):124-30.).

The evaluation items of structure for Job satisfaction and Professional compensation did not obtain acceptable Kappa indexes and were removed from the instrument. Although these issues are relevant, not all judges considered these items essential to assess the Mobile Emergency Care Service structure.

It is the right of every citizen to receive quality public healthcare service. To ensure this right of access, and in order to spread a new culture of humane care, the Ministry of Health launched the National Humanization of Hospital Care Program in 2000. This program proposes a set of integrated actions aimed at changing the user to the standard of care in public health services in Brazil, improving the quality and effectiveness of care provided by these institutions(2727 Barbosa GC, Meneguim S, Lima SAM, Moreno V. Política Nacional de Humanização e Formação dos Profissionais de Saúde: revisão integrativa. Rev Bras Enferm [Internet]. 2013 [citado 2013 nov. 7];66(1):123-7. Disponível em: http://www.scielo.br/pdf/reben/v66n1/v66n1a19.pdf
http://www.scielo.br/pdf/reben/v66n1/v66...
).

The same authors emphasize that in 2004, the National Humanization of Hospital Care Program was replaced by a policy that runs through different actions and management levels of the Unified Brazilian Health System, constituting a public policy of assistance and not a specific program; the National Policy of Humanization. In this perspective, the process of humanizing refers to providing access, to receiving the patients with a risk rating in emergency situations (triage), privacy and guidelines on assistance and the state of costumer health, thereby confirming the importance of several proposed items of process in the instrument(2727 Barbosa GC, Meneguim S, Lima SAM, Moreno V. Política Nacional de Humanização e Formação dos Profissionais de Saúde: revisão integrativa. Rev Bras Enferm [Internet]. 2013 [citado 2013 nov. 7];66(1):123-7. Disponível em: http://www.scielo.br/pdf/reben/v66n1/v66n1a19.pdf
http://www.scielo.br/pdf/reben/v66n1/v66...
).

The importance of evaluating the Response time is confirmed by a study conducted in France by medical emergency pre-hospital service, which reported that one way to assess the effectiveness of these services is to count response time taken for the arrival of an ambulance to the place of occurrence after a call. This measure is an important performance indicator, since this can be directly related to patient survival(2828 Aboueljinane L, Sahin E, Jemai Z, Marty J. A simulation study to improve the performance of an emergency medical service: application to the French Val-de-Marne department. Simul Model Pract Theory. 2014;47(1):46-59.).

The item of Multiprofessional articulation in an emergency mobile pre-hospital service is of paramount importance for synchronized execution of work process as a team. This indicator is recognized as a prerequisite for the quality of care(2525 Andrade MAC, Artmann E, Trindade ZA. Humanização da saúde em um serviço de emergência de um hospital público: comparação sobre representações sociais dos profissionais antes e após a capacitação. Ciênc Saúde Coletiva. 2011;16 Supl 1:1115-24.,2929 Vituri DW, Inoue KC, Bellucci Júnior JA, Oliveira CA, Rossi RM,. Matsuda LM Welcoming with risk classification in teaching hospitals: assessment of structure, process and result. Rev Latino Am Enfermagem [Internet]. 2013 [cited 2013 Nov 18];21(5):1179-87. Available from: http://www.scielo.br/pdf/rlae/v21n5/0104-1169-rlae-21-05-1179.pdf
http://www.scielo.br/pdf/rlae/v21n5/0104...
).

The Service provided item obtained Kappa index of 0.80, considered substantial or good, but was removed from the instrument as it did not reach the established minimum ratio (0.81) to be considered acceptable. The fact that the Service provided item had Kappa index below the cut-off point can be related to the generic way of describing it. It is possible that a more specific description of the item could get a satisfactory Kappa index, which should be tested in future studies. However, studies that include the Service provided as a quality indicator were not found, which confirms the fragility of this item.

The Relationship between professionals and costumers item, and Costumer opportunity to make complaints were not found in studies that discuss the quality of pre-hospital services. However, these items were kept since Kappa and CVI had acceptable rates. Using the QA-PHC tool in the evaluation of mobile urgency and emergency services will allow for future adjustments and confirmation of the proposed structure. From the standpoint of content, its structure was confirmed.

This study has some limitations that should be mentioned: the QA-PHC instrument was initially developed for health professionals, with the goal of providing clear parameters to evaluate the quality of mobile emergency pre-hospital services. The QA-PHC has not been tested for use between costumers of pre-hospital care service, but this aspect can be explored in future studies. The QA-PHC only includes the categories of structure and process, proposed in the study(3Donabedian A. La dimensión internacional de la evaluación y garantía de la calidad. Salud Publ Mex. 1990;32(2):113-7.), but the results of the evaluation is a very important step in assessing the quality and should be developed through new research.

Another limitation was the sample size, which did not reach the size recommended by the sample size calculation of 26 subjects. However, it is noteworthy that only 20 professionals of the study population met the inclusion criteria for the study, which perhaps could have been overcome with the addition of professionals from other pre-hospital care services, other states or the municipality. This limitation must be overcome in future studies.

Conclusion

It was possible to validate the instrument for assessing the quality of mobile emergency pre-hospital care (QA-PHC) by evaluating the structure and process of these services. Judges' contributions have improved/optimized the instrument. Kappa index and CVI of the items remaining in the instrument were considered excellent and the content has been validated with 17 items. Note that the content validation is an initial step in the validation process, which must be confirmed and improved in future studies with different populations and with costumers of the service.

The QA-PHC instrument can contribute to the evaluation of emergency mobile pre-hospital care services, by indicating weaknesses and allowing rearrangement of structure and process in order to improve the quality of care provided to the population.

  • 1
    Silva NC, Nogueira LT. Avaliação de indicadores operacionais de um serviço de atendimento móvel de urgência. Cogitare Enferm. 2012;17(3):471-7.
  • 2
    Brasil. Ministério da Saúde. Portaria n. 1864, de 29 de setembro de 2003. Institui o componente pré-hospitalar móvel da Política Nacional de Atenção às Urgências, por intermédio da implantação do Serviço de Atendimento Móvel de Urgências em municípios e regiões de todo o território brasileiro: SAMU 192 [Internet]. Brasília; 2003 [citado 2013 jul. 11]. Disponível em: http://dtr2001.saude.gov.br/sas/PORTARIAS/Port2003/GM/GM-1864.htm.
    » http://dtr2001.saude.gov.br/sas/PORTARIAS/Port2003/GM/GM-1864.htm
  • 3
    Donabedian A. La dimensión internacional de la evaluación y garantía de la calidad. Salud Publ Mex. 1990;32(2):113-7.
  • 4
    Bosi MLM, Pontes RJS, Vasconcelos SM. Quality dimensions in health evaluation: manager's conceptions. Rev Saúde Pública [Internet]. 2010 [cited 2012 July 5];44(2):318-24. Available from: http://www.scielo.br/pdf/rsp/v44n2/en_12.pdf
    » http://www.scielo.br/pdf/rsp/v44n2/en_12.pdf
  • 5
    Donabedian A. The quality of care: how can it be assessed? J Am Med Assoc [Internet]. 1988 [cited 2011 Nov 28];260(12):1743-8. Available from: http://post.queensu.ca/~hh11/assets/applets/The_Quality_of_Care__How_Can_it_Be_Assessed_-_Donabedian.pdf
    » http://post.queensu.ca/~hh11/assets/applets/The_Quality_of_Care__How_Can_it_Be_Assessed_-_Donabedian.pdf
  • 6
    O'Dwyer G, Matta IEA, Pepe VLE. Avaliação dos serviços hospitalares de emergência do estado do Rio de Janeiro. Ciênc Saúde Coletiva. 2008;13(5):1637-48.
  • 7
    Silva LG, Matsuda LM, Waidman MAP. The structure of a public emergency care servisse from the workers' view: perspectives on quality. Texto Contexto Enferm [Internet]. 2012 [cited 2013 Oct 25];21(2):320-8. Available from: http://www.scielo.br/pdf/tce/v21n2/en_a09v21n2.pdf
    » http://www.scielo.br/pdf/tce/v21n2/en_a09v21n2.pdf
  • 8
    Pena MM, Silva EMS, Tronchin DMR, Melleiro MM. The use of the quality model of Parasuraman Zeithaml and Berry in Health Services. Rev Esc Enferm USP [Internet]. 2013 [cited 2014 June 20];47(5):1227-32. Available from: http://www.scielo.br/pdf/reeusp/v47n5/0080-6234-reeusp-47-05-1227.pdf
    » http://www.scielo.br/pdf/reeusp/v47n5/0080-6234-reeusp-47-05-1227.pdf
  • 9
    O'Dwyer GO, Oliveira SP, Seta MH. Avaliação dos serviços hospitalares de emergência do programa QualiSUS. Ciênc Saúde Coletiva. 2009;14(5):1881-90.
  • 10
    Pilatti LA, Pedroso B, Gutierrez GL. Propriedades Psicométricas de Instrumentos de Avaliação: um debate necessário. Rev Bras Ensino Ciên Tecnol [Internet]. 2010 [citado 2013 out. 23];3(1):81-91. Disponível em: http://revistas.utfpr.edu.br/pg/index.php/rbect/article/view/619/469
    » http://revistas.utfpr.edu.br/pg/index.php/rbect/article/view/619/469
  • 11
    Barbetta PA. Estatística aplicada às ciências sociais. 7ª ed. Florianópolis: UFSC; 2007.
  • 12
    Al-Shaqsi S. Current challenges in the provision of ambulance services in New Zealand. Int J Emerg Med [Internet]. 2010 [cited 2011 Jan 18];3(4):213-7. Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3047887/.
    » http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3047887/
  • 13
    Siriwardena AN, Shaw D, Donohoe R, Black S, Stephenson J. Development and pilot of clinical performance indicators for English Ambulance Services. Emerg Med J. 2010;27(4):327-31.
  • 14
    Stelfox HT, Bobranska-Artiuch B, Nathens A, Straus SE. A systematic review of quality indicators for evaluating pediatric trauma care. Crit Care Med [Internet]. 2010 [cited 2011 Jan 18];38(4):1187-96. Available from: http://www.ncbi.nlm.nih.gov/pubmed/20154596
    » http://www.ncbi.nlm.nih.gov/pubmed/20154596
  • 15
    El Sayed MJ. Measuring quality in emergency medical services: a review of clinical performance indicators. Emerg Med Inter [Internet]. 2012 [cited 2013 Jan 18]; 161630:1-7. Available from: http://www.hindawi.com/journals/emi/2012/161630/
    » http://www.hindawi.com/journals/emi/2012/161630/
  • 16
    Pereira MG. Epidemiologia: teoria e prática. Rio de Janeiro: Guanabara Koogan; 1995.
  • 17
    Fleiss JL. Reliability of measurement. In:. Fleiss JLThe design and analysis of clinical experiments. New York: John Wiley; 1999. p. 1-31.
  • 18
    Perroca MG, Gaidzinki RR. Avaliando a confiabilidade interavaliadores de um instrumento para classificação de pacientes: coeficiente Kappa. Rev Esc Enferm USP. 2003;37(1):72-80.
  • 19
    Randolph JJ. Online Kappa calculator [Internet]. 2008 [cited 2013 Jan 22]. Available from: http://justus.randolph.name/kappa
    » http://justus.randolph.name/kappa
  • 20
    Ebben RHA, Vloet LCM, Schalk DMJ, Mintjes-de Groot JAJ, Achterberg T. An exploration of factors influencing ambulance and emergency nurses' protocol adherence in the Netherlands. J Emerg Nurs. 2014;40(2):124-30.
  • 21
    Dawson S, King L, Grantham H. Review article: improving the hospital clinical handover between paramedics and emergency department staff in the deteriorating patient. Emerg Med Australa. 2013;25(5):393-405.
  • 22
    Hignett S, Crumpton E, Coleman R. Designing emergency ambulances for the 21st Century. Emerg Med J. 2009;26(2):135-40.
  • 23
    Dantas RAN, Costa IKF, Nóbrega WG, Dantas DV, Costa IKF, Torres GV. Ocorrências realizadas pelo serviço de atendimento móvel de urgência metropolitano. Rev Enferm UFPE On line [Internet]. 2014 [citado 2014 jun. 10];8(4):842-9. Disponível em: http://www.revista.ufpe.br/revistaenfermagem/index.php/revista/article/view/4077.
    » http://www.revista.ufpe.br/revistaenfermagem/index.php/revista/article/view/4077
  • 24
    Farris D. Leveraging organizational structure in safety and emergency preparedness programs. J Chem Health Saf [Internet]. 2012 [cited 2013 Jan 19];19(4):9-14. Available from: http://www.sciencedirect.com/science/article/pii/S1871553212000266
    » http://www.sciencedirect.com/science/article/pii/S1871553212000266
  • 25
    Andrade MAC, Artmann E, Trindade ZA. Humanização da saúde em um serviço de emergência de um hospital público: comparação sobre representações sociais dos profissionais antes e após a capacitação. Ciênc Saúde Coletiva. 2011;16 Supl 1:1115-24.
  • 26
    Gerdtz MC, Daniel C, Dearie V, Prematunga R, Bamert M, Duxbury J. The outcome of a rapid training program on nurses' attitudes regarding the prevention of aggression in emergency departments: a multi-site evaluation. Int J Nurs Stud. 2013;50(11):1434-45.
  • 27
    Barbosa GC, Meneguim S, Lima SAM, Moreno V. Política Nacional de Humanização e Formação dos Profissionais de Saúde: revisão integrativa. Rev Bras Enferm [Internet]. 2013 [citado 2013 nov. 7];66(1):123-7. Disponível em: http://www.scielo.br/pdf/reben/v66n1/v66n1a19.pdf
    » http://www.scielo.br/pdf/reben/v66n1/v66n1a19.pdf
  • 28
    Aboueljinane L, Sahin E, Jemai Z, Marty J. A simulation study to improve the performance of an emergency medical service: application to the French Val-de-Marne department. Simul Model Pract Theory. 2014;47(1):46-59.
  • 29
    Vituri DW, Inoue KC, Bellucci Júnior JA, Oliveira CA, Rossi RM,. Matsuda LM Welcoming with risk classification in teaching hospitals: assessment of structure, process and result. Rev Latino Am Enfermagem [Internet]. 2013 [cited 2013 Nov 18];21(5):1179-87. Available from: http://www.scielo.br/pdf/rlae/v21n5/0104-1169-rlae-21-05-1179.pdf
    » http://www.scielo.br/pdf/rlae/v21n5/0104-1169-rlae-21-05-1179.pdf
  • (
    Extracted from the thesis "Avaliação da qualidade da assistência prestada pelo Serviço de Atendimento Móvel de Urgência do Rio Grande do Norte", Universidade Federal do Rio Grande do Norte, 2014.

Publication Dates

  • Publication in this collection
    June 2015

History

  • Received
    02 July 2014
  • Accepted
    10 Nov 2014
Universidade de São Paulo, Escola de Enfermagem Av. Dr. Enéas de Carvalho Aguiar, 419 , 05403-000 São Paulo - SP/ Brasil, Tel./Fax: (55 11) 3061-7553, - São Paulo - SP - Brazil
E-mail: reeusp@usp.br