AGREEMENT OF AN INSTITUTIONAL RISK CLASSIFICATION ASSESSMENT PROTOCOL¹

ABSTRACT

Objective:

to assess the agreement among nurses in the application of an institutional assessment protocol with risk classification in a hospital emergency unit.

Method:

quantitative and retrospective study of cross-sectional approach, carried out at the adult emergency unit of a general hospital in the State of Santa Catarina, based on medical records. The data collection took place between October and December of 2013. The sample consisted of 380 medical records, selected by simple randomization. The degree of agreement was determined by the Kappa coefficient.

Results:

the general agreement among nurses in the application of the institutional protocol was substantial (Kappa=0.786) representing 331 (87.2%) files. Among the declassification levels, there was almost a perfect agreement for all of them (Kappa=1 for the red level, Kappa=0.836 for the orange level, Kappa=0.884 for the yellow level, Kappa=0.865 for the green level and Kappa=1 for the blue level).

Conclusion:

the general agreement in the application of the institutional protocol was substantial, showing an almost perfect agreement at all levels of classification. There was an overestimation and underestimation of the risk classification in relation to the protocol. It is suggested that the nurses who perform the risk classification receive training from the institution regarding the applicability of the protocol, in order to minimize the overestimation and underestimation of the risk classification. This study contributes to assess the application of a new implanted protocol, with subsidies for a better agreement in its implementation by nurses.

DESCRIPTORS:
User embracement; Evaluation studies; Emergency nursing; Health services research; Nursing

RESUMO

Objetivo:

avaliar a concordância entre os enfermeiros na aplicação de um protocolo institucional de avaliação com classificação de risco em uma unidade de emergência hospitalar.

Método:

estudo quantitativo de abordagem transversal e retrospectivo, realizado na unidade de emergência adulto de um hospital geral do Estado de Santa Catarina, baseado em fichas de atendimento. A coleta de dados ocorreu entre outubro e dezembro de 2013. A amostra fora constituída de 380 fichas de atendimento, selecionadas por aleatorização simples. O grau de concordância se determinou pelo coeficiente Kappa.

Resultados:

a concordância geral entre os enfermeiros na aplicação do protocolo institucional foi substancial (Kappa=0,786) representando 331 (87,2%) fichas. Entre os níveis de classificação houve concordância quase perfeita para todos eles (Kappa=1 para o nível vermelho, Kappa=0,836 para o nível laranja, Kappa=0,884 para o nível amarelo, Kappa=0,865 para o nível verde e Kappa=1 para o nível azul).

Conclusão:

a concordância geral na aplicação do protocolo institucional se mostrou substancial, sendo evidenciada uma concordância quase perfeita em todos os níveis de classificação. Houve superestimação e subestimação da classificação de risco em relação ao protocolo. Sugere-se que os enfermeiros que realizam a classificação de risco recebam treinamento por parte da instituição quanto à aplicabilidade do protocolo, a fim de minimizar a superestimação e subestimação da classificação de risco. Este trabalho contribui ao avaliar a aplicação de um protocolo novo implantado, com subsídios para uma melhor concordância na sua implementação por enfermeiros.

DESCRITORES:
Acolhimento; Estudos de avaliação; Enfermagem em emergência; Pesquisa sobre serviços de saúde; Enfermagem

RESUMEN

Objetivo:

evaluar la concordancia entre los enfermeros en la aplicación de un protocolo institucional de evaluación con clasificación de riesgo en una unidad de emergencia hospitalaria.

Método:

estudio cuantitativo de abordaje transversal y retrospectivo realizado en la unidad de emergencia para adultos de un Hospital General del Estado de Santa Catarina y basado en fichas de atendimiento. La obtención de datos ocurrió entre Octubre y Diciembre del 2013. La muestra fue constituida por 380 fichas de atendimiento seleccionadas por aleatorización simple. El grado de concordancia se determinó por el coeficiente Kappa.

Resultados:

la concordancia general entre los enfermeros en la aplicación del protocolo institucional fue substancial (Kappa=0,786) representando 331 fichas (87,2%). Entre los niveles de clasificación hubo una concordancia casi perfecta para todos ellos (Kappa=1 para el nivel rojo, Kappa=0,836 para el nivel naranja, Kappa=0,884 para el nivel amarillo, Kappa=0,865 para el nivel verde y Kappa=1 para el nivel azul).

Conclusión:

la concordancia general en la aplicación del protocolo institucional se mostró substancial y quedó en evidencia una concordancia casi perfecta en todos los niveles de clasificación. Hubo sobreestimación y subestimación de la clasificación de riesgo en relación al protocolo. Se sugiere que los enfermeros que realizan la clasificación de riesgo reciban entrenamiento por parte de la institución sobre la aplicabilidad del protocolo con el objetivo de minimizar la sobreestimación y subestimación de la clasificación de riesgo. Este trabajo contribuye para evaluar la aplicación de un nuevo protocolo implantado, con subsidios para una mejor concordancia en su implementación por los enfermeros.

DESCRIPTORES:
Acogimiento; Estudios de evaluación; Enfermería en emergencia; Investigación sobre servicios de salud; Enfermería

INTRODUCTION

The waiting time for care in emergency units is a worldwide problem and has negative impact, especially in high-risk patients who require immediate intervention.¹ A systematic review has shown that the overcrowding of emergency rooms is a condition that influences the delay of care, with implications for the patient's safety and quality of the care provided. The analysis has showed a positive correlation between overcrowding and waiting time with increased mortality, in addition to verifying that many patients leave the service without even being seen by a health professional.²

In Brazil, this problem is evidenced in many emergency services, in which there are long lines and people disputing the care, whose only criterion is the time of arrival. The non-distinction of the risks and the lack of prioritization of care can lead certain cases to worsen in the waiting room, causing death due to the absence of care at the appropriate time.³

In an attempt to address the challenges related to the organization of emergency services and to reduce the waiting time, the actions of the Ministry of Health (MS - Ministério da Saúde) have been focused on the reorganization of the work process, in order to meet the different degrees of specificity of patients with acute aggravations.⁴

From this perspective, the country has now the Risk Classification Embracement (ACCR - Acolhimento com Classificação de Risco), whose proposal is to operate the work processes in health in order to provide care for all those who look for the public health services, and to equalize the care.^3,5

The equalization of the ACCR intends to optimize the waiting time according to the severity of the clinical condition of the users. The MS recommends that the risk classification follows a guiding protocol and that it is a private task of the nurse.⁶

There are internationally several Risk Classification protocols, among which the following stand out: Emergency Severity Index (ESI); Australasian Triage Scale (ATS); Canadian Triage and Acuity Scale (CTAS); Manchester Triage System (MTS); Andorran screening model and Spanish screening system (MAT-SET - Modelo Andorrano de triagem e sistema espanhol de triagem).⁷ In Brazil, these protocols are also used, however, the MS offers the health services the possibility of developing their own protocols based on those already existing, but adapted to the local profile and the context of their insertion in the health network.³

Given this alternative, many emergency services in Brazil have been using institutional protocols for the ACCR. Among the pioneers, the Municipal Hospital of Paulínia and the Municipal Hospital Dr. Mario Gatti stand out, using the Canadian model adapted to the local realities.⁸

The implementation of adapted protocols makes it essential to assess the agreement of professionals regarding them, in order to guarantee the patient's safety. However, national studies on risk classification still poorly assess the nurses' agreement.⁹

Thus, considering that the ACCR protocols are guidelines for the assessment of the severity of the users, and that the agreement between the assessment of professionals and the institutional protocol are essential to ensure adequate response time of health interventions as well as the safety of these users, aspect that is already consolidated in the international protocols,¹⁰ this study aims at evaluating the agreement among nurses in the application of an institutional assessment protocol with risk classification in a hospital emergency unit.

METHOD

Quantitative and retrospective study of cross-sectional approach, carried out at the adult emergency unit of a general hospital in the State of Santa Catarina. In this service, the ACCR protocol, instituted in 2011, was developed by medical professionals and nurses based on the MTS.

The aforementioned protocol is composed of 42 distinct flowcharts and a list of clinical signs (known as discriminators), which guide the nurses' decision-making process. Based on the main symptom presented by the patient, the professional must choose one of the flowcharts to proceed with the assessment. The classification in one of the five clinical priority levels is defined for each patient through the selected flowchart.

The five levels and their respective target times for care are: Priority degree I: emergency, red color; target time: 0 minute; Priority degree II: very urgent, orange color; target time: 10 minutes; Priority degree III: urgent, yellow color; target time: 2 hours; Priority degree IV: not very urgent, green color; target time: 6 hours; Priority degree V: not urgent, blue color; target time: 8 hours. The 42 flowcharts available in the protocol are: asthma, self-harm, headache, strange behavior, convulsions, foreign body, diabetes, diarrhea, dyspnea, hematological disease, mental disease, sexually transmitted diseases, abdominal pain, neck pain, sore throat, lower back pain, testicular pain, chest pain, apparent intoxication, rash, unconsciousness, exposure to chemical agents, wounds, pregnancy, digestive bleeding, vaginal bleeding, malaise in adults, local infections and abscesses, bites and stings, dental problems, extremity problems, nasal problems, ear problems, ophthalmic problems, urinary problems, falls, burns, anal complaints, overdosage and poisoning, traumatic brain injury, vertigo and dizziness, vomiting.

The population was composed of 27,292 medical records, provided by the statistical care bulletin of the adult emergency unit, which correspond to all the care provided between August 1, 2011 and August 31, 2012, a temporal cut-off chosen to contemplate the first year after the implantation of the institution's ACCR protocol, which had not been assessed until this study was carried out.

The sample calculation was performed by SEstatNet11 website and adopted a 95% confidence interval. The sample consisted of 380 medical records and the sampling was randomized through a simple draw. As inclusion criteria, the patients' medical records that passed the risk classification and which contained the description of the assessment and the classification attributed by the nurse have been considered. The medical records that were incomplete, without risk classification, and those used in the pre-test of the data collection instrument were excluded.

For the data collection, a check-list type instrument was elaborated with the following information: classification of the nurse; flowchart chosen by the nurse; reclassification flowchart; and, reclassification. In order to identify the need for adjustments in the instrument, a pre-test was performed with 20 medical records, chosen at random and excluded later.

The data collection took place from October to December of 2013. The reclassification was performed by the researchers using the same flowchart chosen by the nurse at the time of the embracement. According to the items described in the flowchart and existing records of complaints, it was possible to identify if the classification (obtainment of the priority level), established at the time of embracement, corresponded to that obtained in the reclassification, or if the flowchart used to define the level of priority was consistent with the complaint.

The statistical software R Core Team version 3.0.1 was used for treatment of the data. The descriptive analysis was carried out and the degree of general agreement was obtained, as well as the levels of risk classification of the nurse and the reclassification obtained by the study.

The degree of agreement between two or more independent observers or coders regarding the scores of an instrument is one of its reliability measures. The agreement index is calculated to assess the strength of the relation between the classifications. Thus, when the scores given by two independent observers to the same phenomenon are congruent, it means that these scores are probably accurate and reliable.¹²

The Kappa coefficient assesses the degree of agreement beyond what would be expected only by chance. This measure of agreement is graded on a scale of values where 1 means total agreement and values close to or below 0 indicate no agreement. For this study, it was considered Kappa <0 no agreement; 0-0.20, poor; 0.21-0.40, weak; 0.41-0.60, regular; 0.61-0.80, strong; 0.81-1.00, almost perfect.¹³

The study was approved by the Research Ethics Committee of the Universidade Federal de Santa Catarina, protocol No.355.461/13 and CAAE No. 15741513.9.0000.0121, and it complies with the Resolution No. 466/12 of the National Health Council.

RESULTS

A total of 380 medical records were assessed, which were completed by the 17 nurses who performed the ACCR in the emergency unit. These professionals have an average age of 32 years old and an average service time of four years unit. As for complementary training: two nurses are Ph.D. in nursing; seven have master's degree in nursing; six have specialization; and, two have the nursing graduation.

Of the 380 medical records assessed, those classified as yellow (n=191, 50.3%) and green (n=156, 41.1%) prevail, which correspond to ACCR Priority Degrees III and IV, respectively. Regarding the agreement among the nurses in the classification and reclassification of the records, there was almost perfect agreement in all degrees of priority, being highlighted the degrees I and V (Kappa=1; p<0.001), which represent the colors red and blue, in sequence.

It was identified that the greatest disagreement (Kappa=0,865; p<0,001) among nurses occurred in the priority degree IV (green color), corresponding to 26 (6.8%) records (Table 1), reclassified to priorities III (n=5; 1.3%) and V (n=21; 5.5%). Of the other reclassified records (n=23; 6.1%), 21 (5.5%) classified in Degree III (yellow color) moved to the priorities II (n=6; 1.6%) and IV (n=15; 3.9%), and 02 (0.5%) of Degree II (orange color) were reclassified to priority III.

Table 1
Agreement among nurses in the classification and reclassification of risk in a hospital emergency unit. Florianópolis, SC, Brazil, 2013. (n=380)

Priority degree	Color	Classified medical records		Classification with disagreement		Kappa	p* Value
		n	%	n	%
I	Red	2	0.5	-	-	1	<0.001
II	Orange	29	7.6	2	0.5	0.836	<0.001
III	Yellow	191	50.3	21	5.5	0.884	<0.001
IV	Green	156	41.1	26	6.8	0.865	<0.001
V	Blue	2	0.5	-	-	1	<0.001
Total		380	100.0	49	12.8	0.786	<0.05

• ^*
  Kappa Test.

Regarding the general agreement among the nurses in the classification and reclassification of risk, it should be highlighted that of the 380 (100%) medical records assessed, 331 (87.2%) presented agreement (Kappa=0.786; p<0.05) (Table 1), being considered substantial according to the Kappa value.

DISCUSSION

Based on the characterization data of the nurses, the qualification of these professionals was evidenced. It should be emphasized that nurses are not required to have any specialization as a prerequisite to act in risk classification and this fact may be related to the functional career plan that encompasses additional monetary to salary with incentive to undertake postgraduate and courses. Another study¹⁴ has found the same participants' profile in an institution that is similar to the one investigated.

In this study, it should be highlighted that the risk classifications of the medical records were classified as Priority Degree III, yellow, followed by the Priority Degree IV, green, that is, case of lower severity from an emergency point of view. This may be justified since it is a hospital in a context in which healthcare networks still lack the structuring to absorb these less immediate demands. A similar result was identified in a study in Saudi Arabia, being most cases classified as urgent and less urgent, which correspond to levels III and IV of the CTAS.¹⁵

Thus, the constant search for emergency services for cases of lesser severity may indicate obstacles in the use of the healthcare network and/or the vulnerability of people who need care repeatedly.¹⁶ The lack of knowledge of the population regarding the provision of health services or the inadequate use of these services can make the classification of risk ineffective in some aspects, since the care provided to users classified as less serious becomes superficial and inadequate.¹⁷

Regarding the agreement, there was a substantial agreement in the assessment with risk classification carried out by nurses and it was almost perfect, as analyzed among the different levels of priority. This outcome may be related, in part, to the nurses' ability to make decisions and clinical experience, attributes that help to obtain the correct level of priority. This study corroborates that classification is a complex activity that depends on the skills and competences of nurses.⁹ The insertion of nurses during and after risk classification requires choices and attitudes that reinforce their professional autonomy and personal abilities, not limited to the registration and classification of the user through discriminators.^18-19

The results referring to agreement resemble those found and proved in other studies regarding different protocols. In a study carried out with 69 nurses in four Swiss hospitals, it was possible to observe that the agreement among nurses who used the American screening protocol (ESI) was good, despite the low agreement evidenced.²⁰ In a study developed in Australia with the objective of assessing the agreement through a protocol of classification of risk, it was obtained a degree of agreement that varied from regular to almost perfect.²¹

In Brazil, a similar research revealed that nurses' agreement on the assignment of priority levels regarding the institutional protocol was regular.⁹ Another study that verified the agreement between an institutional protocol and the MTS obtained agreement ranging from regular to substantial between the protocols.⁸

Regarding the nurses' agreement on the different levels of priority, there was complete agreement at the opposite levels (Grade I, red color; and Grade V, blue color), which can be attributed to the clarity in identifying these situational extremes of care (emergency versus non-urgent) and to the nurse's easy interpretation of the discriminators of the institutional protocol. Differently, the implementation of the Canadian protocol in Saudi Arabia showed that the highest agreement among nurses was assigned in Grade III of the screening scale.¹⁵

Regarding the disagreements, the majority were in classifications of lower complexity, that is, patients should have been classified at lower levels of priority, which characterizes a super classification of risk.

The super classification phenomenon observed in another study showed that there is a greater probability of disagreements in the classification between neighboring colors than between opposing colors.⁸ This overestimation of risk, also known as overtriage, may be associated to the fact that the professional identifies slowness in the delay for care, and assigns a higher level of priority so the cases can be attended in a timely manner and the flow of care can be improved. It should be highlighted that the overestimation of the risk determines the emergency services charge due to an increase in the demand for care.⁹ In addition, it entails unnecessary bed occupancy, generating material expenses, workers' amounts, financial losses, exposure of the user to unnecessary procedures, and consequent worsening of the quality of care provided.

The super classification is corroborated by the literature dealing with the different screening systems known worldwide. Studies involving the MTS also identified a high rate of super classification, ranging from 7.6 to 54%. It shows that many patients tend to be unnecessarily screened with higher priority and few patients screened with low urgency, producing a significant impact on the waiting time of the patients in the emergency after screening, with a probable delay for those classified as more urgent.²² Regarding the ATS, another study showed super classification in the patients' screening, but with a less representative percentage (20.7%).²³

In the present study, although less frequent, cases of underestimation of the complaint were also identified, in which users should be classified into higher levels of care, a phenomenon known as undertriage. These are considered potentially serious errors, whereby patients may be adversely affected by the postponement to the beginning of healthcare while awaiting erroneously, or that they may even die while waiting for a long time.

The percentage of underestimation identified in this investigation is close to the ones of other studies. In a MTS research, the undertriage rate ranged from 11 to 25%, with patients in urgent conditions being classified as low urgency, results that have raised concern regarding the application of the protocol as a screening tool for emergency conditions and that indicate the relevance of new studies to confirm the findings.²² Another analysis showed that the percentage of undertriage in Degrees I and II of ATS was 18.5%, relevant data since it is endangering the life of the critical patient.²³

It should be highlighted that the sub-classification of the priority level can lead to loss and increase in the time spent on patient care. An incompletely assessed complaint may underestimate the risk involved and its safety.²⁴ A study comparing sub-classification and super classification rates after the adequacy of an ACCR protocol showed that the super-classification rates improved from 53% to 38%, and sub-classification rates improved from 47% to 16%, respectively.²⁵

Thus, the search for reliable classification instruments is important to guarantee nurses' support in their decision-making. One factor that must be considered for the success of determining the risk priority is to follow the recommendations of the guiding protocol in full.⁸

It should be mentioned that this study presents as limitations the use of only the records to determine the agreement and the difference of two years between the time of the clinical assessment and the registration, which could bring different results if they were consecutive or simultaneous. It is also pointed out the need for other statistical tests to complement the agreement. The temporal clipping of the data regarding the dynamism of the emergency sectors also makes it impossible to generalize the findings. However, it is believed that the present work of assessing the application of a new protocol implemented, contributes with subsidies for a better agreement in its implementation by nurses.

CONCLUSION

The agreement among the nurses in the application of an institutional protocol of assessment with risk classification proved to be substantial, being evidenced at all levels of priority an almost perfect agreement, with emphasis on the Degrees I and V, despite the difference of two years between the assessment moments. Although professional experience and specific training are not required to act in the risk classification, it is suggested that nurses who perform the ACCR receive training from the institution regarding the applicability of the protocol, in order to minimize the overestimation and the underestimation of the risk classification identified in the study.

• 1
  Manuscript extracted from dissertation - Accuracy of risk classification by nurses from a hospital emergency unit, presented to the (Multidisciplinary Professional Master’s Program in Health). Universidade Federal de Santa Catarina (UFSC), in 2013.

REFERÊNCIAS

• ¹
  Guttmann A, Schull MJ, Vermeulen MJ, Stukel TA. Association between waiting times and short term mortality and hospital admission after departure from emergency department: population based cohort study from Ontario, Canada. BMJ [Internet]. 2011 [cited 2016 Aug 16]; 342:d2983. Available from: http://www.bmj.com/content/bmj/342/bmj.d2983.full.pdf
  » http://www.bmj.com/content/bmj/342/bmj.d2983.full.pdf
• ²
  Carter EJ, Pouch SM, Larson EL. The relationship between Emergency Department Crowding and patient outcomes: a systematic review. J Nurs Scholarsh [Internet]. 2014 [cited 2017 Jun 03]; 46(2):106-15. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4033834/
  » https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4033834/
• ³
  Ministério da Saúde (BR), Secretaria de Atenção à Saúde, Política Nacional de Humanização da Atenção e Gestão do SUS. Acolhimento e classificação de risco nos serviços de urgência. Brasília (DF): MS; 2009.
• ⁴
  Souza CC, Araújo FA, Chianca TCM. Scientific literature on the reliability and validity of the Manchester Triage System (MTS) protocol: a integrative literature review. Rev Esc Enferm USP [Internet]. 2015 [cited 2016 Aug 8]; 49(1):144-51. Available from: http://www.scielo.br/pdf/reeusp/v49n1/0080-6234-reeusp-49-01-0144.pdf
  » http://www.scielo.br/pdf/reeusp/v49n1/0080-6234-reeusp-49-01-0144.pdf
• ⁵
  Costa PCP, Garcia APRF, Toledo VP. Welcoming and nursing care: a phenomenological study. Texto Contexto Enferm [Internet]. 2016 [cited 2016 Set 20]; 25(1):e4550015. Available from: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0104-07072016000100324&lng=pt
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0104-07072016000100324&lng=pt
• ⁶
  Conselho Federal de Enfermagem (COFEN). Resolução n. 423 de 9 de abril de 2012: Normatiza, no âmbito do Sistema Cofen/Conselhos Regionais de Enfermagem a participação do Enfermeiro na atividade de classificação de risco [Internet]. Brasília (DF): COFEN; 2012 [cited 2016 Mar 10]. Available from: http://www.cofen.gov.br/wpcontent/uploads/2012/03/Res_423_2012_pag1.pdf
  » http://www.cofen.gov.br/wpcontent/uploads/2012/03/Res_423_2012_pag1.pdf
• ⁷
  Ministério da Saúde (BR), Agência Nacional de Saúde Suplementar. Acolhimento com classificação de risco [Internet]. Brasília (DF): MS; 2012 [cited 2016 Aug 8]. Available from: http://www.ans.gov.br/images/stories/prestadores/E-ACE-01.pdf
  » http://www.ans.gov.br/images/stories/prestadores/E-ACE-01.pdf
• ⁸
  Souza CC, Toledo AD, Tadeu LFR, Chianca TCM. Risk classification in an emergency room: agreement level between a Brazilian institutional and the Manchester Protocol. Rev Latino-am Enfermagem [Internet]. 2011 [cited 2016 May 28]; 19(1):26-33. Available from: http://www.scielo.br/pdf/rlae/v19n1/05.pdf
  » http://www.scielo.br/pdf/rlae/v19n1/05.pdf
• ⁹
  Oliveira GN, Vancini-Campanharo CR, Okuno MFP, Batista REA. Nursing care based on risk assessment and classification: agreement between nurses and the institutional protocol. Rev Latino-am Enfermagem [Internet]. 2013 [cited 2016 Jul 15]; 21(2):500-6. Available from: http://www.scielo.br/pdf/rlae/v21n2/0104-1169-rlae-21-02-0500.pdf
  » http://www.scielo.br/pdf/rlae/v21n2/0104-1169-rlae-21-02-0500.pdf
• ¹⁰
  Pinto Júnior D, Salgado PO, Chianca TCM. Predictive validity of the Manchester Triage System: evaluation of outcomes of patients admitted to an emergency department. Rev Latino-am Enfermagem [Internet]. 2012 [cited 2016 Aug 10]; 20(6):1041-7. Available from: http://www.scielo.br/pdf/rlae/v20n6/05.pdf
  » http://www.scielo.br/pdf/rlae/v20n6/05.pdf
• ¹¹
  Nassar SM, Wronscki VR, Ohira M. SEstatNet - Sistema Especialista para o Ensino de Estatística na Web [Internet] [cited 2016 Aug 10]. Available from: http://sestatnet.ufsc.br
  » http://sestatnet.ufsc.br
• ¹²
  Polit DF, Beck CT. Fundamentos de pesquisa em enfermagem: avaliação de evidências para a prática da enfermagem. 7 ed. Porto Alegre (RS): Artmed; 2011.
• ¹³
  McHugh ML. Interrater reliability: the kappa statistic. Biochemia Medica [Internet]. 2012 [cited 2016 Aug 10]; 22(3):276-82. Available from: http://dx.doi.org/10.11613/BM.2012.031
  » http://dx.doi.org/10.11613/BM.2012.031
• ¹⁴
  Inoue KC, Bellucci JJA, Papa MAF, Vidor RC, Matsuda LM. Evaluation of quality of risk classification in emergency services. Acta Paul Enferm [Internet]. 2015 [cited 2017 Jun 07]; 28(5):420-5. Available from: http://dx.doi.org/10.1590/1982-0194201500071
  » http://dx.doi.org/10.1590/1982-0194201500071
• ¹⁵
  Alquraini M, Awad E, Hijazi R. Reliability of Canadian emergency department Triage and Acuity Scale (CTAS) in Saudi Arabia. Int J Emerg Med [Internet]. 2015 Dec [cited 2016 Mar 15]; 8(1):80. Available from: http://www.ncbi.nlm.nih.gov/pubmed/26251308
  » http://www.ncbi.nlm.nih.gov/pubmed/26251308
• ¹⁶
  Acosta AM, Lima MADS. Frequent users of emergency services: associated factors and reasons for seeking care. Rev Latino-am Enfermagem [Internet]. 2015 [cited 2016 Aug 9]; 23(2):337-44. Available from: http://www.scielo.br/pdf/rlae/v23n2/0104-1169-rlae-23-02-00337.pdf
  » http://www.scielo.br/pdf/rlae/v23n2/0104-1169-rlae-23-02-00337.pdf
• ¹⁷
  Nascimento ERP, Hilsendeger BR, Neth C, Belaver GM, Bertoncello KCG. Classificação de risco na emergência: avaliação da equipe de enfermagem. Rev Enferm UERJ [Internet]. 2011 [cited 2015 Jul 15]; 19(1):84-8. Available from: http://www.facenf.uerj.br/v19n1/v19n1a14.pdf
  » http://www.facenf.uerj.br/v19n1/v19n1a14.pdf
• ¹⁸
  Chaves CS, Santos DA, Teixeira Silva LL, Ferreira MLR, Chianca, TCM. Nurses' perception about risk classification in emergency services. Invest Educ Enferm [Internet]. 2014 [cited 2017 Jun 07]; 32(1):78-86. Available from: http://www.scielo.org.co/scielo.php?script=sci_arttext&pid=S0120-53072014000100009&lng=en&tlng=pt
  » http://www.scielo.org.co/scielo.php?script=sci_arttext&pid=S0120-53072014000100009&lng=en&tlng=pt
• ¹⁹
  Moreira DA, Tibães HBB, Batista RCR, Cardoso CML, Brito MJM. Manchester Triage System in primary health care: ambiguities and challenges related to access. Texto Contexto Enferm [Internet]. 2017 [cited 2017 Jun 12]; 26(2):e5970015. Available from: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0104-07072017000200302&lng=pt
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0104-07072017000200302&lng=pt
• ²⁰
  Jordi K, Grossmann F, Gaddis GM, Cignacco E, Denhaerynck K, Schwendimann R, et al. Nurses' accuracy and self-perceived ability using the Emergency Severity Index triage tool: a cross-sectional study in four Swiss hospitals Scand J Trauma Resusc Emerg Med [Internet]. 2015 [cited 2016 Aug 15]; 23:62. Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4551516/pdf/13049_2015_Article_142.pdf
  » http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4551516/pdf/13049_2015_Article_142.pdf
• ²¹
  Grouse AI, Bishop RO, Bannon AM. The Manchester Triage System provides good reliability in an Australian emergency department. Emerg Med J [Internet]. 2009 [cited 2016 Aug 15]; 26(7):484-6. Available from: http://www.ncbi.nlm.nih.gov/pubmed/19546267
  » http://www.ncbi.nlm.nih.gov/pubmed/19546267
• ²²
  Parenti N, Reggiani MLB, Iannone P, Percudani D, Dowding D. A systematic review on the validity and reliability of an emergency department triage scale, the Manchester Triage System. Int J Nurs Stud [Internet]. 2014 Jul [cited 2016 Aug 8]; 51(7):1062-99. Available from: http://www.ncbi.nlm.nih.gov/pubmed/24613653
  » http://www.ncbi.nlm.nih.gov/pubmed/24613653
• ²³
  Ebrahimi M, Heydari A, Mazlom R, Mirhaghi A. The reliability of the Australasian Triage Scale: a meta-analysis. World J Emerg Med [Internet]. 2015 [cited 2016 Aug 18]; 6(2):94-9. Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4458479/pdf/WJEM-6-94.pdf
  » http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4458479/pdf/WJEM-6-94.pdf
• ²⁴
  Souza CC, Chianca LM, Diniz AS, Chianca TCM. Main complaints of emergency patients according to the Manchester protocol for risk classification. Rev Enferm UFPE On line [Internet]. 2012 [cited 2016 Aug 9]; 6(3):540-8. Available from: http://www.revista.ufpe.br/revistaenfermagem/index.php/revista/article/view/2207/pdf_101
  » http://www.revista.ufpe.br/revistaenfermagem/index.php/revista/article/view/2207/pdf_101
• ²⁵
  Mullan PC, Torrey SB, Chandra A, Caruso N, Kestler A. Reduced overtriage and undertriage with a new triage system in an urban accident and emergency department in Botswana: a cohort study. Emerg Med J [Internet]. 2014 [cited 2017 Jun 10]; 31(5):356-60. Available from: https://www.ncbi.nlm.nih.gov/pubmed/23407375
  » https://www.ncbi.nlm.nih.gov/pubmed/23407375

Publication Dates

• Publication in this collection
  2018

History

• Received
  11 Oct 2016
• Accepted
  22 Aug 2017