Care protocols with risk classification in pediatrics: inter-observer reliability

Abstract

Objective

To analyze the inter-observer reliability of the second edition of the Care protocols with risk classification in pediatrics, in the clinical practice of urgent/emergency care.

Method

Methodological study, developed through assessment of the inter-observer reliability of two Trained Inter-observers Triage Nurses (TITN) and two Untrained ITNs (UITN) with the researcher, with a total of 400 classifications. Data was analyzed using the Kappa Coefficient, with a 95% Confidence Interval (CI).

Results

A total of 80.0% of patients were classified as standard(green-50.7%) and non-urgent (blue-29.3%); substantial and “almost perfect” agreement between the nurses and the researcher (Kappa between 0.62 and 1.0) was found. “Almost perfect” agreement was found between TITN-2 and the researcher (Kappa 1.0), UITN-2 and the researcher (Kappa 0.877) and TITN-1 and the researcher (Kappa 0.725). Substantial agreement was found between UITN-1 and the researcher (Kappa 0.619).

Conclusion

Care protocols with risk classification in pediatrics is a reliable technology to guide nurses in the triage of patients in pediatric emergency/urgent care situations.

Reception; Pediatric; Risk assesment/classification; Validation studies; Child

Resumo

Objetivo

Analisar a confiabilidade interobservadores da segunda edição do protocolo de Acolhimento com Classificação de Risco em Pediatria, na prática clínica de urgência/emergência.

Métodos

Estudo metodológico, desenvolvido por meio da confiabilidade interobservadores de dois Enfermeiros Classificadores Interobservadores Treinados (ECIT) e de dois ECI Não Treinados (ECINT) com a pesquisadora, totalizando 400 classificações. Dados analisados utilizando-se do Coeficiente Kappa, com Intervalo de Confiança (IC) de 95%.

Resultados

Predomínio de 80,0% de pacientes classificados como menor urgência (verde-50,7%) e não urgente (azul-29,3%); revelou-se substancial “quase perfeita” concordância entre os enfermeiros e a pesquisadora (Kappa entre 0,62 e 1,0). Houve “quase perfeita” concordância entre ECIT-2 e pesquisadora (Kappa 1,0), ECINT-2 e pesquisadora (Kappa 0,887) e ECIT-1 e pesquisadora (Kappa 0,725). Houve substancial concordância entre ECINT-1 e pesquisadora (Kappa 0,619).

Conclusão

O Protocolo de ACCR em Pediatria é uma tecnologia confiável para direcionar enfermeiros à classificação de risco em situações de urgência/emergência pediátrica.

Acolhimento; Pediatria; Avaliação de risco/classificação; Estudos de validação; Criança

Introduction

Clinical judgment and decision-making are considered broad and significant challenges involved in the different situations in the healthcare practice. Among these, urgent/emergency situations stand out. In these situations, it is necessary to prioritize the information received and facilitating conditions in order to determine the severity of the conditions and the risk of death of patients visiting healthcare institutions.

In these situations, the healthcare professionals working in urgent/emergency units, especially the nurses, need knowledge, ability and attitude to properly care, listen and guide the healthcare users in these units¹and to triage patients arriving at the hospital.

Experts in the field²clarify that the term risk classification indicates the act of identifying the main issue manifested or reported by patients and/or their companions, in order to establish a waiting list based not on order of arrival, but on clinical risk. Therefore, the triage process consists in conducting the collection of anamnesis and the physical examination of the signs and symptoms, aiming at determining the priority of care related to the conditions expressed, the possible complications and the risk of death.

To perform the risk classification, health institutions suggest and use health technologies, such as admission protocols to guide the Care with Risk Classification.

A review carried out to identify the health technology instruments used in the triage/risk classification of children or adolescents indicated nine instruments for this purpose: Pediatric Assessment Triangle, Canadian Pediatric Triage and Acuity Scale and its computerized version - Staturg, Emergency Triage Assessment and Treatment, Emergency severity index for pediatric triage, Pediatric Taiwan Triage System, Pediatric Triage and Acuity System, Manchester Triage System for pediatric emergency care and Pediatric Triage Tape.³

In addition to the findings of this review, the Care protocols with risk classification in pediatrics stands out as another triage instrument. This protocol assists in the characterization, evaluation and classification of the risk presented by the child and/or adolescent, to determine the priority of treatment in the emergency/urgent care service. This instrument, developed by the Health Department of Fortaleza-Ceará-Brazil, uses five colors to classify the risk and determine the expected waiting time for medical care: red (priority I), immediate medical attention; orange (medical care in up to 15 minutes; can be reassessed by the nurse every 15 minutes); yellow (medical care in up to 30 minutes; can be reassessed by the nurse every 30 minutes); green (medical care in up to 60 minutes, can be reassessed by the nurse every 60 minutes); and blue (patient will receive medical care in order of arrival or will be directed to the primary care unit). This instrument was evaluated by experts specialized in pediatrics and triage, and its content and its appearance were considered valid.⁴

Based on the study mentioned, the need to assess the inter-observer reliability and the validity of the protocol in the clinical practice was evidenced. An instrument is considered reliable when it is able to accurately measure a phenomenon and to produce results that are consistent in time, space and with different observers, if properly used.⁵

This study is justified and relevant since it presented the evaluation of the reliability of this instrument in the clinical practice, assessing its adequacy for the decision-making process and for adequate and safe determination of the priority of care of children and/or adolescents in emergency/urgent care services. The study also made it possible to use the instrument to provide a quality care and improve the risk classification process in the admission of the targeted population in a timely and effective manner, in order to prevent and minimize complications and risk of death at the emergency/urgent health service.

Therefore, the objective of this study was to assess the inter-observer reliability of the second edition of the Pediatric Risk Classification in clinical practice for emergency/urgent care.

Methods

Methodological study, with a quantitative approach, aimed at evaluating the Care protocols with risk classification in pediatrics as a reliable instrument that can be used by health professionals in pediatric emergency/urgent care units.

The study was conducted in the emergency unit of a municipal hospital reference in pediatric care in the city of Fortaleza-CE-Brazil, where there are children and adolescents in high risk for complications.

For the selection of the children and/or adolescents, the inclusion criteria were: being from zero to 16 years old, seeking care in the emergency/urgent care unit of the institution assessed and being with a companion. The exclusion criterion was: people who were seeking elective care with a physician or nurse (pediatrics).

For sample size calculation, the recommendations of the Kendall’s coefficient of concordance, used in attribute agreement analysis, were followed. This coefficient also indicated the degree of correlation of ordinal rankings carried out by different evaluators when analyzing the same samples.

The Kendall’s coefficient determined a number of at least 96 children and/or adolescents as sample size for each inter-observer nurse. However, a total sample of 100 children and/or adolescents for each Trained Inter-observer Triage Nurse (TITN-1 and 2) and Untrained Inter-observer Triage Nurse (UITN-1 and 2), that means, each nurse, along with the researcher, evaluated and classified 100 children and/or adolescents independently. Therefore, the final sample consisted of 400 children and/or adolescents in the risk classification in pediatrics.

Data collection was conducted with the inter-observer nurses and the researcher. The reliability and degree of correlation of the risk classification were determined by TITN-1 and TITN-2 and the researcher, and UITN-1 and UITN-2 and the researcher.

The inclusion criteria for the UITN were: not having participated in the training course on the Pediatric Risk Classification Protocol, having at least two years of experience and to be working in the institution’s risk classification during the period of data collection. The exclusion criteria were: being on vacation, maternity leave or sick leave during the period of data collection. After the selection, four untrained nurses willing to contribute were selected; however, during the collection, there was one withdrawal and one sick leave. Thus, only two UITN were assessed, and it was agreed with each of them that the researcher would collect the data according to the schedule pre-established by the Institution’s nursing coordination.

For the TITNs, inclusion criteria were: participating and having more than 80% frequency in the 40-hour training course on the Pediatric Risk Classification Protocol and in the discussion group about improving the applicability of the protocol. The exclusion criteria were: being on vacation, maternity leave or sick leave in the period of data collection and being a temporary professional in the period of data collection.

The data collection began with the UITN-1 evaluating the children and/or adolescents and determining the priority of care of each one. The researcher observed, captured the information described by the nurse and followed the priority of care recommended by the second edition of the Pediatric Risk Classification Protocol according to the clinical situation of each patient. This process was repeated with the 100 children evaluated by the researcher and by each of the evaluator nurses, following the sequence: UITN-1, UITN-2, TITN-1 and TITN-2.

A total of 400 classifications were attained for the assessment of the inter-observer reliability. The data collection lasted about eight months and the approximate time took by the triage nurse with each patient was five minutes, varying depending on the complexity and severity of the main complaint reported by the child, adolescent or companion. The data was collected in day and night shifts, according to the schedule and availability of the triage nurses. The period of data collection was extended due to the seasonality of common diseases in pediatric emergency/urgent care units.

The data collection instruments were: 1) Characterization of the triage nurses on age, level of education, time of experience in pediatrics, mean time of experience in triage, participation in the Training Course on the Pediatric Risk Classification Protocol, use of the Pediatric Risk Classification Protocol and difficulty in using the Pediatric Risk Classification Protocol; 2) Triage instrument containing data regarding identification and classification of the children and/or adolescents, such as age (in months), gender, level of education, vital signs, anthropometric data, main complaint, time spent in the evaluation, waiting time for medical care, clinical discriminators (main clinical conditions related to the main complaint), pain scale and determination of risk classification based on the five colors of the protocol.

In addition, the researcher obtained individualized data by approaching the guardians of the children and/or adolescents who sought care. The objectives of the study were presented and the signing of the consent form was requested for these participants, respecting ethical and legal aspects.

To analyze the reliability of the second edition of the Pediatric Risk Classification Protocol, the kappa coefficient was used. This coefficient measures the level of agreement, the consistency of the evaluations or the relative position of the evaluations of the data by the evaluators/judges. This index assesses the capacity that an instrument has to reach the same results when applied by two or more observers, giving the magnitude of the inter-observer reliability as result.

According to scholars, there are six classifications for the interpretation of the Kappa test, with the following agreement values: No agreement (0); Slight agreement (> 0 to 0.19); Fair agreement(0.20 to 0.39); Moderate agreement (0.40 to 0.59); Substantial agreement (0.60 to 0.79); and “Almost perfect” agreement or Excellent (0.80 to 1).

Kappa values greater than 0.60 - substantial agreement - are recommended to obtain reliable results. If the values are lower than 0.60, the level of agreement between the judges or the adequacy of the instrument should be questioned.

The study complied with the principles of Resolution 466/12 of the National Health Council/Ministry of Health, and was approved for development under the number 1.282.924 and CAAE nº 49076915.7.0000.5054.

Results

Regarding the characterization of the participants, four were nurses, three were female and one male. The mean age of the nurses was 26 years and five months. All of them had graduate certification degree and two participated in research groups and had scientific publications related to the subject Child Health. All had professional experience in this area and also in emergency/urgent care, with a mean experience of four years.

The comparison between the groups showed that the TITNs had less time of clinical experience in pediatrics (at least one year of professional experience), whereas the UITNs had a mean of five years of experience. Participating nurses reported that they rarely used the Protocol studied and that they had no difficulty using the second edition of the Pediatric Risk Classification Protocol, classifying it as excellent.

Among the children and/or adolescents evaluated, 51.2% (205) were male; the mean age was 50.2 months (approximately four years) with a median of 26.5 months (two years) and standard deviation of 40.3 months (three years).

Regarding the risk classification determined by the inter-observer nurses and by the researcher, equivalence between these was evidenced by the agreement between the priorities of care established by the nurses in the pediatric emergency/urgent care practice, as shown in table 1.

Table 1
Distribution of the children according to the risk classification assessed through the Pediatric Risk Classification Protocol, conducted by Inter-observer Triage Nurses and by the researcher

Triage classification	Inter-observer triage nurse	Researcher
	n(%)	n(%)
Orange - Priority II	1(0.3)	9(2.3)
Yellow - Priority III	79(19.8)	80(20.0)
Green - Priority IV	203(50.7)	205(51.2)
Blue - Priority V	117(29.3)	106(26.5)
Total	400(100.0)	400(100.0)

The data in table 1 shows a 50.7% (203) predominance of patients classified as green - priority IV (standard) and a 29.3 % (117) predominance of blue - priority V (non-urgent) given by the TITNs, totaling a sample of 80%. These results were similar to the classification given by the researcher, who classified 77.7% of the children as green and blue.

Considering the comparison of the classification given by the TITNs and the UITNs with the researcher classification, the importance of evaluating the degree of correlation of the Pediatric Risk Classification Protocol regarding its legitimacy in contrasting groups in the Pediatric emergency/urgent care practice is evident.

It should be noted that there were no children or adolescents in this study who were considered immediate, that is, none of the participating nurses nor the researcher attributed the red classification (priority I), which means the patient is at immediate risk of complications or death.

Regarding the orange risk classification (priority II), none of the patients evaluated by the UITN group and only one patient evaluated by the TITN group was classified as requiring medical attention within 15 minutes. The researcher, however, identified 4.5% (9) of the children and/or adolescents who sought very urgent care as orange priority.

Regarding the other classification colors, most of, many of the patients (41%) were classified as green (priority IV) and 39% were classified as blue (priority V) by the TITNs. The same was predominant in the UITN group, which attributed green priority to 49.5% and blue priority to 29.5% of the evaluated patients. For the researcher, there were 93% standard care patients (green) and 61% (122) non-urgent patients (blue).

The data determine the patient profile in the institution studied, which were mostly classified with standard care (green) and non-urgent (blue). Regarding the comparison between the risk classification attributed by the group of trained inter-observer nurses (T1 and T2) and by the researcher in the inter-observer reliability phase, the data found is described in table 2.

Table 2
Risk classification attributed by the TITN and by the researcher in the inter-observer reliability phase

Classification by the researcher		Classification by the TITN								Total	Kappa	p-value
		Orange		Yellow		Green		Blue
		T1	T2	T1	T2	T1	T2	T1	T2
Orange	T1	1	-	1	-	1	-	0	-	3
	T2	-	0	-	0	-	0	-	0	0	T1 0.726	T1 0.060
Yellow	T1	0	-	12	-	6	-	2	-	20
	T2	-	0	-	22	-	0	-	0	22	T2 1.000	T2 0.0001
Green	T1	0	-	3	-	45	-	3	-	51
	T2	-	-	-	0	-	34	-	0	34
Blue	T1	0	-	0	-	1	-	25	-	26
	T2	-	-	-	0	-	0	-	44	44
Total	T1	1	-	16	-	53	-	30	-	100
	T2	-	-	-	22	-	34	-	44	100

An “almost-perfect” agreement was found between UITN-2 and the researcher (Kappa = 1; p < 0.0001). Between UITN-1 and the researcher, substantial agreement was obtained (Kappa = 0.726; p = 0.060). Table 3 displays the data regarding the agreement of the risk classifications attributed by the group of non-trained inter-observer nurses (U1 and U2) and by the researcher in the inter-observer reliability phase.

Table 3
Risk classification attributed by the UITN and the researcher in the inter-observer reliability phase

Classification by the Researcher		Classification by the UITN						Total	Kappa	p-value
		Yellow		Green		Blue
		U1	U2	U1	U2	U1	U2
Orange	NT1	1	-	1	-	0	-	2
	NT2	-	1	-	1	-	0	2	NT1 0.619	NT1 0.067
Yellow	NT1	15	-	4	-	3	-	22	NT2 0.887	NT2 0.410
	NT2	-	18	-	1	-	0	19
Green	NT1	5	-	40	-	7	-	52
	NT2	-	2	-	48	-	1	51
Blue	NT1	0	-	3	-	21	-	24
	NT2	-	0	-	1	-	27	28
Total	NT1	21	-	48	-	31	-	100
	NT2	-	21	-	51	-	28	100

Table 3 shows results that demonstrate a substantial agreement (Kappa = 0.619; p = 0.067) between the UITN-1 and the researcher. The UITN-2 had an “almost perfect” agreement with the researcher (Kappa = 0.887; p = 0.410). However, this index was not statistically significant when compared to trained nurses.

It is possible to perceive that the data presented higher percentages in the diagonals of table 3. Considering that the agreement between the observers, higher agreement was found for standard classification (green), with very few differences in agreement for non-urgent classifications (blue). In other words, it was verified that there was no significant difference between the means of the total scores given by the trained nurses and the untrained nurses and the scores given by the researcher. Therefore, “almost perfect” agreement was found with the TITNs (Kappa = 0.863 e p = 0.030), whereas substantial agreement was found with the UITNs (Kappa = 0.753 and p = 0.355).

The results can indicate that the protocol is reliable, simple and self-explanatory to determine the risk classification of children and/or adolescents in emergency/urgent care situation. This means that the protocol, whether used by a trained or untrained nurse, is reliable in producing the same results on repeated evaluations, being a consistent and predictable tool for determining priority of care.

However, it is important to note that differences of agreement were found between the inter-observer nurses and the researcher, but this was not statistically significant. However, this finding in the risk classification may demonstrate a need for greater sensitivity for the clinical complaint in order to determine the discriminators and the respective clinical indicators in the situation of the child and/or adolescent.

In this aspect, data showed that some patients were classified by the researcher with a higher priority of care, while the nurses indicated a lower level of care priority compared to the protocol. This may be due to the direction of the main clinical complaint and can indicate a situation of underestimation of the priority of care. This demonstrates the importance of using the protocol as a health technology for determining the priority of care in a reliable and safe manner for children and/or adolescents in emergency/urgent care situation.

Discussion

For the legal practice of the profession, the nurse who performs the triage must have in-depth knowledge about what is an urgent/emergency care situation and an elective situation, in order to assess the clinical signs with skill and direction and make the decision about which service should be indicated to provide adequate care, thus reducing the subjectivity bias on the emergency/urgent service.

Age was identified as an important socio-demographic characteristic, since the prevalence of infants and preschool children was verified. This data can be compared to a pediatric emergency study in a teaching hospital in Mato Grosso do Sul, Brazil, in which out of the 1236 patients attended, 988 (79.9%) were infants and preschool children. Therefore, it is known that this age group may be more predisposed to diseases and can require more attention from the health team.⁶

However, another study assessed 2,153 medical records of children and adolescents attending an emergency unit in Alagoas and found patients between 10 and 18 years as the most representative age group, with 931 (43.2%) patients.⁷

The most urgent situations (red or orange) present in pediatric urgent/emergency units include cardiorespiratory arrest, severe sepsis, convulsions, among others, which require immediate attention, indicated with the activation of a sound alarm. The results are in agreement with other studies, which also indicated that these occurrences were not very frequent in the pediatric services.⁶,⁸

The present study identified that more than half of the children and adolescents analyzed were classified as standard (green), indicating that many of the hospital emergency units remain as a gateway for the Unified Health System. This result also shows that parents or guardians of children and/or adolescents often go to these units instead of going to primary health care units.⁶The additional number of patients in emergencies also occurs because users consider these sectors as more accessible, since they offer greater resources, such as consultations, laboratory exams, medications, image exams, among others. The search for these services in an erroneous way compromises the functioning of the emergency units and this is mainly due to lack of orientation of the population and insufficient structuring in the basic health care networks.⁹

Regarding the inter-observer reliability, the results agree with international studies that discuss the degree of agreement between nurses using Triage Protocols to determine the priority of care in emergency/urgent situations. These studies also include the validity and the reliability of the protocols in Pediatrics, and present instruments that contain reliable information to define care protocols that improve and broaden the practice of nurses.¹⁰-¹⁴

The data are similar to another study¹⁵which found “almost perfect” agreement (Kappa = 0.884) between the complaint given at admission and the discharge medical diagnosis, indicating that the Canadian scale evaluated in this study represented a new instrument to be used in emergency services.

Another study¹⁶ also obtained similar results, finding substantial or “almost perfect” degrees of agreement between the four observer nurses, with Kappa values ranging from 0.68 to 0.90. This demonstrated that the classification instrument presents reliability to be used in the nurse’s managerial practice as a diagnostic tool to determine the category of care to which the patient belongs.

However, it is important to emphasize the importance of continued formal education for professionals and undergraduate students, aiming to learn and practice the use of this technology in order to improve performance and reliability in the pediatric triage, a need that has also been highlighted by national and international authors.¹⁵-¹⁷

This also shows the importance of the triage for the care implementation, since it provides greater accessibility to emergency services, prioritizes the most serious cases, and is operative when the situation demands. It promotes a lower injury to users’ health, since it facilitates the classification and orientation of the flow of patients, prioritizing the most serious patients in relation to the least severe ones. Therefore, the triage protocols promote a better organization of emergency services, which have as main objective guaranteeing the Unified Health System (SUS) principles: universality, resoluteness and humanization of the care provided.⁸,¹⁸,¹⁹

As limitation of the study, the nurses demonstrated difficulty in valuing and adhering to the training course on the use of the Pediatric Risk Classification Protocol as a health technology. Regarding the clinical profile of the children and adolescents attended at the studied institution, there was no case of an emergency with red classification, needing immediate medical care and requiring the activation of the alarm, on days and times of data collection. The difficulty of involving health managers in the implementation of the Pediatric Risk Classification Protocol in the pediatric emergency was also a limitation, since a reduction of human resources was often found, with only one nurse on the 12-hour scale and lack of adequate material to carry out the physical examination of children and/or adolescents in emergency situation.

The present research brought as contribution to the clinical practice the Pediatric Risk Classification Protocol as a reliable health technology, since its use can configure an appropriate and safe strategy, providing the triage nurse with adequate direction to determine the priority of care of children and adolescents in pediatric emergency situations.

Conclusion

The Pediatric Risk Classification Protocol for care, developed in Fortaleza-Ceará-Brazil, was identified as reliable to classify the priority of care of children and adolescents in an emergency/urgent situation, based on the degree of suffering and risk of complications and death. The use of this instrument in the waiting lines in pediatric emergencies contributed for quick and effective decision making in the care of these health users. Regarding the inter-observer reliability of the Pediatric Risk Classification Protocol, equivalence was evidenced through the degree of agreement of trained nurses and the researcher, as well as untrained nurses and the researcher, during the evaluation of children and/or adolescents in situation of emergency.

Therefore, the protocol studied was established as reliable tool for the triage of children and adolescents. It can be used by any nurse, whether trained on not on its use, to determine the priority of care. Therefore, it is possible to infer that the Protocol is a reliable technology to determine the priority of care of children and/or adolescents in an emergency situation, regardless of training.

Acknowledgments

To the funding granted by the National Council for Scientific and Technological Development (CNPq) and the Coordination for the Improvement of Higher Education Personnel (CAPES).

Referências

• ¹
  Weykamp JM, Pickersgill CS, Cecagno D, Vieira FP, Siqueira HC. Welcom in gwith risk classification in urgent and emergency services: applicability in nursing. Rev Rene. 2015; 16(3):327-36.
• ²
  Magalhães-Barbosa MC, Prata-Barbosa A, Cunha AJ, Lopes CS. CLARIPED: a new tool for risk classification in pediatric emergencies. Rev Paul Pediatr (English Edition). 2016; 34(3):254-62.
• ³
  Felipe GF, Silva VM, Cardoso MV, Lima FE, Magalhães FJ. Produção científica sobre triagem e acolhimento com classificação de risco em pediatria. In: Seminário Nacional de Pesquisa em Enfermagem, 18º., 2015, Fortaleza. Anais. Fortaleza: ABEn; 2015.
• ⁴
  Magalhães FJ. Validação do protocolo de acolhimento com classificação de risco em pediatria [dissertação]. Fortaleza (CE): Universidade Federal do Ceará, Faculdade de Farmácia, Odontologia e Enfermagem; 2012.
• ⁵
  Silveira MF, Almeida JC, Freire RS, Haikal DA, Martins AE. Propriedades psicrométricas do instrumento de avaliação da qualidade de vida: 12-item health survey (SF-12). Ciênc Saúde Coletiva. 2013; 18(7):1923-31.
• ⁶
  Salgado RM, Aguero FC. Perfil dos pacientes pediátricos atendidos na emergência de um hospital universitário. Pediatria (São Paulo). 2010; 32(2):90-7.
• ⁷
  Simons DA, Monlleó IL, Simons AS, Araújo Junior JL. Adequação da demanda de crianças e adolescentes atendidos na unidade de emergência em Maceió, Alagoas, Brasil. Rev Bras Saúde Mater Infant. 2010; 10(1):59-67.
• ⁸
  Veras JE, Rodrigues AP, Silva MJ, Aquino PS, Ximenes LB. Competências de enfermeiras para a promoção da saúde durante atendimentos pediátricos em unidade de emergência. Acta Paul Enferm. 2015; 28(5):467-74.
• ⁹
  Oliveira DA, Guimarães JP. A importância do acolhimento com classificação de risco nos serviços de emergência. Cad Saúde Desenvol. 2013; 2(2):25-44.
• ¹⁰
  Jafari-Rouh AH, Sardashti S, Taghizadieh A, Soleimanpour H, Barzegar M. The Emergency Severity Index, version 4, for pediatric triage: a reliability study in Tabriz Children’s Hospital, Tabriz, Iran. Int J Emerg Med. 2013; 6(36):1-5.
• ¹¹
  Zachariasse JM, Seiger N, Rood PP, Alves CF, Freitas P, Smit FJ, Roukema GR, Moll HA. Validity of the Manchester Triage System in emergency care: A prospective observational study. PlosOne. 2017; 2:1-14.
• ¹²
  Azeredo TR, Guedes HM, Almeida RA, Chianca TC, Martins JC. Efficacy of the Manchester Triage System: a systematic review. Int Emerg Nurs. 2015; 23(2):47-52.
• ¹³
  Parenti N, Reggiani ML, Iannone P, Percudani D, Dowding D. A systematic review on the validity and reliability of na emergency department triage scale, the Manchester Triage System. Int J Nurs Studies. 2014; 51(7):1062-9.
• ¹⁴
  Santos MA. Acolhimento com classificação de risco: um fio guia da administração em emergência. Rev Saúde Desenvol. 2014; 6(3):56-69.
• ¹⁵
  Pires PS. Tradução para o português e validação de instrumento para triagem de pacientes em serviço de emergência: “Canadian Triage and Acuity Scale” (CTAS) [tese]. São Paulo: Universidade de São Paulo, Escola de Enfermagem da USP; 2003.
• ¹⁶
  Perroca MG, Gaidzinski RR. Avaliando a confiabilidade intervaliadores de um instrumento para classificação de pacientes - coeficiente Kappa. Rev Esc Enferm USP. 2003; 37(1): 72-80.
• ¹⁷
  Sieger N, Van Veen M, Almeida H, Steyerberg EW, Van Meurs AH, Carneiro R, et al. Improving the Manchester Triage System for Pediatric Emergency Care: An International Multicenter Study. PlosOne. 2014; 9(1):83267.
• ¹⁸
  Inoue KC, Bellucci JJ, Papa MA, Vidor RC, Matsuda LM. Evaluation of quality of Risk Classification in Emergency Services. Acta Paul Enferm. 2015; 28(5):420-5.
• ¹⁹
  Souza CC, Araújo FA, Chianca TC. Produção científica sobre a validade e confiabilidade do Protocolo de Manchester: revisão integrativa da literatura. Rev Esc Enferm USP. 2015; 49(1):144-51.

Publication Dates

• Publication in this collection
  May-Jun 2017

History

• Received
  7 Mar 2017
• Accepted
  5 June 2017