Scientific Literature on the Reliability and Validity of the Manchester Triage System (MTS) Protocol: A Integrative Literature Review

Producción científica acerca de la validez y confiabilidad del protocolo de manchester: revisión integradora de la literatura

Abstracts

OBJECTIVE

To analyze the scientific production about the validity and reliability of the Manchester Triage System (MTS) protocol.

METHOD

A descriptive study of an integrative literature review. Articles on the validity and reliability of the MTS developed with children and adults published between 1999 and 2013 were included.

RESULTS

14 articles were selected from a total of 8438, nine of validity and five of reliability. The reliability of the MTS ranged from moderate to almost perfect, with higher intra-evaluation. Regarding validity, the results seem to point to equivalent and satisfactory sensibility and specificity levels of the MTS. The instrument proved to be a good predictor of the need for hospitalization and of hospital mortality.

CONCLUSION

The reliability and validity of the MTS obtained in the studies is varied. It is recommended that new studies indicate necessary modifications to the MTS so that it is more safely used by nurses.

Triage; Emergency Medical Services; Emergency Nursing; Validity of Tests; Reproducibility of Results; Review

OBJETIVO

Analizar las producciones científicas acerca de la validez y confiabilidad del Protocolo de Manchester (MTS).

MÉTODO

Estudio descriptivo de revisión integradora de la literatura. Fueron incluidos artículos acerca de la validez y confiabilidad del MTS, desarrollados con niños y adultos, publicados entre 1999 y 2013.

RESULTADOS

Se seleccionaron 14 artículos de un total de 8.438, siendo nueve sobre validez y cinco sobre confiabilidad. La confiabilidad del MTS varió de moderada a casi perfecta, siendo mayor intraevaluadores. En cuanto a la validez, los resultados parecen apuntar hacia niveles equivalentes y satisfactorios de sensibilidad y especificidad del MTS. El instrumento se mostró un buen pronosticador de la necesidad de hospitalización y de mortalidad hospitalaria.

CONCLUSIÓN

La confiabilidad y validez del MTS logradas en los estudios son variadas. Se recomiendan nuevos estudios que apunten las modificaciones necesarias en el MTS a fin de que los enfermeros lo utilicen con más seguridad.

Triaje; Servicios Médicos de Urgencia; Enfermería de Urgencia; Validez de las Pruebas; Reproducibilidad de Resultados; Revisión

OBJETIVO

Analisar as produções científicas acerca da validade e confiabilidade do Protocolo de Manchester (MTS).

MÉTODO

Estudo descritivo de revisão integrativa da literatura. Foram incluídos artigos sobre validade e confiabilidade do MTS, desenvolvidos com crianças e adultos, publicados entre 1999 a 2013.

RESULTADOS

Selecionou-se 14 artigos de um total de 8.438, sendo nove sobre validade e cinco sobre confiabilidade. A confiabilidade do MTS variou de moderada a quase perfeita, sendo maior intra-avaliadores. Quanto à validade, os resultados parecem apontar para níveis equivalentes e satisfatórios de sensibilidade e especificidade do MTS. O instrumento mostrou-se bom preditor da necessidade de internação e de mortalidade hospitalar.

CONCLUSÃO

A confiabilidade e validade do MTS obtidas nos estudos são variadas. Recomenda-se novos estudos que indiquem as modificações necessárias no MTS para que o mesmo seja utilizado com maior segurança pelos enfermeiros.

Introduction

Emergency services are an important component of health care in Brazil and have a greater demand than the capacity can attend. Overcrowding is the picture of the imbalance between supply and demand for care in emergency services, being aggravated by organizational problems such as service in order of arrival and no establishment of clinical criteria, which can cause serious harm to patients(¹).

In an attempt to address challenges related to the organization of emergency services, the actions of the Ministry of Health have been facing the need to reorganize the work process in order to meet the varying specificity and resolution in assistance provided to patients with acute illnesses. Thus, in 2004 the reception with a risk classification was appointed as the device of change in the work of attention, management and production in health(²).

The risk classification is a dynamic process of identifying patients who require immediate treatment according to the potential risk of harm to the patient's health or the degree of pain(²). Its importance is to prevent complications and to identify acute cases with a significant risk of death for individuals(³).

The risk classification must be performed by a nursing professional with a degree, preferably with experience in the emergency department and after specific training for the proposed activity(²).

The assignment of a degree of risk to the patient is a complex process of decision making and many rating scales have been developed to help guide the nurse's evaluation(⁴). These scales aim to optimize the waiting time according to the severity of the clinical condition of the patients, in order to quickly treat the most severe symptoms and reduce negative impacts on prognosis, due to delay in treatment(⁵). It has been recommended to use scales that stratify risk into five priority levels, as they presented greater reliability and validity in assessing the patient(⁶).

The standardization process of reception with a risk classification has been sought through the adoption of measures that standardize the nurse's evaluation. The Brazilian Ministry of Health recommends that the risk assessment should follow a protocol director(²). Among the existing triage scales, the Manchester Triage Protocol (MTS) has been adopted in most emergency services as a director of risk classification instrument, currently being used in 16 (61.5%) of the 26 Brazilian states and the Federal district(⁷).

The screening/triage scale of MTS classifies patients into five priority levels: level 1 (emergency, should receive immediate medical attention); level 2 (very urgent, medical evaluation needed within 10 minutes); level 3 (urgent, medical evaluation within 60 minutes); Level 4 (a little urgent, medical evaluation within 120 minutes); level 5 (not urgent, and can wait 240 minutes for medical care). Thus, the MTS was established as a management tool to manage clinical risk of administrating medical attention waiting time, prioritizing the most severe patients(⁸).

As it is a new theme/subject in Brazil, there are few studies aimed at evaluating the validity and reliability of protocol drivers to determine the degree of risk for patients in emergency departments, especially the MTS. It is noteworthy that the MTS is a protocol of English origin and there are no studies that address its translation and validation for use in Brazil.

For studies designed to test measurement instruments, reliability is considered the main criterion for the investigation of their quality. It consists of the ability of an instrument to consistently and accurately measure what is intended. Validity refers to the consistency of the results to be obtained by employing a particular method of investigation. It can be divided into: internal validity (the ability of the instrument to measure what it intends to measure) and external validity (generalizability of research findings to other locations or samples)(⁹).

Thus, this study aimed to analyze scientific works about the reliability and validity of the Manchester Triage System protocol. To have knowledge of studies that investigate the validity and reliability of the MTS is important for professional nursing practice, since the use of the protocol is the legal and scientific support in which the nurse has her clinical decision-making action rooted to.

Method

This is a descriptive study of a comprehensive literature review(¹⁰). This is the broadest review of research methods by allowing simultaneous inclusion of experimental and non-experimental studies to fully understand the phenomenon under study. Furthermore, it allows the combination of theoretical and empirical literature data. For conducting the research, the subsequent steps were followed: defining the research question and objectives of the integrative review, establishing sample selection criteria, defining the information to be extracted from selected articles, analysis of results, discussion and presentation of the results and presentation of the review(¹⁰).

This study was guided by the following question: What are the publications in the national and international literature about the validity and reliability of the Manchester Triage System protocol for determining the priority of patients who seek emergency care services?

The literature survey was conducted online using the databases of Latin American Literature and Caribbean Health Sciences (LILACS), PubMed, and the Cochrane Library. Searching was also conducted in the Scientific Electronic Library Online (SciELO) collection.

The national / domestic search was conducted through the use of controlled descriptors contained in Descriptors of Health Science (MeSH) of the Virtual Health Library (VHL): "triage" "Nursing," "emergency medical services," "validity of tests" and "reproducibility of results." For international database searches, the same descriptors were used in English. AND was used for the Boolean operator combination of descriptors (Chart 1).

Chart 1
Systematic electronic search in LILACS, PubMed, Cochrane and SciELO

The criteria for sample selection were: articles published in national and international literature whose focus has been the study of the reliability and validity of the Manchester Protocol, conducted with children and/or adults, published between the years 1999-2013.

Below is the flow chart of the selection process of articles in the study sample (Figure 1).

Figure 1
Flow chart of the selection process for the literature found on validity and reliability of the MTS, published between 1999 and 2013, in LILACS, PubMed, Cochrane and SciELO databases.

For the third step (Data Extraction) a form was developed in order to facilitate the analysis and characterization of the sample articles. It contained the following information: identification of the article and authors, source location, objectives, study design and level of evidence, results, key findings and recommendations for practice(¹⁰). Data were analyzed according to the contents of the articles using descriptive statistics. It is noteworthy that the levels of evidence ranged from I to VII (¹¹) being: Level 1 - meta-analyzes or systematic reviews; Level II - Controlled Randomized Clinical Trial; Level III - Clinical trial without randomization; Level IV - Cohort studies and case-control; Level V - Systematic review of descriptive studies; Level VI - Descriptive studies; Level VII - Expert opinion.

Results

Of the 14 articles selected, one (7.1%) was published in 2012, four (28.6%) in 2011, four (28.6%) in 2009, two (14.3%) in 2010, two (14.3%) in 2008 and one (7.1%) in 2006. The data show an increasing production of validity and reliability of the Manchester Protocol in the past five years (from 11 to 78.6%).

Regarding the origin of the studies, only two (14.3%) were performed in Brazil, seven (50.0%) in the Netherlands, three (21.5%) in Portugal, one (7.1%) in Sweden and one (7.1%) in Australia. This finding points to an incipient production in Brazil in conducting research that aims to test the validity and reliability of the Manchester Protocol, although this is the protocol used in most Brazilian states to diagnose the clinical risk of patients seeking emergency services(⁷).

With regards the origin of the journal in which the article was published, the majority (9 - 64.3%) were published in medical journals, three (21.4%) in nursing journals (two in a Brazilian journal - Latin American Journal of Nursing), and two (14.3%) in pediatrics journals.

Below is a summary of articles about the reliability and validity of the Manchester Protocol published in the last 15 years (1999-2013) in the databases consulted (Chart 2 and 3).

Chart 2
Results of the search for articles about the reliability of the MTS, published between 1999 and 2013, in LILACS, PubMed, Cochrane and SciELO databases

It is noted that of the five identified articles that dealt with the reliability of the MTS, two (40%) were performed in the Netherlands(¹³-¹⁴), one (20%) in Brazil(¹²), one (20%) in Sweden(¹⁵) and one (20%) in Australia(¹⁶). Four (80%) studies were conducted with adults (¹²,¹⁴-¹⁶) and one (20%) with a pediatric population(¹³).

Chart 3
Results of the search for articles about the validity of the MTS, published between 1999 and 2013 in LILACS, PubMed, Cochrane and Scielo databases

Of the nine articles on validity of MTS, five (55.5%) were performed in the Netherlands(¹⁸-¹⁹,²²,²⁴-²⁵), three (33.3%) in Portugal(²⁰-²¹,²³) and one (11.2%) in Brazil(¹⁷). Four (44.5%) studies were conducted with adult populations(¹⁷,²⁰-²¹,⁷), three (33.3%) in the pediatric population(¹⁹,²⁴-²⁵) and two (22.2%) with adult and pediatric populations(¹⁸,²²).

Discussion

The search for reliable measurement instruments is important to ensure security in the risk classification decision-making of nurses. However, the construction of clear flows/streams of service organization to ensure successful risk classification is required, guided by the priority established for each patient and the complexity level of each station that makes up the network of health care(²⁶).

Note the fact that the majority (11 to 78.6%) of the studies were of evidence level VI, which corresponds to descriptive studies. Only 3 (21.4%) were cohort studies, classified as level IV evidence(¹¹). These findings point to the need for new research to generate relevant evidence in order to contribute to the decision making of nurses in triage. In short, future systematic reviews and meta-analyzes of validity and reliability of the MTS may be made from data from studies conducted in several countries, with similar methodological designs. We emphasize that the knowledge levels of the evidence from the studies is important to assist nurses in selecting the best practices to be incorporated into care.

The reliability of a measuring instrument is mainly measured by the reproducibility of test results(⁹). That is, the ability of the instrument to repeatedly measure the same results, evaluated under the same conditions. The calculation of the Kappa index is the most commonly performed to assess the reliability of an instrument, and measures the intra or inter-observer agreement beyond that expected by chance. The Kappa coefficient can range from 0 to 1, where 0 means no correlation and 1 is perfect agreement(²⁷). In this study, there were only five articles(¹²-¹⁶) found that addressed the reliability assessment of the MTS, and all used the Kappa statistic to measure the reliability of the scale. These findings suggest that further studies on the reliability of this protocol must be designed, since knowing the reproducibility of the MTS ensures safer practice for the nurse, and the assignment of the patient's severity level reflects his real health condition.

Authors showed that the MTS is more inclusive, meaning it raised the level of priority for patients when compared to Canadian ESI scales and a Dutch screening/triage protocol(¹²-¹⁴,¹⁸,⁷). A study in Sweden showed that the MTS has raised the priority of less urgent patients, showing a low accuracy for triage cases of minor urgency(¹⁵). These findings may imply that the MTS provides greater certainty in identifying critically ill patients. On the other hand, increasing the priority level of less severe cases when compared to other protocols may lead to an increased demand for care in emergency services, contributing to overcrowding in these services.

The reliability of the MTS ranged from good to very good from a study in pediatric emergency service (Kappa: 0.65 to 0.83)(¹³). In two other studies, one in Sweden and one in Australia, the MTS showed good reliability between observers, with Kappa indices of 0.80 and 0.81, respectively(¹⁵-¹⁶). The agreement between the MTS and two other protocols for risk classification was also evaluated. In the first comparing the MTS to Canadian protocol, the MTS showed an average agreement when considering the misclassification that occurred between neighboring colors (K = 0.48), and good when considering the misclassification occurred between extreme colors (K = 0.61)(¹²). In the second study, the MTS showed better reliability and inter/intra-observer when compared to the ESI scale, and the MTS obtained 90% agreement in judgments and this rate dropped to 73% when using the ESI(¹⁴).

The inter-observer agreement reflects the ability of the MTS to assign the same severity level to a patient when used by different nurses. The studies found(¹²-¹⁶) indicate that the agreement between observers ranged from .40 to .81, meaning that the reliability of the MTS ranged from moderate to almost perfect, as to the parameters adopted by Landis and Koch(²⁷).

The intra-observer correlation reflects the reproducibility ability of the MTS results, that is to ensure that the severity level assigned to the patient is the same when the protocol is used more than once at different times by the same nurse to evaluate the same patient, as their clinical condition remains constant. The studies found(¹³-¹⁴) indicate that the intra-observer agreement was between 0.65 and 0.84, showing a reliability that varies from substantial to almost perfect (²⁷).

A study the emergency service of a children's hospital in the Netherlands showed that when compared to the ESI scale, the MTS showed better agreement, ranging from 0.65 to 0.83(¹³). The same result was observed in a study performed in an Medical Academic Center in Amsterdam when comparing the correlation in both protocols, where the value of the MTS Kappa varied from 0.68 to 0.83, while the ESI ranged from 0.37 to 0.55(¹⁴). Another study also showed that the MTS is a reliable triage for the emergency department, especially when evaluated for extreme errors, where the range of Kappa was 0.40 to 0.80, an average of 0.63(¹⁶).

Thus, it was realized that the reliability of the MTS ranged from moderate to almost perfect, being greater when the intra-observer agreement was evaluated. These findings point to the need for a revision of the MTS in order to identify existing weaknesses in the protocol and that lead to differences in nurses' classification in order to achieve better levels of internal and inter-observer reliability. In clinical practice, it is observed that one of the possible causes of disagreement between nurses' classification is the operational definition of discriminators. Many of these are repeated and have no clear description of how they should be evaluated. This problem may originate in the failure to follow the established methodological steps for translating and adapting the protocol for use in Brazil. Thus, a methodological study which does the translation, adaptation and validation of the MTS for use in Brazil is suggested.

Of the 14 selected studies, the majority (9 to 64.2%) were centered on the validity of the MTS. This was measured by reference to the following outcomes: evolution of condition/illness severity(¹⁷); sensitivity and specificity(¹⁸,²¹); percentage of overtriage and undertriage(¹⁵,¹⁸,²⁴-⁷); necessity of hospitalization(¹⁹,²²-²³) and mortality rate(²⁰,²³).

A study(¹⁷) that correlated the classification of the patient obtained using the MTS with the same level of severity measured by the Therapeutic Intervention Scoring System (TISS - 28) instrument. This system quantifies the therapeutic interventions performed by nursing, taking into consideration the complexity, the degree of invasion and the time spent to perform certain procedures for critical care patients, which may in turn scale the severity of it.

The findings of the study(¹⁷) showed that patients achieved different scores for the TISS-28, and those who were classified as red by the MTS showed more severe evolution than groups classified as yellow and orange. It was concluded that the levels of classification established by the MTS are valid for predicting the outcome of patients, having the TISS-28 instrument as a parameter for assessing the severity (average score: yellow 6.5 points; orange 11.5 points; red 22 points, p <0.001). The MTS was shown to be a valid instrument to organize nursing work, since it is possible to accurately predict which patients from those classified at the highest levels of severity will require a greater demand of nursing care according to the risk levels established by the protocol. Such information contributes to the service organization and assists in increasing care efficiency.

Another way to check the validity of instruments is through their sensitivity and specificity. Sensitivity is defined as the proportion of individuals with a given condition who are correctly identified by some test(⁹). In this study, accuracy can be defined as the classification of individuals at higher levels of severity, since their real health condition is serious. Specificity is defined as the ability to correctly identify individuals who do not have a sickness/illness(⁹), that is the ability of the MTS to categorize patients at lower priority levels since their actual health condition is not serious.

In order to compare the accuracy and specificity of the ESI and MTS triage protocols, the study showed that both protocols have low accuracy at all levels of classification, but the specificity for triaging patients at levels 1 and 2 was high for both protocols(¹⁸). In other words, the MTS is specific in accurately identifying patients that do not belong at the higher levels of severity, but it does not accurately identify patients classified in other levels of severity.

In contrast, a study that assessed the accuracy and specificity of the MTS in patients with acute coronary disease showed that the sensitivity of the MTS to identify high priority patients was 87.3%(²¹), meaning the MTS is valid to correctly identify the patients with a serious illness. Researchers at an emergency hospital department in the Netherlands also showed that the sensitivity of the MTS to detect emergencies was 63% and the specificity was 78%(²⁶).

The studies found that assessed the validity of the MTS sensitivity and specificity show mixed results, and seem to point to equivalent and satisfactory instrument levels of sensitivity and specificity. However, it is recommended to conduct further studies aimed at evaluating the accuracy and specificity of the MTS for determining the priority of patients in emergency departments, in order to obtain more uniform/consistent results.

Another parameter used to assess the validity of the MTS in the studies was the percentage of patients triaged above the level of real severity, called overtriage in English, and patients triaged below the level of real severity, called undertriage.

In a study conducted in the Netherlands, the percentage of patients triaged for levels of severity less than their real condition presented was 15%, and 96% of these patients belonged to a lower category than their real classification level of severity. In the same study, ranking patients above the actual level of severity occurred in 40% of patients, mainly those classified in lower categories of the MTS(²⁵). When evaluating emergencies in a pediatric hospital, 54% of patients were triaged above the level of real severity, and 12% were triaged for lower levels of severity than the real condition presented. The percentage of patients triaged for lower severity levels using the MTS was lower (11%) compared to the ESI scale (20%)(²²). These findings reinforce the inclusive character of the MTS and the low accuracy in even identifying patients with lower levels of urgency.

The need for hospitalization was also indicated as a parameter for measuring the validity of the MTS. Studies found a positive association between the risk levels established by the MTS and the need for hospitalization(²²-²³). This means the higher the level of urgency, the greater the need for hospitalization.

Just as the need for hospitalization, the mortality rate was configured as an important outcome used to measure the validity of the MTS. A study in Portugal(²³) showed a strong association between the priority group established by the MTS and mortality rates in the short term, and the higher the priority level set in the MTS, the higher the mortality rate. Among patients classified as red, 31.8% died versus 1.4% mortality in patients classified as blue.

Studies have been conducted to verify the ability of the MTS to triage patients with typical chest pain(²⁰-²¹). The MTS has been able to accurately assign a level of emergency or very urgent priority to 87.3% of patients with acute coronary syndrome(²¹). This finding is corroborated by another study(²⁰) which showed that early mortality in patients with acute myocardial infarction classified using the chest pain flowchart was lower among those belonging to the highest severity levels, meaning that the protocol identified severe cases who received early treatment, resulting in a greater chance of survival for the patient(²⁰).

It is noteworthy that the studies found in this integrative review for the stipulated time period (1999-2013), refer to the assessment of English and Portuguese versions of the MTS. Recently, a study conducted in Germany(²⁸) evaluated the validity and reliability of the German version of the MTS. The German version differs from the English version because it was subject to the cultural adaptation process that resulted in changes to the presentation of flowcharts and discriminators. The results showed that the agreement between nurses for the German version was almost perfect (K = 0.95), with a higher Kappa value to that found in studies evaluating the English and Portuguese versions of the MTS. This indicates that these versions require revision as to the validity of content in order to increase its reliability. Regarding validity, the German version of the MTS proved to be equivalent to the English and Portuguese versions, being a good predictor of hospital admission, the need for hospitalization in the intensive care unit, and mortality.

Conclusion

From the evaluated studies, it is concluded that the reliability of the MTS varies between moderate (K = 0.40) to almost perfect (K = 0.84), being higher intra-evaluators. This finding points to the need for revision of the system/protocol in order to increase its reproducibility, which is one of the most important conditions to ensure the reliability of a measuring instrument.

Importantly, the risk classification is dependent on the nurse-patient interaction to correctly identify the main complaint that will dictate the flowchart choice and discriminators used to define the risk level of the patient. The guarantee of a place and adequate resources for patient assessment, and prior training of nurses in the use of the protocol are needed to increase the reliability of the classification risk interventions.

Regarding validity, the MTS proved to be a more inclusive protocol, meaning that increases in the given severity level were represented by the actual condition presented by the patient. These findings could be supported by higher percentages of patients triaged above the level of real severity when compared to those triaged for lower levels of severity to which they belonged. Thus, we conclude that the MTS protects patients by ensuring a higher level of priority and a shorter waiting time for medical evaluation.

The MTS was shown to be a good predictor of the need for hospitalization and mortality, thereby being a useful tool not only for the management of clinical risk at the entrance doors of emergency care services, but also to help manage the demands of subsequent risk classification, such as the human resources and technology necessary for patient care according to the priority levels set by the MTSl.

Protocol directors/drivers are needed to guide the assessment of the nurse's risk classification and to reduce the subjectivity and bias inherent in clinical decision-making. From the articles found, we realized the need for new studies to investigate the validity and reliability of the MTS, to recommend necessary changes for it to be used more safely by nurses in the clinical risk management of patients in emergency services.

• ¹
  Brasil. Ministério da Saúde. Política Nacional de Atenção as Urgências [Internet]. Brasília; 2006 [citado 2013 dez. 18]. Disponível em: http://bvsms.saude.gov.br/bvs/publicacoes/politica_nac_urgencias.pdf
  » http://bvsms.saude.gov.br/bvs/publicacoes/politica_nac_urgencias.pdf
• ²
  Brasil. Ministério da Saúde; Secretaria de Atenção à Saúde. Política Nacional de Humanização da Atenção e Gestão do SUS. Acolhimento e classificação de risco nos serviços de urgência [Internet]. Brasília; 2009 [citado 2013 dez. 18]. Disponível em: http://bvsms.saude.gov.br/bvs/publicacoes/acolhimento_classificacao_risco_servicos_2009.pdf
  » http://bvsms.saude.gov.br/bvs/publicacoes/acolhimento_classificacao_risco_servicos_2009.pdf
• ³
  Pires PS. Escala Canadense de Triagem e Acuidade (CTAS): validação e aplicação. Rev Emerg. 2005;1(2):14-9.
• ⁴
  Azevedo JMR, Barbosa MA. Triagem em serviços de saúde: percepções dos usuários. Rev Enferm UERJ. 2007;15(1):33-9.
• ⁵
  Bullard MJ, Unger B, Spence J, Grafstein E; CTAS National Working Group. Revisions to the Canadian Emergency Department Triage and Acuity Scale (CTAS) adult guidelines. CJEM. 2008;10(2):136-51.
• ⁶
  Farrohknia N, Castrén M, Ehrenberg A, Lind L, Oredsson S, Jonsson H, et al. Emergency Department Triage Scales and their components: a systematic review of the scientific evidence. Scand J Trauma Resusc Emerg Med [Internet]. 2011 [cited 2014 Oct 5];19:42. Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3150303/
  » http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3150303/
• ⁷
  Grupo Brasileiro de Classificação de Risco. Mapa de utilização do protocolo de Manchester no Brasil [Internet]. Belo Horizonte; 2014 [citado 2014 out. 5]. Disponível em: http://www.gbcr.org.br/mapa
  » http://www.gbcr.org.br/mapa
• ⁸
  Grupo Brasileiro de Classificação de Risco. O Sistema Manchester de Classificação de Risco [Internet]. Belo Horizonte; 2010. Disponível em: http://www.gbcr.org.br/
  » http://www.gbcr.org.br/
• ⁹
  Polit D, Beck CT, Hungler B. Fundamentos de pesquisa em enfermagem: métodos, avaliação e utilização. 5ª ed. Porto Alegre: Artmed; 2004.
• ¹⁰
  Mendes KDS, Silveira RCCP, Galvão CM. Revisão integrativa: método de pesquisa para a incorporação de evidências na saúde e na enfermagem. Texto Contexto Enferm. 2008;17(4):758-64.
• ¹¹
  Galvão CM. Níveis de evidências [editorial]. Acta Paul Enferm [Internet]. 2006 [citado 2014 jul. 30];19(2):V. Disponível em: http://www.scielo.br/pdf/ape/v19n2/a01v19n2.pdf
  » http://www.scielo.br/pdf/ape/v19n2/a01v19n2.pdf
• ¹²
  Souza CC, Toledo AD, Tadeu LFR, Chianca TCM. Risk classification in an emergency room: agreement level between a Brazilian institutional and the Manchester Protocol. Rev Latino Am Enfermagem. 2011;19(1):26-33.
• ¹³
  Van Veen M, Teunen-van VFM, Steyerberg EW, van Meurs AH, Ruige M, Strout TD, et al. Repeatability of the Manchester Triage System for children. Emerg Med J. 2010;27(7):512-16.
• ¹⁴
  Storm-Versloot MN, Ubbink DT, Chin a Choi V, Luitse JS. Observer agreement of the Manchester Triage System and the Emergency Severity Index: a simulation study.. Emerg Med J 2009;26(8):556-60.
• ¹⁵
  Olofsson RN, Gellerstedt M, Carlström ED. Manchester Triage in Sweden: interrater reliability and accuracy. Int Emerg Nurs. 2009;17(3):143-8.
• ¹⁶
  Grouse AI, Bishop RO, Bannon AM. The Manchester Triage System provides good reliability in an Australian emergency department.. Emerg Med J 2009;26(7):484-6.
• ¹⁷
  Pinto Júnior D, Salgado PO,. Chianca TCM Predictive validity of the Manchester Triage System: evaluation of outcomes of patients admitted to an emergency department. Rev Latino Am Enfermagem [Internet]. 2012 [cited 2014 June 17];20(6):1041-47. Available from: http://www.scielo.br/pdf/rlae/v20n6/05.pdf
  » http://www.scielo.br/pdf/rlae/v20n6/05.pdf
• ¹⁸
  Storm-Versloot MN, Ubbink DT, Kappelhof J,. Luitse JS Comparison of an informally structured triage system, the emergency severity index, and the Manchester triage system to distinguish patient priority in the emergency department. Acad Emerg Med. 2011;18(8):822-9.
• ¹⁹
  Veen VM, Steyerberg EW, Lettinga L, Ruige M, van Meurs AH, van der Lei J, et al. Safety of the Manchester Triage System to identify less urgent patients in paediatric emergence care: a prospective observational study. Arch Dis Child. 2011;96(6):513-8.
• ²⁰
  Providência R, Lourenço PG, Barra S, Silva J, Seca L, Antunes A, et al. Importance of Manchester Triage in acute myocardial infarction: impact on prognosis.. Emerg Med J 2011;28(3):212-6.
• ²¹
  Pinto D, Lunet N, Azevedo A. Sensibilidade e especificidade do Sistema de Manchester na triagem de doentes com síndrome coronária aguda. Rev Port Cardiol. 2010;29(6):961-87.
• ²²
  Wulp IVD, Schrijvers AJP, Stel HF. Predicting admission and mortality with the Emergency Severity Index and the Manchester Triage System: a retrospective observational study.. Emerg Med J 2009;26(7):506-9.
• ²³
  Martins HMG, Dominguez LMCC, Freitas P. Is Manchester (MTS) more than a triage system? A study of its association with mortality and admission to a large Portuguese hospital.. Emerg Med J 2009;26(3):183-6.
• ²⁴
  van Veen M, Steyerberg EW, Ruige M, van Meurs AH, Roukema J, van der Lei J, et al. Manchester triage system in paediatric emergency care: prospective observational study. BMJ. 2008;337:a1501
• ²⁵
  Roukema J, Steyerberg EW, van Meurs A, Ruige M, van der Lei J, Moll HA. Validity of the Manchester Triage System in paediatric emergency care.. Emerg Med J 2006;23(12):906-10.
• ²⁶
  Albino R, Grosseman S, Riggenbach V. Classificação de risco: uma necessidade inadiável em um serviço de urgência de qualidade. ACM Arq Catarin Med. 2007;36(4):70-5.
• ²⁷
  Landis JR, Kock GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159-74.
• ²⁸
  Graff I, Goldschmidt B, Glien P, Bogdanow M, Fimmers R, Hoeft A, et al The German Version of the Manchester Triage System and its quality criteria: first assessment of validity and reliability. PLoS ONE. 2014;9(2):e88995.

Publication Dates

• Publication in this collection
  Jan-Feb 2015

History

• Received
  17 Aug 2014
• Accepted
  19 Oct 2014