Predictive validity of the Manchester Triage System: evaluation of outcomes of patients admitted to an emergency department

Abstracts

OBJETIVO: evaluar la validez predictiva del protocolo de clasificación de riesgo de Manchester implantado en un hospital municipal de Belo Horizonte, Minas Gerais. MÉTODO: estudio de cohorte prospectivo y analítico. La muestra estratificada por colores de la clasificación fue de 300 pacientes. El final evaluado fue la puntuación por el Therapeutic Intervention Scoring System - 28, lograda por los pacientes en cada grupo de clasificación después de 24 horas de la admisión en el servicio de urgencia. RESULTADOS: entre los pacientes, 172 eran hombres (57%); la media de edad de los pacientes evaluados fue de 57,3 años. La mediana de puntuación del índice de gravedad en el grupo amarillo fue 6,5 puntos; en el grupo naranja, 11,5 puntos y, en el grupo rojo, 22 puntos, habiendo diferencia estadística significante entre los tres grupos (p<0,001). CONCLUSIÓN: los datos refuerzan que los pacientes evolucionan con niveles de gravedad diferentes entre los grupos de colores de clasificación.

Triaje; Evolución Clínica; Enfermería; Servicios Médicos de Urgencia

OBJETIVO: avaliar a validade preditiva do protocolo de classificação de risco de Manchester, implantado em um hospital municipal de Belo Horizonte, Minas Gerais. MÉTODO: trata-se de estudo de coorte prospectivo e analítico A amostra estratificada por cores da classificação foi de 300 pacientes. O desfecho avaliado foi a pontuação pelo Therapeutic Intervention Scoring System - 28, obtida pelos pacientes em cada grupo de classificação, após 24 horas da admissão no serviço de urgência. RESULTADOS: entre os pacientes, 172 eram homens (57%) e a média de idade dos pacientes avaliados foi de 57,3 anos. A mediana de pontuação do índice de gravidade no grupo amarelo foi de 6,5 pontos; no grupo laranja, 11,5 pontos e, no grupo vermelho, 22 pontos, havendo diferença estatística significante entre os três grupos (p<0,001). CONCLUSÃO: os dados reforçam que os pacientes evoluem com níveis de gravidade diferentes entre os grupos de cores de classificação.

Triagem; Evolução Clínica; Enfermagem; Serviços Médicos de Emergência

ORIGINAL ARTICLE

Predictive validity of the Manchester Triage System: evaluation of outcomes of patients admitted to an emergency department¹

Domingos Pinto Júnior^I; Patrícia de Oliveira Salgado^II; Tânia Couto Machado Chianca^III

^IMSc, RN, Hospital Municipal Odilon Behrens, Brazil

^IIDoctoral Student, Escola de Enfermagem, Universidade Federal de Minas Gerais, Brazil. RN, Hospital Municipal Odilon Behrens, Brazil

^IIIPhD, Full Professor, Escola de Enfermagem, Universidade Federal de Minas Gerais, Brazil

^{Corresponding Author}

ABSTRACT

OBJECTIVE: to assess the predictive validity of the Manchester Triage System implemented in a municipal hospital in Belo Horizonte, MG, Brazil.

METHOD: cohort prospective and analytical study. The sample of 300 patients was stratified by color groups. The outcome measured was the scores, obtained by patients in each classification group in the Therapeutic Intervention Scoring System - 28, 24 hours after admission to the emergency department.

RESULTS: A total of 172 (57%) patients were men and the average age of all patients was 57.3 years old. The median score concerning the severity of their conditions was 6.5 points in the yellow group, 11.5 in the orange group, and 22 points in the red group. Statistically significant differences were found among the three groups (p<0.001).

CONCLUSION: the data confirm that the conditions of patients within the color groups progressed at different levels of severity.

Descriptors: Triage; Clinical Evolution; Nursing; Emergency Medical Services.

Introduction

The use of protocols to guide health workers when determining priority of treatment has been adopted to classify risks. This triage process is not a new situation. Countries such as Australia, the United Kingdom, Canada and the United States have developed their own protocols to guide the classification of patients to define priority of treatment upon entry into Emergency Departments (ED)⁽¹⁾.

The State Health Department in Minas Gerais, Brazil opted to standardize the reception of patients in health services through the implementation of a triage protocol imported from Manchester, England⁽¹⁾. This protocol was developed in 1994 by a group of professionals specializing in triage. The Manchester Triage System (MTS^©) classifies risks into five categories. Based on the identification of the patient's main complaint, a specific flowchart, guided by discriminators and presented in the form of questions, is selected. The status of each discriminator is established based on the patient's clinical history and symptoms classifying the individual into five categories: immediate (red); very urgent (orange), urgent (yellow), standard (green) and non-urgent (blue). There is a triage target time for each category: 0, 10, 60, 120 and 240 minutes, respectively. Hence, care delivery is organized according to the severity of the patients' conditions so that the most severe cases have priority of care⁽²⁾.

One study recommends the use of scales to stratify the risk of patients into the five levels, because scales have higher validity and reliability in assessing the clinical conditions of patients⁽³⁾. Currently, the protocols most frequently used worldwide to classify risks in urgent/emergency services are: the Australian Triage Scale (ARS^©), the Canadian Triage Acuity Scale (CTAS^©), Emergency Severity Index (ESI^©) and the Manchester Triage System (MTS^©). All these scales organize care delivery into five levels of priority.

Similar to other risk classification scales, the MTS^© recognizes that a triage method should provide health workers not with a diagnosis but with a clinical priority. The establishment of the exact diagnosis at the time of triage is doomed to failure. Additionally, clinical priority is much more closely linked to the needs of patients at the time they seek care in the service than to the precise diagnosis of diseases⁽³⁾.

Some questions have emerged after the implementation of these triage models in emergency services and conducting studies assessing the MTS^© is essential. A study⁽⁴⁾ assessing the degree of agreement between the MTS^© and a Brazilian institutional protocol showed that the MTS^© is more inclusive and classifies more patients at higher levels of priority. It means that demand for care in ED is increased but, at the same time, it is safer for patients, who receive care more readily. The authors, however, stress the need for studies validating the MTS^© for the Brazilian context⁽⁴⁾.

There are few Brazilian studies assessing the MTS^© despite its increasingly frequent use to classify risks in public health services. The objective of protocols that classify risks is to identify patients with severe conditions and give them priority of care⁽⁵⁾. Nonetheless, a question arises: can the protocol, in addition to giving priority to urgent care, predict the conditions of which patients will progress less satisfactorily?

International studies have also verified the evolution of patients after the triage. A study conducted in Portugal sought to determine whether the protocol could, in addition to classifying the risks of patients, predict the progression of their conditions⁽⁶⁾. According to the study's author, the MTS^© is capable of discriminating among patients with high and low probabilities of death, as well as among those who will remain in the service and those who will be discharged.

Another study conducted in Holland also assessed the ability of the MTS^© and the ESI^© to predict mortality and hospitalization. Both systems were capable of accurately predicting hospital admission. The mortality of patients was also associated with the categories of higher priority in both protocols⁽⁷⁾. Nonetheless, studies assessing the protocol's level of prediction need to be implemented.

The predictive validity of an instrument refers to its ability to discriminate between the performances or behaviors of individuals in relation to some criterion in the future. Therefore, the predictive value of a classification refers to how well it is supported by the clinical progress of patients, or how the classification obtained by patients at the time of admission is confirmed by the progress of their conditions during their stay in the emergency service⁽⁸⁾.

This study was conducted to evaluate the predictive validity of the MTS^© implemented in a municipal hospital in Belo Horizonte, MG, Brazil.

This type of study is important to the goal of better identifying the implications of the process of classifying risks, which is a relatively recent field in our practice. Additionally, the protocols used to evaluate and classify risk, especially the MTS^©, need to be studied in more detail so that the results can support improvement of care delivery.

Method

A cohort, prospective and analytical study was developed in a large hospital in Belo Horizonte, MG, Brazil. A total of one thousand patients per day, almost exclusively patients covered by the Brazilian Unified Health System (SUS), receive care in this facility. The institution has 402 beds, 135 in the ED. The hospital is one of the main entry points in the city to care for clinical urgencies.

This study complies with the guidelines established by Resolution 196/96, the Brazilian Council of Health, concerning research involving human subjects. It was approved by the Ethics Research Committee at UFMG (process No. 0033.0.216.203-09) and by the Ethics Research Committee at the Odilon Behrens Municipal Hospital and ALERT^®. Individuals were included in the study only after consenting and signing free and informed consent forms. Consent regarding those in mental states of considerable confusion and/or unconsciousness was obtained from their legal representatives.

Population and sample

The study's population was composed of all the patients admitted to the hospital's emergency room and who were assessed and classified by the nurses through the MTS^© from March 31^st to September 15^th, 2010. The computation of the sample size took into account that the sample should enable inferential statistical analysis among the various groups of classification (immediate, very urgent, and urgent) in addition to establishing differences among them through the test of hypotheses. For that, we opted to use a calculation stratified by colors. A power of 80% was considered and the permutation test was used to define the difference that would be considered significant among groups. The permutation test uses resampling through statistical simulation to compare the proportions among the different risk classifications⁽⁹⁾. Hence, a sample size of 300 patients, 100 for each classification group, was found.

Inclusion criteria

The patients admitted to the ER and who underwent the risk classification process and remained in the hospital for at least 24 hours after the classification were included in the study. Those classified into the green and blue groups (standard or non-urgent cases) were not included in the study because, according to the internal flow established in the institution's care protocol, these patients are referred to primary health care or outpatient services and do not remain in the service after classification.

Data collection

Data were collected from April 2^nd to September 16^th 2010. Data collection was performed by one of the researchers and collaborators who consulted the ALERTA^® system daily. This computer system was implemented in the ER together with the MTS^©. Those patients classified into the red, orange, or yellow groups and who remained in the hospital for more than 24 hours and less than 48 hours were selected for the study.

An instrument validated for Brazil, the Therapeutic Intervention Scoring System (TISS-28) was used to measure the severity of the patients' conditions⁽¹⁰⁾. TISS was designed in 1974 and updated in 1983^(11-12) to measure the severity of a patient's condition and the nursing workload in Intensive Care Units (ICUs) based on the quantity of nursing and medical therapeutic interventions. It is based on the assumption that the greater the number of procedures and interventions required by a patient, the more severe the patient's condition. In 1996, to facilitate its application in practice and make it a more adjustable index to measure nursing workload, extensive changes were implemented and the new version is now called TISS-28⁽¹³⁾.

Because TISS-28 was validated for Portuguese, it is a practical and easy-to-apply instrument, useful to assess the severity of patients' conditions, and is especially important for nursing clinical practice, we opted for applying it as an instrument to measure the severity of patients' conditions in the hospital's ER.

The collaborators were trained prior to data collection in order to obtain an acceptable level of inter-rater agreement in the application of TISS-28. This procedure aimed to standardize the meaning of each of the instrument's items and to avoid interpretation bias in the assessment procedures. A pilot study was implemented with 30 patients after training. These patients were simultaneously and independently evaluated by the collaborators and one nurse experienced in the use of the TISS-28 in his clinical practice, who was considered to be the gold standard evaluator in this study. To ensure simultaneous and independent assessment, an agreement was established prior to the assessments that the raters would not dialogue, exchange information or clarify doubts during the application of the TISS-28. Hence, each rater was aware of the score and items concerning the instrument s/he was responsible for. Data were then processed and assessed to verify agreement between the collaborators and the nurse considered to be the gold standard.

The level of agreement was computed by the Kappa coefficient⁽¹⁴⁾. Only those collaborators who achieved a minimum agreement of 0.80 or 80% in the Kappa coefficient with the gold standard nurse were selected to collect data with the TISS-28.

Data processing and analysis was performed through the Statistical Package for Social Sciences (SPSS), version 15.0. To characterize the sample in relation to gender, descriptive analysis with absolute frequencies and Person's Chi-square test were applied to verify differences in relation to gender among the color groups. P-values <0.05 were considered significant.

In regard to age, a descriptive analysis with simple frequency, central tendency measures and dispersion was used. The Mann-Whitney non-parametric test with Bonferroni correction was performed to verify the age differences among the groups of colors^(15-16).

The analysis per color group was performed using the average score obtained from the TISS-28 in each category of classification to verify whether the difference found among the color groups was significant. Then, the medians of each group and differences found were analyzed using the Mann-Whitney test with Bonferroni correction^(15-16).

Results

Among the 300 studied patients, 172 (57%) were men and 128 (43%) were women. No statistical differences were found among patients in relation to gender in the studied groups, according to Person's Chi-square test.

The age of patients ranged from 1 to 100 years old, with an average of 57.3 years. The red group presented a median age of 60 years old; the median age of the orange group was 58 years old; and 57 years old was the median age in the yellow group. No statistical difference was found in the comparison among groups in terms of age.

We analyzed the scores obtained by patients on the TISS-28 to measure the severity of their conditions. The median score obtained by the yellow group was 6.5 points; 11.5 by the orange group; and 22.0 points was the median obtained by the red group.

The distribution of scores obtained by the patients in the red, orange, and yellow groups in TISS-28 is presented in Figure 1.

A statistically significant difference was found among the groups in relation to the score obtained on the TISS-28. The Mann-Whitney test applied with Bonferroni correction indicated a p-value below 0.001 in all the comparisons of medians. Bonferroni correction indicated a significant p-value below 0.0167.

Discussion

The average age of patients in this study was 57.3 years old, which is considered high when compared to the averages reported in the literature^(12-13). Average ages of 42.4 years old and 38.7 years old were found in a study comparing the MTS^© and ESI^©, respectively^(4,7).

This study was conducted in an ED caring for patients with urgent trauma and clinical conditions. Clinical urgencies accounted for most of the care provided and most frequently occur in older patients, which increases the time patients remain within health services⁽¹⁷⁾. The flowcharts used to classify risk in this study confirm that most patients seeking the hospital presented clinical conditions. Additionally, one of the inclusion criteria was remaining within the service for at least for 24 hours, which may have favored the selection of older individuals. Another factor that may have increased the patients' average age was the exclusion of standard and non-urgent categories, which differs from the criterion used in the studies previously mentioned^(4,7).

The predominance of male individuals in the studied sample is corroborated by another study that also found a larger number (56%) of male patients⁽⁷⁾.

Different scores were obtained by the color groups in the application of TISS-28 aiming to evaluate the severity of the patients' conditions after risk classification. It is important to note that the instrument to assess the severity of conditions was applied between 24 and 48 hours after the patients were admitted to the service. Therefore, the results show that patients tended to progress differently according to the classification of risk.

The difference among the classification groups and the scores obtained on the TISS-28 was significant. International studies have tried to assess the progression of patients after classification, however, these studies used indirect indicators of severity, such as mortality and hospitalization^(6-7). None of the studies applied a validated instrument to measure the severity of the studied patients' conditions to assess their progress during their stay in the health service. It is important to note that in this study, increasing averages were found in the scores obtained on the TISS-28 among patients classified into the yellow, orange and red groups: 6.5, 11.5 and 22 points, respectively. These averages confirm that patients' conditions progressed differently during their hospitalization within the service. Therefore, the risk classifications established by the MTS^© were considered to be predictors of the severity of patients' conditions.

A study designed to determine whether the MTS© could detect which patients admitted to the ED would require critical care was performed⁽¹⁸⁾. These authors report that 67% of the patients classified in the immediate and very urgent categories required intensive care. The MTS^©, however, failed to correctly classify 5.5% of the cases. These patients were classified in lower priority categories but their conditions progressed after classification and deterioration of their vital signs required critical care during their stay in the ED. The authors concluded that the MTS^© was sensitive enough to detect which patients would require critical care but failed to identify some cases in which the patients' clinical conditions deteriorate after arriving at the ED⁽¹⁸⁾.

Similar to the aforementioned study, this study's results showed that some patients in the very urgent (orange) and urgent (yellow) groups were outliers, that is, presented quite discrepant results in relation to the group as a whole⁽¹⁸⁾. The red group did not present any outliers but had a large standard deviation. Some scores obtained on the TSS-28 by the patients in these groups were well above what was expected, two to three showing standard deviations above the average, showing that some patients were initially classified in less urgent categories, but their clinical condition worsened after they were admitted to the ED.

Therefore the importance of patients who have already being received a risk classification being continually assessed after being admitted to the service is apparent. Authors argue that risk classification is a dynamic process, therefore it demands constant reevaluation until the patient receives treatment specific to his/her problems^(2,4).

Limitations and Strengths

The TISS-28, used as an instrument to measure the severity of patients' conditions, required that patients who remained in the service less than 24 hours were excluded from the analysis. Such a procedure may have led to the groups being more homogeneous, that is, patients in the red group who died before 24 hours were excluded, while in theory, the less severe patients in the red group would have been excluded from the study. Patients in the yellow group who were discharged before 24 hours were also excluded from the study and only the most severe patients were included. Therefore, we consider that the exclusion of patients who stayed less than 24 hours in the service made the group more homogeneous. Despite this limitation, important results were found in this study and significant differences were found among the red, orange and yellow groups in relation to the score obtained in TISS-28.

It is also important to justify the choice of the TISS-28 as instrument to measure the severity of patients in the ED. It is known that it is a validated instrument to be used in ICUs and has also been applied in other situations such as patient transportation⁽¹⁹⁾. The objective of this study was to verify how the patients would progress some time after their risk classification, but we also aimed to verify this progress through clinical and physiological aspects presented by patients instead of using mortality rates or hospitalizations as reported by other studies. Thus, the TISS-28 was chosen based on the fact that its applicability was known and that it was able to meet the proposed objectives.

Conclusions

The conclusion is that, in addition to establishing priority for patients in the ED, the MTS^© is capable of predicting the progress of patients during their stay in the facility. The analysis showed that the conditions of patients classified at different levels of risk progressed differently.

Patients admitted into the ED obtained different scores on the TISS-28 over the progression of their conditions. The red group presented a more severe progression in relation to the orange and yellow groups. The condition of the orange group progressed less severely than that of the red group and more severely than the yellow group.

Hence, the classification of risk through the MTS^© was able to predict which patients presented a greater probability of unfavorable outcomes. These findings' contributions concern not only health professionals but also the health system, which needs to be organized to offer more efficient and, at the same time, more efficacious service to those seeking Emergency Departments.

Received: Nov. 11^th 2011

Accepted: Oct. 12^th 2012

• 1. Grupo Brasileiro de Classificação de Risco (BR). Histórico da Classificação de Risco. 2009. [acesso 3 dez 2010]. Disponível em:
• 2. Freitas P. Triagem no serviço de urgência: Grupo de Triagem de Manchester. 2.ed. Portugal: BMJ Publishing Group; 2002. 149 p.
• 3. Gilboy N, Tanabe P, Travers D, Rosenau AM, Emergency Severity Index, Version 4: Implementation Handbook [Internet]. Rockville, MD: Agency for Healthcare Research and Quality; 2005 [acesso 2 dez 2011]. 95 p. Disponível em: http://www.ahrq.gov/research/esi/esihandbk.pdf
• 4. Souza CC, Toledo AD, Tadeu LFR, Chianca TCM. Classificação de risco em pronto-socorro: concordância entre um protocolo institucional brasileiro e Manchester. Rev. Latino-Am. Enfermagem. [periódico na Internet]. 2011 [acesso 7 nov 2011]; 19(1):[08 telas]. Disponível em: http://www.scielo.br/pdf/rlae/v19n1/pt_05.pdf
• ⁵
  Ministério da Saúde (BR). HumanizaSUS - Acolhimento com avaliação classificação de risco: um paradigma ético-estético no fazer em saúde. Brasília (DF): Secretaria Executiva / Núcleo Técnico da Política Nacional de Humanização; 2004.
• 6. Martins HMG, Cunã LMCD, Freitas P. Is Manchester (MTS) more than a triage system? A study of its association with mortality and admission to a large Portuguese hospital. Emerg Med J. 2009;26:183-6.
• 7. Wulp IV, Schrijvers AJP, Stel HFV. Predicting admission and mortality with the Emergency Severity Index and Manchester Triage System: a retrospective observational study. Emer Med J. 2009;26:506-9.
• 8. Polit DF, Hungler BP. Fundamentos de pesquisa em Enfermagem. 3. ed. Porto Alegre: Artes Médicas; 1995. 391 p.
• 9. Good PI. Permutation Tests: a pratical guide to resampling methods for testing hypotheses (Springer Series in Statistics). 2th.ed. Nova York: Springer; 2000.
• 10. Santos DS, Chianca TCM, Werli A. Estudo de revisão sistemática sobre o Sistema de Pontuação de Intervenções Terapêuticas 28. Rev Enferm UFPE on line [periódico na Internet]. 2010 abr/jun [acesso 30 mar 2012]; 4(2):413-9. Disponível em: http://www.ufpe.br/revistaenfermagem/index.php/revista/article/view/870/pdf_3
• 11. Cullen DJ. Therapeutic intervention scoring system: a method for quantitative comparison of pacient care. Crit Care Med. 1974;2(2):57-60.
• 12. Kenne AR, Cullen DJ. Therapeutic intervention scoring system: update 1983. Crit Care Med. 1983;11(1):1-3.
• 13. Miranda DR, Rijk, AD, Schaufeli W. Simplified therapeutic intervention scoring system: the TISS - 28 itens-result from a multicenter study. Crit Care Med. 1996;24(1):64-73.
• 14. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33Suppl 1:159-74.
• 15. Mann HB, Whitney DR. On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other. Annals Mathematical Statistics. 1947;18(1):50-60.
• 16. Dunnett CW. A multiple comparisons procedure for comparing several treatments with a control. J Am Stat Assoc. 1955;50(272):1096-121.
• 17. Conselho Nacional de Secretários de Saúde (BR). SUS: avanços e desafios. Conselho Nacional de Secretários de Saúde. Brasília (DF): CONASS; 2006.
• 18. Cooke MW, Jinks S. Does the Manchester triage system detect the critically ill? J Accid Emerg Med. 1999;16(3):179-81.
• 19. Lee LLY, Yeung KL, Lo WYL, Lau YSC, Tang SYH, Chan JTS. Evaluation of a simplified therapeutic intervention scoring system (TISS-28) and modified early warning score (MEWS) in predicting physiological deterioration during inter-facility transport. Resuscitation. 2008;76(1):47-51.

1

Paper extracted from Master's Dissertation "Valor preditivo do protocolo de classificação de risco em unidade de urgência de um hospital municipal de Belo Horizonte" presented to Escola de Enfermagem, Universidade Federal de Minas Gerais, Brazil

Publication Dates

• Publication in this collection
  17 Dec 2012
• Date of issue
  Dec 2012

History

• Received
  11 Nov 2011
• Accepted
  12 Oct 2012