Risk Classification in an Emergency Room : Agreement Level Between a Brazilian Institutional and the Manchester Protocol

The aim of this study was to assess the level of agreement between an institutional protocol and the Manchester protocol for the risk assessment of patients attended in an emergency room of a public hospital in Belo Horizonte - MG - Brazil. This is a descriptive and comparative study, in which 382 patients' reports were evaluated and the risk was classified, using the institutional protocol and the Manchester protocol. Rates were calculated through weighted and unweighted kappa, in order to determine the level of agreement between the protocols. The results showed that the correlation between the protocols is average when considering that classification errors occurred between neighboring colors (kappa=0.48), and good when considering that classification errors occurred between extreme colors (kappa=0.61). The Manchester protocol increased the patients' level of priority of patients and has been considered more inclusive.


Introduction
Urgency and emergency services represent an important component of Brazilian healthcare.In recent years, the demand for urgency and emergency care has increased, mainly due to the increase in the number of accidents and urban violence.The reality of overloaded Brazilian emergency care units is aggravated by organizational problems, such as attendance in order of arrival without establishing clinical criteria, which can entail severe damage for patients (1) .
Aware of existing problems in the urgency area, in 2004, the Brazilian Ministry of Health launched the National Humanization Policy Folder, in which welcoming with assessment and risk classification is appointed as a device to change health care work and production, particularly at urgency services.Risk classification is a dynamic process of identifying patients who need immediate treatment, according to the potential risk, health problems or degree of suffering.Care should be prioritized in line with the severity of the patient's clinical condition, instead of the order of arrival at the service (2) .
urgency services, and should follow a guiding protocol (3) .
In general, the use of scales/protocols has been recommended which rank risks in five levels, as these display better reliability, validity and trustworthiness levels to assess the patient's clinical condition (4) .Among these scales/protocols, the following can be mentioned: the North American scale -"Emergency Severity Index (ESI)"; the Australian scale -"Australasian Triage Scale (ATS)"; the Canadian protocol -"Canadian Triage Acuity Scale (CTAS © )" and the British protocol -"Manchester Triage System" -Manchester Protocol (5) .Besides these, other protocols have been developed, such as the protocol of Hospital Municipal Odilon Behrens -HOB -Minas Gerais, Brazil (6) , which was based on preexisting scales and protocols, using the Canadian data collection method.The HOB was the first hospital in Minas Gerais that put in practice risk classification and serves as a Brazilian reference for the use of this device.
Today, the Brazilian Ministry of Health have attempted to standardize the welcoming process with risk classification by adopting measures that unify this process across the national territory (7) .In this sense, in Minas Gerais, technological modernization and the implantation of the computerized Manchester protocol were chosen to guide risk classification at all urgency services in the State.In some institutions, however, especially the HOB, a protocol constructed by the hospital team was already being used.Some authors affirm that there is no standard protocol in health measurement (8) .Measurement demands indicators that represent a concept.There is no infallible measurement instrument.Instead, situational and environmental factors, tendentiousness in answers, personal factors and alterations in the data collection method can contribute to measurement errors (9) .When using instruments from another language, cross-cultural adaptation is needed, even when the language is the same, as problems can arise in the cultural adaptation process.In this process, some items can be changed to maintain the same concept, and there is not much clarity as to how much of the original measurement properties are maintained in the adapted instrument (10) .

The clinical patient risk classification criteria
established in both protocols (HOB and Manchester) include the priority levels and color attributed, the waiting time for medical attendance, nursing re-assessment and the method to assess the main complaint, layering the risk in five distinct levels which have been indicated by colors for the sake of easier visualization.Differences between both protocols exist, however, especially regarding waiting time for medical attendance, time for nursing re-assessment (not included in the Manchester Protocol), the method to assess the main complaint, layered through flowcharts in the HOB Protocol and symptoms in the Manchester Protocol.Thus, at this moment of putting in practice a new protocol to replace a preexisting one, a comparative study is needed between the two protocols for patient classification at urgency and emergency units.
It is known that the Manchester Protocol is used in many countries for patient risk classification at emergency and urgency services and that, before deciding on its practice at institutions in Minas Gerais state, a group of Brazilian specialists assessed the feasibility of adapting this protocol to the Brazilian reality.There is no research, however, on differences in risk classification results when For data analysis, SPSS (version 17.0) and R (version 2.7.1) statistical software was used.Descriptive analysis was carried out, and sensitivity and specificity measures were calculated for each classification level.The agreement level between the risk classification according to the HOB and Manchester protocol was measured by Cohen's Kappa ratio.Kappa ratios between 0 and 0.20 were considered bad agreement; between 0.21 and 0.40 weak agreement, between 0.41 and 0.60 average agreement; between 0.61 and 0.80 good agreement; between 0.81 and 1 excellent agreement.At a 95% confidence interval, the weighted Kappa ratio was calculated to analyze general agreement between the two protocols, as well Kappa not weighted by color, in order to identify the main points of agreement and disagreement between the protocols (11) .In addition, sensitivity and specificity ratios were calculated for each classification level.Sensitivity ratios were calculated to evidence the probability that a user classified at a given risk according to the HOB protocol would be classified at the same priority level according to the Manchester protocol.Specificity was used to evidence the probability that users who were not classified at a given risk level according to the HOB protocol were not classified at the same risk level either when using the Manchester protocol.
To check for the existence of correlation and its strength between the patient's age and the attributed risk classification level, Spearman's non-parametric correlation test and the linear regression analysis tests were used, respectively.To analyze the influence of vital data on risk classification according to both protocols, the Mann-Whitney test was used.The variables that were compared were the risk classification, using the two protocols (ordinal categorical variable) between the two independent groups (patient files with complete

Results
Out of 204 patients, the majority (53.4%) was female, with a mean age of 39.32 years (minimum 13 and maximum 89; median 36 and SD=17.38 years).
According to the nomenclature of the HOB protocol, a range of patient complaints is perceived, including urgent complaints related to respiratory failure, digestive hemorrhage, acute hemiparesis, severe abstinence from alcohol and drugs, convulsion and cranial trauma, besides non-urgent complaints (flue symptoms, light to moderate pain, vomiting and diarrhea without dehydration).
According to the Manchester protocol nomenclature, on the other hand, pain was the main patient complaint, with headache and abdominal pain corresponding to 31.5% of all complaints, followed by malaise in adults (14.0%), thoracic pain (10.8%) and limb problems (7.8%).
According to the study protocols, risk classification per color permitted an analysis of the patients' distribution in absolute figures (Table 1).It should be highlighted that, in 38 cases, the classification could not be accomplished by both protocols, as these flowcharts contained no guiding elements compatible with the main complaints that were described, which is why they were removed from the agreement analysis.In all cases that could not be reclassified, the main complaint was not well described, with a lack of records on the start, intensity and duration of the symptoms.
To assess the general agreement level between the two protocols, the Kappa ratio with linear and squared weighting was calculated.The Kappa ratio with global linear weighting equaled 0.48, against 0.61 with squared weighting, based on which it can be inferred that the agreement level is average when considering disagreements between neighboring colors, and good when considering disagreements between extreme colors.
The analysis of agreement level per color (Table 2) revealed average agreement between the protocols for patients classified as orange (Kappa=0.53),followed by weak agreement for patients classified as green (Kappa=0.37),yellow (Kappa=0.26),red (kappa=0.25)and bad agreement for blue (kappa=0.06).In orange, the agreement level between the protocols was medium (kappa=0.53),with 65% of patients classified as orange by both protocols.In view of a 0.65 sensitivity level, it can be affirmed that patients classified as orange by the HOB protocol would also be classified as orange by the Manchester protocol, and that patients not classified as orange by the HOB protocol would not be classified as such either by the Manchester protocol, with sensitivity equaling 0.90.
Among patients classified as orange by both protocols, 82.5% manifested some type of pain, and "intense pain" findings were definitive for agreement levels on the color orange in the patients' classification.
In yellow, the agreement level between both protocols was weak, with kappa equaling 0.26.Among

Discussion
The main complaints the patients in this study presented during the risk classification confirm literature findings about emergency hospitals delivering care to truly urgent as well as non-urgent cases.It should be highlighted, however, that the concept of urgency and emergency can differ in users and health workers' view.
The population in general cannot distinguish between urgency and emergency (12) .Health professionals appoint divergences between the health needs that make users turn to the urgency unit and the work goal at this site, revealing dissatisfaction with patients' excessive demands, whose needs can be classified as urgency or emergency (13) .Thus, some authors highlight the need for public policy strategies to enhance the efficiency and quality of urgency services (14) .
The Manchester protocol increased the percentage of patients classified as red, orange and green by 4.4%, 0.7% and 12%, respectively, and raised the priority level of those classified as blue by 10.2% in comparison with the results of the risk classification according to the HOB protocol.This demonstrated that the Manchester protocol was more inclusive (Table 1).A study carried out to verify the association between classification according to the Manchester protocol and mortality and hospitalization rates at the urgency service showed that this protocol is a powerful tool to distinguish between patients with high and low mortality risks, as well as to distinguish between patients who will not need hospitalization and who should return home (15) .
The analysis of the global agreement between the HOB and Manchester protocols shows a greater probability of classification "errors" between neighboring colors than between extreme colors.With a view to investigating nurses' accuracy for the risk assessment and classification according to the HOB protocol, a low accuracy level was found between service nurses' classification and standard protocol.Total agreement or disagreement was not found at any classification level when comparing the nurses' classifications with the institutional protocol (7) .Reliability analysis among nurses was assessed when using the Manchester protocol, showing a kappa ratio between 0.40 and 0.80, with a median ratio of 0.63.The Manchester protocol is a reliable instrument for use at emergency departments (16) .
Based on literature findings, it can be inferred that the intra and inter-rater agreement levels are higher when using the Manchester protocol (17) .It should be highlighted, however, that there are less studies assessing the HOB protocol than those assessing the Manchester protocol.These findings are attributed to the classification method inherent in each protocol.The Manchester protocol's arrangement in guiding flowcharts facilitates nursing assessment, turning the classification process more secure and neutral.
The Manchester protocol increases the number of patients classified as red, green and blue and decreases the priority of patients classified as orange and yellow when compared with each priority level based on the HOB protocol.Hence, when using the Manchester protocol, a decrease is observed in the priority levels of patients classified under orange and yellow to green and blue.This ratifies literature findings (17) .
The Manchester protocol is considered a sensitive tool to detect, at the entry door of the emergency services, patients who will need critical care.Based on the risk classification according to the Manchester protocol, 67% of patients admitted to critical care areas were classified as red or orange.The protocol showed defects, however, to detect the cases of patients whose clinical condition worsened after going through the risk classification (18) .This result underlines the need to constantly reassess patients after the risk classification, www.eerp.usp.br/rlae Rev. Latino-Am.Enfermagem 2011 Jan-Feb;19(1):26-33.
until treatment is administered to solve their problems.

Complete records on the main complaint influence
the risk level patients are attributed when using the HOB protocol, with higher priority levels for patients with complete records on the main complaint.Based on this finding, it can be inferred that a complete assessment of the main complain raises the priority level patients are attributed, underlining the importance of nursing skills and competencies for the correct and complete assessment and description of the main complaint the patient presents at the time of welcoming and risk classification.

Conclusion
It can be concluded that the agreement between the protocols is average when considering the classification errors that occurred between neighboring colors, and good when considering classification errors between extreme colors.This was proven in the analysis on the agreement level at each classification level, in which most disagreements were related to neighboring higher or lower priority levels.In general, the Manchester protocol increased patients' priority level when compared with the HOB protocol.This shows that the Manchester protocol is more inclusive.
using the Manchester Protocol and other local protocols like the HOB.Until date, no publications were found on the validation of the Manchester Protocol in Brazil within criteria established in literature.Hence, this study was designed to verify the agreement level between the HOB and Manchester Protocols for risk classification of users attended at the HOB emergency unit.Method This descriptive and comparative study was carried out at the HOB.The hospital is part of the Belo Horizonte municipal health network.The study population comprised 339,133 care forms, corresponding to all files of patients attended at the HOB emergency unit since Welcoming with Risk Classification was put in practice (September 22 nd 2005) until September 22 nd 2007.Patient files were included when the users went through risk classification and the files contained records that identified the nursing professional who delivered care and described the evaluation and the classification the user received.The sample contained 382 files.Sampling was based on simple randomization of the files and an electronic draft of the patient record numbers.Stat Calc, Epi Info software, version 3.4.1 was used for sample calculations.The expected minimal agreement level between the two protocols was set at 80%, with an acceptable error margin of 5%, 95% confidence level and 5% loss rate.For data collection, documentation in the drafted files were subject to content analysis, seeking the following data: complaints the patient presented, vital Souza CC, Toledo AD, Tadeu LFR, Chianca TCM.www.eerp.usp.br/rlaedata, previous history, age and risk classification at the end of the assessment carried out by the nurse.The complaints were literally transcribed in a Word document, in the form of clinical cases, so that the patients' risk classification could be redone according to the HOB and Manchester Protocols.The patients' re-classification according to the HOB Protocol was done manually.For the Manchester Protocol, the risk classification software was used with the computerized protocol.Criteria were set for comparison between the priority levels and color standard for both protocols.Thus, color red II in the HOB Protocol was considered orange, in view of existing correspondence between the priority levels in both protocols.
records of vital data and without complete records of vital data).This research complied with National Health Council Resolution 196/96 on research involving human beings.Approval was obtained from the Research Ethics Committee at UFMG (COEP-UFMG -Opinion Etic No529/08), from the Superintendence and Research Ethics Committee at Hospital Municipal Odilon Behrens, and from the company ALERT ® , which holds the copyright of the Portuguese Screening Group for the trading and use of the Manchester Protocol.
the 109 patients classified as yellow by the HOB protocol, only 35% were also classified as such by the Manchester protocol.At a low sensitivity level of 0.35, one may say that patients classified as yellow by the HOB protocol would also be classified as yellow by the Manchester protocol.At a specificity level of 0.88, patients not classified as yellow by the HOB protocol would not be classified as such either by the Manchester protocol.Among patients showing agreement between classifications, the discriminator "warm adult" was the determinant element for classification under yellow in both protocols.The specific discriminators "moderate pain" (5-13.2%),"pleuritic pain" (5-13.2%)and "colic"(3-7.9%)were also important to determine classification at this level.At priority level IV (green color), the agreement level between the protocols was also weak (kappa=0.37).Out of 127 patients classified as green by the HOB protocol, 77% were also classified as green by the Manchester protocol.It can be affirmed that a patient classified as green by the HOB protocol would also be classified as green by the Manchester protocol (Sensitivity = 0.77) and that a patient not classified as green by the HOB protocol would not be classified at this priority level either by the Manchester protocol (Specificity = 0.63).The agreement level on patients classified as blue was the lowest of all (Kappa=0.06).Out of 43 patients classified as blue by the HOB protocol, only 3 (7%) were classified as such by the Manchester protocol.Therefore, for the study sample, it can be affirmed that patients not classified as blue by the HOB protocol would not be classified as blue by the Manchester protocol (Specificity = 0.97), and that patients classified as blue by the HOB protocol are also classified as such by the Manchester protocol (Sensitivity = 0.07).Regarding the analysis of how age influences risk classification, Spearman's non-parametric correlation test showed, at a 5% significance level, that age is associated with risk classification (r=0.1;p=0.04) for the HOB protocol.On the other hand, the linear www.eerp.usp.br/rlaeSouza CC, Toledo AD, Tadeu LFR, Chianca TCM.regression analysis showed that the strength of the association between age (predictive variable) and risk classification is very small and showed to be insignificant to change the risk level attributed in clinical practice.According to Spearman's non-parametric correlation test, no significant association was found between risk classification according to the Manchester protocol and the patients' age (r=0.09 and p= 0.10).The results of the Mann-Whitney test indicated no association between the complete recording of vital data and the risk classification.P-values corresponded to 0.53 and 0.30 for the HOB and Manchester protocols, respectively.The results of the Mann-Whitney test demonstrated a significant association between risk classifications according to the HOB protocol for patients whose files contained complete records on the main complaint in comparison with those showing incomplete data (p-value=0.002).Priority levels were higher among patients in the group with a complete description of the main complaint assessment.No statistically significant association was found in the attributed risk classification between the group with and without complete records on the main complaint (p-value=0.94) in the risk classification according to the Manchester protocol.
These study results indicate that the Manchester protocol is well adapted to the HOB reality and, as it has been used in a computerized way, future research on technology use by nurses responsible for risk classification and validation studies of the Manchester protocol should be carried out, as this new technology is being put in practice in Brazil, representing an area with a lack of research.It is known that risk classification is a relatively new activity in Brazilian nursing activities, and that it has increasingly conquered its space.Thus, it is fundamental for schools to invest in the education of skilled professionals who can respond to market needs in this area.For nurses working with risk classification, skills for qualified listening, assessment, correct and detailed recording of the main complaint, teamwork, clinical reasoning and mental agility for decision making are essential, as well as knowledge on support systems in the care network with a view to responsible patient forwarding when necessary.It can undoubtedly be affirmed that the use of protocols to support risk classification offers a legal framework for safe nursing work.It cannot be ignored, however, that this is a process of welcoming and classifying.It is important to highlight that listening is the principle and willingness to listen the requisite to start a welcoming relation with the user, as that is the only way to guarantee a humanized risk classification process, enhancing the population's access to health services, thus reaching the central goal of qualified care delivery to SUS users.