Risk classification in a pediatric service : evaluation of the structure , process , and outcome

Rev Bras Enferm. 2022;75(1): e20210022 https://doi.org/10.1590/0034-7167-2021-0022 8 of ABSTRACT Objectives: to evaluate a pediatric risk evaluation and classification service in the aspects of structure, process, and outcome. Methods: a cross-sectional, descriptive, and exploratory study in which qualitative data complement the results. Data collection by non-participant observation, interview, and consultation of medical records. Results: we observed two hundred and four visits. The service works in a makeshift room and without privacy. The nurses did not use various materials/equipment available for evaluation, rarely performed a physical examination, used the interview most of the time, and did not consult the protocol. Parents/ companions received no information, and medical records showed incomplete records. Nurses classified 69.1% of the children as non-emergency. The most frequent outcome of the visits was discharge. Conclusions: the service showed weaknesses in the physical structure, in the risk classification process, and the outcomes, but it presents a good resolution in the assistance. We recommend a revision in the protocol used and sensitize nurses about the work process and communication with parents/companions. Descriptors: Risk Assessment; Pediatrics; Emergency Medical Services; Pediatric Emergency Medicine; Nursing.


INTRODUCTION
The Urgency and Emergency Services (UES) seek to welcome and serve users in clinical urgency and emergencies and have as characteristics the continuous care (24 hours), the service organization based on the free demand of users, and the focus on immediate care. However, these units suffer from the indiscriminate search of users and, in the case of Pediatric UESs, of parents who seek immediate care for their children.
Trying to manage overcrowding and improve the quality of UESs, by instituting the National Humanization Policy (NHP), the Brazilian Ministry of Health proposed the Reception with Risk Evaluation and Classification (RREC) strategy as a method of organizing and selecting patients for priority care in these services. The recommendation is that, in the implementation of this service, there be systematic monitoring and evaluation of the actions of the RREC, aimed at improving and correcting any identified failures (1) .
These evaluations should involve the service assessments in all their aspects, including the users' satisfaction and the working conditions of the professionals who work there (2) . It requires the use of evaluation methods appropriate to each objective, aiming at identifying the potentialities and weaknesses to subsidize managers in the planning of strategies that contribute to the improvement of the service offered.
In this context, Donabedian proposes the evaluation of the quality of health services based on three fundamental dimensions: Structure -it evaluates the items related to human resources, physical, material, equipment, and financial aspects necessary for care; Process -judges the working mode of the professionals; and the Result -understands the health condition of the person who received care in the services evaluated (3) . Thus, the objective of continuously monitoring the quality of a service is to exercise continuous surveillance so that deviations from established standards can be early identified and corrected (4) .

OBJECTIVES
To analyze a pediatric risk classification service in the structure, process, and outcome aspects.

Ethical aspects
The present study was approved by the Ethics and Research Committee of the Hospital das Clínicas of the Federal University of Goiás, according to resolution No. 466/12 (5) .

Study design
A descriptive, exploratory, cross-sectional study with a quantitative approach, in which qualitative data were used to complement the results. We used the STROBE (EQUATOR network) framework for observational studies in epidemiology.

Period and place of study
We carried out the study in a medium-sized and complex public hospital located in the countryside of the State of Goiás.
It is the only Public pediatric SUE in the municipality and meets spontaneous demand from patients in the Municipality and neighboring regions, and the service operates 24 hours a day, with a nurse and a nursing technician per shift.
The assistances are organized according to the severity of the child's situation, indicated by the pediatric RREC service (PRREC), which uses its guiding protocol, implemented in May 2015 (6) to classify the risk of children.
The assistance to the child in the PRREC includes weighing by the nursing technician, followed by the nurse's evaluation for the classification of risk, including interview and physical examination. Based on the signs/symptoms identified, the nurse assigns the child one of the four risk levels of the protocol, represented by the colors: Red -priority 0, emergency and immediate care; Yellow: priority 1 -urgency and care in up to 15 minutes; Green: Priority 2 -minor urgency/non-urgency and assistance within 30 minutes; and Blue: Priority 3 -low complexity consultations and first-come-first-served care in up to three hours (6) .

Population and inclusion and exclusion criteria
At the time of the study, six nurses worked at the PRREC and agreed to participate, according to the inclusion criterion of having at least six months of work, so that the lack of experience or adaptation to work did not interfere with the results.

Study protocol
Data collection took place in 2017, in three stages: nonparticipant observation of care (quantitative data); interview with the nurses who make the Pediatric Risk Evaluation and Classification -PREC (qualitative data); and consultation of the medical records of the children attended in the observation phase (quantitative data). We used the qualitative data to complement/ illustrate quantitative data.
Each step of data collection used a specific instrument built by the research team for this study and based on the literature (3,7) and the research objectives. Specialists in Pediatrics with experience in RREC evaluated, validated, and approved the instruments in their form and content before use.
The first phase, the non-participant observation (without the professionals' knowledge), was carried out on six consecutive days (Sunday to Friday), and all the visits made at the PRREC were observed in the collection periods (8 am to 11 am; 1 pm to 5 pm; and 8 pm to 11 pm), totaling 60 hours and 204 visits. The observation of the visits was carried out through the communication window between the UES reception room and the PREC room, and the observations registered in the specific checklist.
At the interview stage, we informed the nurses about the study, its objectives, the stage performed previously (observation) and sequential phases (interview and medical records consultation), and all agreed to participate. The interviews were individual, scheduled with the participants, and conducted in a private place. Before starting, we advised the nurses about the study, its objectives, and its free participation. They then received the free and informed consent form for reading and clarifying doubts and expressed their agreement by signing it. The interviews, directed by semi-structured script, had an average duration of 15 minutes. They were recorded on digital media and transcribed in full by the interviewer.
In the third stage, we collected the information from the medical records of the 204 children whose care was observed in the first stage and recorded in a specific form.

Analysis of results and statistics
We entered the quantitative data in statistical software, analyzed it by descriptive statistics, and presented it in simple and relative frequencies. The Kappa index calculated the level of agreement between the risk classification performed by the service nurse and the researchers (with the institutional protocol), considering excellent (K > 0.75), median (K = 0.40-0.75), and low (K < 0.40) compliance (8) .
We entered the data from the interviews into an initial analysis spreadsheet, using a code to identify each participant and keep their identity confidential. We read each interview repeatedly, organizing the information and separating it by the topics: structure, process, and the result of the service. We used these data in the analysis of the outcomes to complement/illustrate the quantitative data.

Evaluation of the structure of the service
Six nurses worked in the PRREC sector, and four (66.37%) were women. They were between 26 and 31 years old, two to nine years of graduation, and five had a specialization degree. Their working time in the sector ranged from six months to three years, all worked on a 12-hour regime with 60 hours of rest, and three had another employment relationship. Three nurses received training before starting work in the PRREC sector.
The children observed during the assistance were between 0 and 143 (median = 45.5) months old, and 103 (50.9%) were male. The median time interval between reception and PRREC care was 9 minutes, up to 15 minutes for 59.8% of children, 20.6% of the medical records had no such records.
Only a partition with a window separated the PREC room from the reception, making it possible to observe people outside the service. It had a table and chair for the nurse, chairs for child and companion, bathroom, sink, liquid soap and paper towel for hand hygiene, stretcher, scale, ruler for height, thermometer, liquid alcohol, and cotton for hand antisepsis and disinfection of materials. Stethoscope, wall clock, and sphygmomanometer were available in 130 (63.7%) visits, and pulse oximeter in 126 (61.8%) cases.

Evaluation of the Pediatric Risk Evaluation and classification process
The 204 visits observed were performed by the nurse, assisted by the nursing technician in 165 (80.9%) of them.
The nurse did not use a stethoscope or sphygmomanometer; he used scales in 97.5% of the visits, a thermometer in 93.1%, and a height ruler in 0.5%. The oximeter was used in 68 (54%) of the 126 visits in which it was available. Physical examination of the child has not been performed in 189 (92.6%) cases. In no case, the nurse informed parents/companions about the level of risk assigned to the child or consult the institutional protocol for risk classification. He signed the service card 204 times he was the responsible professional and used his stamp in 203 of them.
The care of the child by the nurse was interrupted in 111 (54.4%) cases, with two (36%), three (7.2%), or more interruptions. Interruptions were made mainly by nurses/nursing technicians from another sector (44 times) and by other patients/companions (38 times).
In 98.5% of the medical records, there was a record of the signs and symptoms identified by the nurse, which the most frequent were weight (98.5%) and temperature (93.6%) of the child. The duration of the complaint(s) was recorded in 82.8% of the visits. Blood pressure rates, auscultation result and child height were absent in 100% of them.
In the interviews, the nurses evaluated the institutional protocol as easy to apply and with a comprehensive language: In terms of language, it is easy to understand, yes, for us who classify. (E3) The protocol directs the approach to the client and the risk classification facilitates the service. It is easy to understand [...] practical, objective and targeted. It meets the main needs of the municipality and region. (E6)

Evaluation of the result
Based on the signs/symptoms/complaints recorded in the medical record, we performed the level of risk assigned by the nurse to the attended child and what would be assigned using the guiding protocol. In this perspective, the nurse classified and indicated the level of risk in 188 (92.2%) visits, failing to classify 16 (7.8%) cases, while using the institutional protocol, only 11 (5.9%) children could not be classified due to the absence of the signs and symptoms recorded in the protocol. Table 1 shows the risk levels assigned by nurses and what would be assigned if they consult the institutional protocol for classification. We observed that Blue (non-urgent) was the most frequent risk level (132; 70.2%) assigned by the nurse; and Green (low urgency) by the protocol (128; 66.3%).
In the interviews, however, the nurses showed no doubt about the attribution of the risk level. They revealed that: they feel competent to classify the child's risk level; they often turn to a colleague to clarify doubts; the institutional protocol is not available in the service for consultation: Regarding the outcome of the children's care, of the 193 cases classified by the institutional protocol, 83.4% were discharged after medical consultation and/or a procedure (examinations, medication, and/or suturing). Only 3 (1.6%) children needed to be admitted to a pediatric clinic, even though they were classified in the Green level, corresponding to little urgency.
Only one of the three children classified as Red had to be observed in the service for less than 12 hours, and the three who had to be admitted to a pediatric clinic were classified as not very urgent (green). Among the cases classified as urgency (27) and low urgency (128), more than half (77.8% and 81.3%, respectively) were discharged after consultation and/or a procedure (exams, medication and/or suturing). Of the 35 children classified in the non-urgent level (Blue), 97.1% were discharged after medical consultation and or a procedure.

DISCUSSION
In the analysis of the PRREC service based on the framework of the structure, process, and work outcome (3) , it was possible to look at the various aspects that interfere with the quality of the service. The service has many positive characteristics, but we need to discuss and reflect on some aspects with professionals interested in improving the quality of the services offered.

Evaluation of the structure of the service
Nurses should perform the risk classification based on a guiding protocol that evaluates the severity or potential for aggravation of the case, as well as the degree of suffering of the patient, which determines the order of medical care (1) . NHP recommends that they have experience in emergency care and be trained to do so (1) . They must also have technical skills, clinical reasoning, qualified listening, and knowledge in Pediatrics to be able to perform care based on the particularities and integrity of the child (9) .
Nurses, whose training can be considered acceptable, attended to all patients, although half of them were not trained for the activity. His working bond with the institution (predominantly, fixed-time contract) is quite unfavorable for teamwork. For the service, each professional "exchange" means disruption in the work of the team, with the arrival of new nurses, with or without experience in urgency and emergency and, in most cases, without prior training for risk classification.
In some visits, the absence of the nursing technician is a negative aspect of the service structure, requiring the nurse to take responsibility even for the tasks of that professional. It results in long waiting for care for each child, with a detriment to the flow of care, deserving the attention of managers.
The physical space of the service is precarious, with the care being carried out in a makeshift room next to the reception of the emergency room, with excessive noise that hinders the attention of the classifier nurse. The communication window between the two rooms interferes with the privacy necessary for the proper physical examination and interview. In summary, the service has acceptable furniture and most of the materials and equipment required for the work of the nurse in this service, although it misses monitor, electrocardiograph, glucometer, and equipment  To evaluate the accordance index between the risk levels assigned by the nurse and those that would be assigned using the current protocol, 26 medical records were excluded that did not have the assigned risk level and/or because they were impossible to classify with the institutional protocol because the signs/ symptoms/complaints described were not included. Then, red, yellow, and green categories were grouped, considering that they included patients who need to be seen in the UES because they are emergencies, urgencies, and little urgencies. The cases classified with blue color formed the group of non-urgent patients ( Table 2).
The nurse classified the majority (69.1%) of the children as non-emergency, while the protocol would classify 82% of the children as emergency/urgency. Although without statistical significance, the comparison between the two classifications was considered low, with the kappa index equal to 0.017 (p = 0.368).
Comparing case-by-case, the nurse classified 98 (48%) children as having a lower risk than they would receive if they used the protocol. The number of cases that the nurse did not classify, but that could be classified by the protocol (16; 7.9%), includes two emergency cases, with an indication of immediate care.
Among the signs/symptoms/complaints registered by the nurse in the child's evaluation and that were not found in the institutional protocol, the most frequent were dry cough and crying child, which appeared in 14 and 10 medical records, respectively. Participants also indicated the need for the protocol revision to include some frequent signs/symptoms/complaints in the child's evaluation and even the inclusion of another level of risk in the classification: and materials for cardiopulmonary resuscitation, such as manual resuscitator or ambu, material for intubation, etc. (7) .
Other studies have observed that the lack of materials necessary for patient examination and risk level assignment as well as structural disorganization are common complaints of nurses in these services (10)(11) . It contrasts with the results of this study, in which these materials were available most of the time, although some were not used in the consultations, and others, in no medical care.

Evaluation of the Pediatric Risk Evaluation and classification process
Patient access to the Risk Evaluation and Classification Service in UESs should be immediate, with sizing of the number of classifiers appropriate to the demand (12) . The Brazilian Risk Classification Group (BRCG), the only legal representative of the Manchester Triage Group in Brazil, determines that the time between the patient's arrival at the service and the risk classification is no longer than ten minutes (13) . In this study, the interval between reception and PRREC was up to 15 minutes in 59.8% of cases, with an average of 11.28 ± 9.929 minutes, being slightly above the recommended by the BRCG, but on the average of other UESs (14)(15)(16) .
Even children classified by the nurse as "emergency" (Red color), whose care should be immediate, had to wait up to 15 (2 cases) and up to 30 minutes (2 cases) for assistance at the PREC. The three children classified by the protocol with the Red color also waited up to 15 minutes for the risk classification. This delay can cause prominent damage to the health of the child and even death because it is consensus that the time between the onset of symptoms and medical care can be decisive for the success of the indicated therapy.
The nurse did not use several materials and some equipment available in the service to exam the child. The sphygmomanometer and stethoscope were not used in any of the visits, and the pulse oximeter was only used in slightly more than half (61.8%) of the cases in which it was available. There was no record of blood pressure rates or child height measurement in 100% of the medical records.
Even recommended by the institutional protocol (6) and the Ministry of Health (7) , we observed that nurses are not adequately rating the verification of vital signs for the screening of children. However, relying on their experience to make inferences about the severity of the situation is an unreliable conduct (17) .
As a low complexity procedure, the nursing technician can measure vital signs, although it can be challenging to obtain in sick children, in an unfamiliar environment, especially in small or small cooperatives (18) . The complete verification of vital signs increases the time spent for screening by about 1.4 minutes but should be performed and used in the screening process whenever the child is sufficiently stable, as it improves the accuracy of the screening and cannot be neglected (18) , since this increase benefits the safety of the child (17) .
An important aspect is a physical examination, which complements the interview (19) and is part of the nurse's conduct to classify risk (7) . Although the interview was used in 98% of the visits, the physical examination was only performed in 7.4% of them. There are no records of the child's height, blood pressure rates, respiratory rate, or pulmonary auscultation results. This fact impairs the efficiency and effectiveness of REC, significantly favored when the decision-making of nurses is based on interview data, with qualified listening, brief physical examination, verification of vital signs, clinical and critical judgment of complaints, psychological and interpersonal analysis (20) .
The medical records include a description of the signs, symptoms, and complaints identified by the nurse, but their duration was only recorded in 82.8% of cases. Considering that the assigned risk level is influenced by the duration of the symptomatology (6)(7) , if there is no such record, there is no way to know if it was considered.
The fact that the nurse does not consult the guiding protocol to make the risk classification of children contributes to the occurrence of failures. The nurses considered themselves competent to assign the level of risk to children and stated that the institutional protocol is easy to understand and apply, practical, clear, and objective while it is not available for consultation in the PRREC room. These results suggest the little importance assigned by the classifying nurses to the protocol (21) .
However, protocols are technical guidelines to systematize the action of professionals, assigning activities and responsibilities of each in the performance of a given task. In the risk classification, the use of the protocol scientifically bases the nurse's decision (20) and it is essential to standardize the care, streamline the flow, increase the security of care and minimize the possibility of errors. However, its implementation requires that it be designed to meet the professionals' needs, the reality of the service and that it be available for consultation at any time, under penalty of not serving its purpose (2) .
Even if nurses feel safe to classify the risk of patients without consulting the protocol, it should always be used. Even in case of doubt at the time of classifying the risk, the option to consult the protocol was not indicated by any of them. This behavior can occur either because of the little importance attributed by the professional to the protocol or because of its inexistence in the spot.
It is clear from this phenomenon how integrated strategies must be in order to change the practice of professionals in the service: the whole team must draw up the protocol, adequate it to the reality of the service, make it continuously available for consultation, professionals must be trained in the activity to which it refers and instructed to use it all the time to ensure the quality of its performance. Finally, the entire service must be regularly evaluated, preferably in a procedure that involves all the aspects involved (structure, process, and outcome (4) ) to identify weaknesses and make the necessary changes.
The frequent interruptions of the nurse's work during the evaluation of the child produce discomfort in parents and children, interrupt the line of reasoning of the professional, delay the flow of care in the sector and hurt ethical principles and patient privacy. Therefore, they should be reduced to the essential minimum, and it is the function of the manager to encourage/ promote the reflection of professionals on the complexity of work processes and on ethical principles and respect for users, awakening, throughout the team, the co-responsibility for the quality of the service performed in the unit (1) . 8 of Risk classification in a pediatric service: evaluation of the structure, process, and outcome Cabral KB, Teixeira CC, Oliveira JMC, Cordeiro JABL, Cruz KCT, Santo LF, et al.

Evaluation of the result
There was a contrast between the risk classification performed by the nurse and the one that would be obtained if the protocol was used. The nurse classified 69.1% as non-emergency, while the protocol classified 82% as emergency/urgency, the latter as the result closest to those found in the literature. Studies conducted with adults attended in UES showed a predominance of the indicative level of low urgency (22)(23) ; or emergency, urgency, and low urgency (24) . In pediatric UES, one study found that 94.4% of the visits were classified as an emergency, high urgency, low urgency, and urgency (9) ; and another, that 70.8% of the children were classified with colors corresponding to the levels of urgency (21) .
This result may contribute to the discussion about the overcrowding of UESs due to the high demand for cases that could be attended in health units with a lower level of complexity. Several studies (9,(21)(22)(23)(24) show the predominance of the number of cases whose indication would be the care in a UES.
The divergence in the two classifications may be closely related to the non-use of the institutional protocol by nurses to classify the risk for children. They consider their experience sufficient and do not need to consult the protocol and do not have it for consultation at the place of visits. Therefore, managers must make the protocol available for consultation at the site of the assistance and adopt strategies that promote the awareness of professionals about the importance of the correct risk classification and the usefulness of the protocol for this purpose.
Therefore, several aspects converge to misconceptions at the time of risk classification: the non-use of the protocol by nurses when assigning the risk level to the child; the non-use of various auxiliary materials and equipment for the evaluation of the child by the nurse; the quality of the nurses' records; the non-performance of the physical examination, as recommended by the Brazilian Ministry of Health (1) ; and the frequent interruptions in the care performed by the nurse.
All these factors are modifiable by changing the conduct of professionals, not involving costs for the service. It is clear the need to modify the service management model, with more direct supervision of the team to detect the weaknesses and propose changes. It is imperative to reorient the work of the classifier nurses, offering training/updating opportunities to improve the quality of the service and, consequently, the safety of the users.
The inadequate risk classification causes damage to the service both when it uses its human/financial/logistical resources to attend non-urgent patients mistakenly classified as emergency/urgency, and when it does not attend those who demand immediate/rapid care and are mistakenly classified as non-urgent. Although the first situation is more acceptable, we need to use a balance to avoid unnecessary use of available resources (25) .
Thus, to organize care and ensure priority to the most serious cases, the risk classification must be reliable, and the managers of the service must ensure this, holding everyone involved accountable.
Also, managers and supervisors need to give special attention to the lack of information to parents about the level of risk of the child and the probable waiting time for medical care since such information is one of the objectives of the risk classification (1) and helps relieve their anxiety and insecurity (26) . It must be provided whenever the user is on service (7) , updating it continuously.
Regarding the outcome of care in the 193 cases classified by the protocol, we observed a lower number of clinical hospitalizations (1.6%) than in other studies (10.4% to 70.1%) performed in a pediatric emergency (9,27) . In another study, 61.8% of the treated children remained under observation and/or under the care of the health team in the pediatric emergency. Only 12.7% of them were hospitalized (9) . Therefore, even if cases with a risk level relative to urgency and low urgency predominated (80.3%), the majority (83.4%) of the children were discharged after consultation with the professional on duty or a specialist and or a procedure, such as examinations, medication and/or sutures. This result indicates the high effectiveness of the service, which provides various diagnostic support services (laboratory, tomography, radiography, etc.) and medical specialties, allowing rapid response to most complaints submitted by children. It also helps to outline the service demand profile, which includes few cases at the emergency level (Red).
Despite the failures in the nurse's records, he signed the care form in all the medical records, except for one record with no identification stamp. However, incomplete completion of medical records and failure to identify the professional responsible for care have legal and financial implications since any procedure performed in public health institutions, including care in the risk classification, can generate income to the health service (10) . We emphasize that the use of the nursing professional stamp is mandatory in the registration of all technical work performed by nursing professionals (28) .

Limitations of the study
The results of this study are limited by the small number of participating professionals and the number of visits observed, and may not fully reflect the reality, restricted to the institution studied. We also did not examine the differences in the professionals' performance by the time of day, making it impossible to study the associations between the quality of the examination performed or possible errors in the risk classification and the time when they occurred (beginning, middle, or end of the work shift), or the occurrence of crying or agitation of the child at the time of care. Therefore, future studies should explore the influence of these and other aspects that may influence the professional's performance in the PREC service.

Contributions to the area of Nursing; Health and Public Policy
The results of this study show that the structure framework, process, and outcome are valuable for the evaluation of health services because it allows identifying the vulnerability of aspects that influence both the satisfaction of the professionals with work and the quality of care offered.

CONCLUSIONS
We identified weaknesses both in the structure and process, and outcome of the investigated service, but, excluding those of structure, almost all the problems identified need to be improved with change in the professionals' conduct. In the evaluation of of Risk classification in a pediatric service: evaluation of the structure, process, and outcome Cabral KB, Teixeira CC, Oliveira JMC, Cordeiro JABL, Cruz KCT, Santo LF, et al.
the process and outcome, the little importance that nurses assigned to the use of the protocol when classifying the risk of the child called out our attention, which may be one of the factors that contribute to the tendency of these professionals to classify most of them as non-urgent, in contrast to what would occur if the protocol was used. Another aspect that stands out is the lack of communication of nurses with the parents/guardians of the child, who remain uninformed about the risk color of the child and the possible waiting time for medical care.
In the process of reorganizing the service, the implementation of an administration model more present in the routine of professionals will allow managers to follow up on visits to identify failures and propose solutions agreed upon all. Finally, care with the training and continuous updating of the professionals who perform the care and the correction of the failures as soon as they are observed contribute to improving the quality of care and the safety of patients.