Sepsis risk assessment : a retrospective analysis after a cognitive risk management robot ( Robot Laura ) implementation in a clinical-surgical unit

Introduction: This study aimed at evaluating the impact of the implementation of a cognitive robot (Robot LauraTM) on processes related to the identification and care of patients with risk of sepsis in a clinical-surgical unit of a private hospital in Curitiba-PR. Methods: The study data were obtained from the retrospective review of medical records of patients identified with infection and/or sepsis, in the period of six months before and after the implementation of such technology in the hospital. In addition, the Average Attendance Time (AAT) was obtained from the autonomous reading of the robot. Results: The average time/median until antibiotic prescription from the first identified sign of infection, with or without sepsis, was 390/77 and 109/58 minutes, respectively, in the six months before and after implementation of the technology. However, this difference was not statistically significant (p = 0.85). Regarding AAT, it was possible to observe a reduction from 305 to 280 minutes when comparing the periods of six months before and after the implementation of the technology (p = 0.02). Conclusion: Technologies such as this may be promising in helping healthcare professionals to identify risky situations for patients, as well as in assisting them to optimize the care required. However, further studies, with a greater number of subjects and with different scenarios, are necessary to consistently validate the results found.


Introduction
Sepsis is a syndrome resulting from pathogenic factors and characteristics of the host (age, comorbidities, genetics, and environment), differing from the infection itself due to the host's deregulated response in the presence of organic dysfunction (Singer et al., 2016).According to the ILAS (Latin…, 2015), sepsis accounts for 25% of ICU (Intensive Care Unit) bed occupancy rates in Brazil and its associated mortality can vary from 29.6 to 54.1% in private and public hospitals, respectively, making it the costliest disease in the health sector.The cost of care for a patient with sepsis is six times higher than of a patient without sepsis, with the approximate cost of $ 25,000 per patient, amounting to a total of $ 17 million per year (Sogayar et al., 2008).
Sepsis syndrome definitions have recently changed (Singer et al., 2016).The ESICM (European Society of Intensive Care Medicine) and the SCCM (Society of Critical Care Medicine) had convened a Third International Consensus Task Force to re-examine definitions of prior conferences of international consensus (Bone et al., 1992;Levy et al., 2003).Summing up these changes, the current definitions of sepsis and septic shock no longer include SIRS criteria (Systemic Inflammatory Response Syndrome) and necessarily consider organic dysfunction signals in a patient with a suspected infection, a situation in which the SOFA (Sequential Organ Failure Assessment) score is the recommended parameter to define patients with sepsis.According to the SOFA score, which was developed by an expert panel in 1996 (Vincent et al., 1996), the worst values recorded for every 24 hour period in the ICU are used to assign changes in the dysfunction status of six organ Sepsis risk assessment: a retrospective analysis after a cognitive risk management robot (Robot Laura  ) implementation in a clinical-surgical unit Res. Biomed. Eng. 2018 December;34(4): 310-316 311/316 Assessment of the impact in the identification of patients with risk of sepsis after implementation of a cognitive robot risk manager systems.Based on this last consensus, hospital mortality due to sepsis was reported as greater than 10%.On the other hand, patients with septic shock, which can be clinically identified by a vasopressor requirement to maintain a mean arterial pressure of 65 mm Hg or greater and a serum lactate level greater than 2 mmol/L (>18 mg/dL) in the absence of hypovolemia, have been associated with hospital mortality rates greater than 40% (Seymour et al., 2016).
Classically, to optimize results in the treatment of sepsis, it is necessary to guarantee rapid diagnosis and early initiation of therapy, mainly with antimicrobial agents, which, when administered within 60 minutes of its recognition, may contribute to improving the survival of patients with this syndrome (Kumar et al., 2006).This strategy is strongly recommended by guidelines for the management of sepsis and septic shock, which advocate that the administration of intravenous antimicrobials has to be initiated as soon as possible after recognition and within one hour for both sepsis and septic shock (Rhodes et al., 2017).
Therefore, it seems reasonable to state that the adequate identification of a septic patient, as early as possible, is an extremely relevant goal to achieve quality of care and positively impact the mortality related to this syndrome.However, recognizing a patient with sepsis may not be as obvious as it seems, and the literature strongly recommends that health institutions deploy coordinated resources and protocols to safely and efficiently achieve this objective (Instituto…, 2015).Obviously, the benefit of this kind of intervention depends upon the patient's underlying short-term risk of mortality.For patients at high short-term risk, aggressive treatment and broad-spectrum empirical antibiotics significantly decrease the mortality risk (Rivers et al., 2001).
Faced with the considerable complexity in identifying and predicting mortality in patients with sepsis, much study has occurred on the use of resources related to machine learning models and artificial intelligence, most of them based on creating models to support clinical decisions and predict mortality in hospitalized patients, especially in intensive care units (Friedman, 2009;Gultepe et al., 2014;Minne et al., 2008).
Therefore, considering the importance and complexity of sepsis detection and management, in addition to all of the technological benefits related to this subject, this study aimed to describe the impact of a new risk-management cognitive robot, first implemented as a pilot project at a private hospital in Curitiba, related to the processes of identification and care for patients at sepsis risk in a clinical-surgical unit.

The cognitive robot
According to information provided by the Laura Team, the Laura Robot™ (#laurabot) is an artificial intelligence device that manages risks by autonomous and agnostic learning of its motors, connected in real time with the databases of systems, equipment generators, and data recorders of the hospital.Thus, with the connection to the hospital database, the robot can alert the care team about patients at risk in real time.
The robot flow includes the following steps: a) remote access to all databases and data-generating equipment of the hospital; b) data mining to classify anomalous, inconsistent, and faulty records; c) classification of these data collections and generation of risk alarms for each patient, based on the training performed by the medical specialist on the algorithms; e) classification of alarms according to their frequency and importance in risky areas, which is visually translated to the care team in sight management panels installed in the hospital nursing posts; f) autonomous activation of the communication spectrum functionality when the most critical risk zone is activated, and the data continues to warn about the damage.This feature also manages the sending of SMS (Short Message Service) and e-mails to the professionals in charge, to draw the attention of the experts on the inherent damage captured by the robot.These alarms are intended to alert health professionals, to anticipate the care directed for patients at risk.

Machine Learning
A Machine Learning algorithm is embedded in the Laura Robot™ (#laurabot) to analyse every vital sign collected from the hospital, in real time.Even though this algorithm was not used for generating alerts, it could show the hospital areas at higher risk by changing the color of the dashboard.Machine learning algorithms used by the Laura Robot™ are based on vital signs and demographic information from patients.Two algorithms are used, jointly: Support Vector Machines (SVM) and Artificial Neural Networks (ANN).The output is an average of the patient deterioration index of both algorithms.

Dataset
The dataset contains the history of the vital signs for all 60 patients in the hospital over one year.As the main goal of the Laura Robot™ is to predict patient deterioration, algorithms are trained using the binary outcome -patient survival or patient death -for each sample.In cases of patient death, vital signs collected up to 6 hours before the outcome are used in the dataset.

Training and validation strategies
The hyperparameters of both SVM and ANN are fine-tuned using the RS (Random Search) algorithm.For SVM, the hyperparameter optimization is performed on K, L, and Z.On the other hand, the optimization for ANN is done for the learning rate and the number of layers and of neurons for each layer.The dataset is divided into 75% for training and validation and 25% for testing.The algorithms are trained using 10-fold cross validation, to check the model's gerality.

Robot implementation process in the hospital
The Laura Robot™ was first connected to electronic medical records of the Nossa Senhora das Graças Hospital (Curitiba, Paraná, Brazil) on July 23, 2016 -this implementation had the financial support of Laura Company.
To perform the first robot learning about the identification of patients with risk of sepsis, a regression was performed in the hospital electronic medical records (Tasy™), including the whole year of 2016, together with the existing spreadsheet database of selected patients who had been identified and included in the institution's sepsis protocol, already in place since 2014.To screen for a possible case of infection, the robot was trained with Systemic Inflammatory Response Syndrome (SIRS) criteria and/or the presence of organic dysfunction, represented by low blood pressure (<90 mm Hg).The other parameters of organ dysfunction coming from lab tests were not fully available for the study analysis, as text data was not read by the technology.Then, after the regression robot-learning period, the technology was fully implemented in two units of the hospital, being one of them specialized in the care for cancer patients and the other for clinical-surgical patients.From this specific moment, which happened on September 23, 2016, the Laura Robot™ started tracking all electronic medical records every 3.8 seconds and analyzing the information regarding vital signs, which were entered by the health care assistance team together with laboratory tests (hemogram and platelets).As long as the Robot Laura™ identified available combination signs and lab results that could represent a risk of infection based on the institution's sepsis protocol, a visual alert was issued on a television screen installed in the nursing station.At the same time, the visual alarm was also issued in case of missing data, identified by the robot, depending on previous data entry routine, whose frequency analysis was autonomously studied by the technology at electronic medical records.Besides, according to the duration of identified risk reading data by the robot, text messages were sent to nursing professionals' cell phones to alert them to verify the conditions of the patient at risk.The frequency of this sending of text messages was based on the AAT.

Patient's and hospital's statistical data
A cohort of patients admitted to one of the two units that received the technology (clinical-surgical unit), previously identified as having an infection by the usual procedures of the institution, was retrospectively analyzed by manual electronic medical record review.The analysis period included six months before (April to September 2016, called Period 1) and six months after (October 2016 to March 2017, called Period 2) the technology implementation.
To analyze its impact on the care process of patients identified with infection in the clinical-surgical unit, specifically, all patients who had come through the Emergency Room (ER) immediately before hospitalization and had the first infection signs identified by the healthcare team at this unit were excluded from the sample.This was because this did not have the robot effectively installed, and during the period of the study, there were some particular assistance process modifications that directly impacted the time to perform antibiotic therapy in cases of suspected sepsis.
Concerning the retrospective classification of cases regarding the presence of sepsis and septic shock, the third international consensus on sepsis definitions (Singer et al., 2016) was used.Cases of infection with no sign of organ dysfunction were classified as "Infection without Dysfunction".Additional information about organ dysfunction evidence was obtained retrospectively, based on previous consensus (Levy et al., 2003).Charlson index was used for the patient's severity classification (Charlson et al., 1987).For the definition of Community-or Hospital-acquired infection, the ANVISA diagnostic criterion was used (Agência…, 2017).
Data on the number of hospital admissions and days of hospital stay at the studied units were provided by the hospital statistics.

Robot data
During the study period, a compiled data spreadsheet relating to patients that had any identified risk was extracted from the robot, specifying patient number and alarm reasons, identified by date and hour of the episode.The robot alarms related to missing data were excluded due to the purpose of this analysis.Also, information about the AAT, which was autonomously calculated by the robot and represented the time for the insertion of any type of data in the electronic medical record, whether vital sign data, prescriptions, evolutions or laboratory tests results.Res. Biomed. Eng. 2018 December;34(4): 310-316 313/316 Assessment of the impact in the identification of patients with risk of sepsis after implementation of a cognitive robot risk manager

Statistical analysis
All variables collected from the electronic medical record were manually included in an Excel™ spreadsheet.For statistical analysis, the software IBM SPSS Statistics™ version 23.0 was used.The statistical analysis for continuous and normal distribution variables was performed using Student's T-test.For variables that do not have a normal distribution, the non-parametric Mann-Whitney test was used.As for the qualitative variables, depending on the sample number, the Chi-square test or the Fisher's exact test was used.The fixed confidence interval was 95% with a significance level of 0.05.

Results
Nine hundred and seventy-four admitted patients were identified at the clinical-surgical unit in the first period and 1086 patients were admitted in the second period, according to hospital statistics data.Sixty of these patients were identified as having infection evidence (30 patients in each period) by the usual active search methods of the institution (antimicrobial consumption, cultures, and sepsis protocol).Of these 60 patients, 36 patients had the first infection signs identified by the healthcare team at the studied unit -a summary profile of these subjects is shown in Table 1.
Comparing the periods before and after the robot implementation, there was no significant difference between the patients' profile.The presence of SIRS criteria at the time of infection diagnosis was evidenced in 89.5% and 76.5% (p = 0.39) and the signal of dysfunction in at least one organ was verified in 57.9% and 41.2% of the patients (p = 0.50), respectively, in the two periods of analysis.Also, there was no difference between the two groups regarding the prevalence of sepsis (42.1% and 35.3%), septic shock (5.3% and 0%), and hospital stay (21.8 and 24.6 days).About 22% (4/18) of the patients evolved to septic shock after the sepsis therapy had been started in period 1 and only 11% (2/17) in period 2, but this difference was not statistically significant.
Community-acquired infection was predominant in the two periods (73.7% and 70.6%), with an insignificant difference (p = 1.00).As for the mortality rate among the 36 patients with first infection evidence at the clinical-surgical unit, 4 deaths occurred among the 19 analyzed cases in period 1 (21.1%)and, in period 2, there were 2 deaths among the 17 cases analyzed (11.8%).However, this difference was not statistically significant (p = 0.66).
Regarding the time interval between the first identified signal of infection (SIRS or organ dysfunction) in the medical records and the prescription of the first dose of the antibiotic, the average/median time was 309/77 and 109/58 minutes for periods 1 and 2, respectively.However, this difference was not considered statistically significant (p = 0.85), as illustrated in Table 2 and Figure 1.
Regarding the analysis of the robot alarms issued in the study period, all 60 hospitalized patients identified with an infection at the clinical-surgical unit were considered, regardless of whether the first sign of infection was recorded in the inpatient unit or emergency room.Then, based on this data, it was possible to observe that approximately 26.2% to 28.9% of the patients admitted in the studied unit had some alarm issued based on SIRS risk assessment or organ dysfunction (systolic blood pressure < 90mmHg) in periods 1 and 2, respectively.However, only 11.7% and 9.5% were diagnosed with unit in question, it was observed that 55 of the 60 patients had some alarm identified by the robot during hospitalization, and this reading was possible through regression in the electronic medical records database.
For the five patients who did not receive an alarm, the investigation in the electronic medical record revealed that the corresponding vital sign data had been recorded in a text format at medical and nurse notes.So it could not be read by the robot.Also, through the information generated autonomously by the robot regarding AAT, which measures the interval between any entry of data in the system, it was possible to verify that there was a statistically significant reduction between the period before and after the implementation of the technology (305 minutes to 280 minutes), possibly showing improvement in the team's performance in including patients' data in the electronic medical record.Table 3 summarizes all this information.Specifically, concerning mortality and sepsis classification, none of the patients with infection without dysfunction died during 30 days of hospitalization, but among those classified as Sepsis and Septic Shock, the mortality proportion was 33.3% and 28.3% in periods 1 and 2, respectively, with the difference having no statistical significance, as shown in Table 4.
Table 2. Analysis of the time interval, in minutes, between the firstly identified signal of infection in the medical records and the first dose of antibiotic prescription (n=36).Table 3. Relationship between hospitalizations in the clinical-surgical unit and the frequency of Laura Robot™ alarms, cases of infection, and cases of sepsis/septic shock (including patients that had admission through Emergency Room n=60).an infection and 5.8% and 5.5% were identified with sepsis or septic shock in periods 1 and 2, respectively.In a complementary way, through analysis of the retrospective cohort of cases identified as infection in the

Discussion
The proportion of patients identified with an infection by the usual methods of the institution in relation to the number of hospitalizations was similar in the two analyzed periods (3.08% and 2.76% before and after the robot implementation, respectively), with the majority of cases having a community origin in both periods (73.7% and 70.6%, respectively).The same similarity could also be observed between the two groups of patients regarding characteristics related to mean age, proportion of genders, presence of comorbidities, and classification of cases as sepsis and septic shock.However, concerning the new sepsis definition, it is interesting to observe that some cases that were identified with any organ dysfunction by the previous classification did not present SOFA score ≥ 2 and were not classified as sepsis (11/19 and 7/17 cases with an identified organ dysfunction, but only 8/19 and 6/17 classified as sepsis by the SOFA ≥ 2).Thus, it is worth mentioning that, despite the fact that these new definitions have been endorsed by many therapy societies throughout the world, it has also generated some controversy, mainly because of increased specificity at the cost of sensitivity (Machado et al., 2016).
Regarding the impact on the processes related to sepsis risk and management assessment, patients admitted to the clinical-surgical unit without previous passage through the ER had an apparent improvement in the interval between the first sign of infection and the antibiotic prescription at the post-robot period (309/77 and 109/58 for mean/ median time in minutes, respectively); however, without statistical significance.Maybe this finding can be explained by the sample number and great variability of the data.Regarding time interval variability between cases, an additional analysis was performed, excluding the two values considered "outliers," as illustrated in Figure 1, but even so, it was not possible to detect statistical significance.Also, another fact that must be considered in this scenario is that the institution had already a sepsis protocol instituted since 2014, which possibly interferes with the care team performance.
It is well known that early recognition of sepsis is one of the keys to affect clinical treatment and reduce mortality (Kumar et al., 2006).Obviously, other factors may affect the mortality rate of patients with sepsis, such as comorbidities and the severity manifested at the time of diagnosis (Yoshihara et al., 2011).In this study, analyzing the global mortality proportion between the 60 patients identified with an infection, 29 had sepsis or septic shock and 9 of them evolved to death (31%) within 30 days of hospitalization.This finding is in line with national data reported in private hospitals (Latin…, 2015), with no difference detected between the two periods (33.8% and 28.3%).However, an interesting point to observe is that, although the prevalence of septic shock was relatively low at the time of the infection diagnosis, during the course of the disease some patients evolved to septic shock despite adequate therapy -4/27 (14.8%) and 2/28 (7.1%) in periods 1 and 2, respectively.In spite of the apparent difference, it was statistically insignificant (p = 0.36).
About the relationship between hospitalizations in the clinical-surgical unit and the frequency of Laura Robot™ alarms, it was possible to notify that approximately 10% of hospitalized patients who had any robot alarm issued had in fact an infection, and cases of sepsis and septic shock represented for about 5%.These results confirm the high sensitivity of SIRS criteria and, in some way, draws attention to the possible high false positive alarms that could desensitize the care time.Thus, it seems rational to try to find better criteria to define patients at sepsis risk, but it is also reasonable to consider SIRS to screen an infection possibility at bedside (Machado et al., 2016).
Regarding the AAT analysis, it was possible to observe a statistically significant reduction from 305 to 280 minutes, which may represent a positive impact on the performance of the care team in including any data at the electronic medical records system, but there was no stratified report available provided by the robot to refine this analysis in the studied period.
In critiquing this study, it must be first pointed out that it was a retrospective review, pre-and post-implementation of a new technology, and the data obtaining process was a manual search on electronic medical records, which presents limitations in itself.Also, the comparisons were made between a group of patients before and after implementation of the robot at the same unit, and no comparison was performed with other hospitalization units with similar profiles and without the technology.Finally, it should be noted that individual patient validation of all alarms generated by the robot was not performed to confirm its relevance and relationship with the diagnosis of possible infection or sepsis.
In conclusion, the main message to be emphasized is that the use of tools such as the one presented in this study has at its core a potential not only for helping healthcare professionals in their practice, but also glimpses the possibility for generation of research, information, and knowledge that can also be reverted to the continuous improvement of processes related to quality of care in health institutions.However, further and prospective studies with larger sample sizes should be conducted to substantially validate the use of technology to improve patient safety.Moreover, comparisons of SIRS-based risk identification systems with other hospital death probability scores may be useful in defining the sensitivity and specificity of the presented technology in predicting more accurately the patients at death risk.Also, ideally, it should be desirable that technologies like this are capable of capturing not only registered data on closed field, but also keywords and laboratory tests monitored during the individual time evolution of each patient, and not only according to pre-defined cut-off points, which do not always reflect the reality of each individual.

Figure 1 .
Figure 1.Boxplot comparison of the time interval, in minutes, between the first identified infection signal registered in the medical records and the first dose of antibiotic prescription (n=36).*The time values 1 and 9 represented by asterisks are outlier, taken from the mean and median calculations.

Table 1 .
Characterization of the sample -36 hospitalized patients who had their first infection signal identified in the clinical-surgical unit.

Table 4 .
Mortality proportion according to sepsis classification among patients identified with an infection in the clinical-surgical unit (including patients admitted through the Emergency Room; n=60).