System for reception and risk classification in obstetrics: a technical quality assessment*

Objective: to assess the technical quality of a decision support system for reception and risk classification in obstetrics. Method: a methodological study of assessment of the system. 12 nurses and 11 information technology (IT) professionals were invited to evaluate the Obstetrics Reception and Risk Classification System (Sistema de Acolhimento e Classificação de Risco em Obstetrícia, SACR-O). Based on the standards of the International Organization for Standardization, the minimum number of evaluators and the characteristics to be evaluated were established: functional suitability, reliability, usability, performance efficiency, compatibility, safety, maintainability, and portability. The characteristics assessed should be given a ≥70% positive assessment to be considered suitable. Results: the characteristics assessed by the nurses and the IT professionals, respectively, were considered adequate: Functional suitability (97% and 98%), Reliability (91% and 94%), Usability (89% and 93%), Performance efficiency (97% and 98%), Compatibility (93% and 100%), and System security (95% and 97%). Maintainability (87%) and Portability (97%) were also evaluated by IT professionals. Conclusion: the technical quality of the SACR-O system was considered excellent by nurses and IT professionals.

The A&CRO handbook aims to provide guidance and standardization of conduct to health professionals who work in childbirth care services, in order to avoid problems in care that may culminate in unfavorable outcomes, as well as to enable qualified access and resolutive care timely for each case (4) .
The obstetrics risk classification is a decision support tool for the immediate identification of the pregnant woman's severity, which ensures fast and safe care according to the risk potential and based on scientific evidence (4)(5)(6) .
The A&CRO protocol allows the pregnant woman to be classified for the medical care, according to the degree of urgency and from a clinical decisionmaking process, in which the maximum recommended waiting time to receive care is determined. There are five priority levels and each one corresponds to a maximum waiting time for medical care, namely: red (immediate care), orange (up to 15 min), yellow (up to 30 min), green (up to 120 min) and blue (not priority or referral according to agreement) (4) .
The A&CRO protocol is an important knowledge base to support obstetrics professionals' decisions, as well as to generate positive impacts on care and error reduction. Thus, it is of great importance that instruments such as the A&CRO protocol are part of the routine of health care institutions and are available for use in electronic devices, integrated into the patient's electronic record (PER), as a form of registration for later consultation and re-evaluation (7)(8) .
In the information age, alternatives to health registering have been proposed in order to find strategies to store and qualify data registration, to build safer and more quality health care (9) .
The focus of the present study, the Obstetrics Reception and Risk Classification System (SACR-O) was developed in a university hospital in the interior of the state of São Paulo as a decision support system, built based on the A&CRO manual proposed by the MoH.
The system guides the nurse with structured questions and fixed values in their answers, based on the signs and symptoms presented by the pregnant woman. When finally filled, the system indicates the risk classification by color, as proposed by the MoH (4) . SACR-O also offers a simulation database, allowing real-time training for undergraduate students, residents, and professionals of maternity hospitals for teaching or continuing education. The system is integrated into the PER, which favors the continuity of care and information security.
Considering the evaluation of health information systems, a systematic review study has shown that the best methods are those that follow the assumptions  (10) .
From this perspective, this study aims to evaluate the technical quality of a decision support system for reception and risk classification in obstetrics.

Method
This is a methodological study for the assessment of a system. Methodological studies are characterized by the development, validation, and evaluation of research tools and methods (11) . Each characteristic is composed of sub-characteristics, totaling 31 sub-characteristics. Given the technical specificities of the maintainability and portability characteristics, these were evaluated only by IT professionals, as pointed out in the literature (14) .
The quality when using a software, or usability, is composed of characteristics and sub-characteristics related to the result of interaction when a product is used in a context of specific use. This system model is applicable to the human-computer interaction system, including using computer systems and using software products. The product quality model, or technical quality, is composed of characteristics and sub-characteristics related to the software's static properties and the system's dynamic properties (13) .
The questionnaire used in this study to assess the technical quality of the system was built on an instrument proposed in a previous study (14) .
The specialists, IT professionals and nurses, In the evaluation of the results, an adapted scale was used with expected values for each characteristic and subcharacteristic; they should be above 70% to be considered satisfactory (14) . The categories and subcategories were evaluated with cutoff rates for weak (25%), regular (50%), good (75%) and excellent (100%) (15) .

Results
In the results' presentation, it was decided to separate the assessment of IT professionals and nurses as the training of these assessors may lead to different criteria in the assessment, according to their field. The characterization of the evaluators is described in Table 1.
In comparison, the nurses were younger and had fewer working years. Regardless of the professional category, most participants had specialist titles ( Table 1).
The assessment of the technical quality's characteristics is described in Figure 1.
All the technical quality categories evaluated were considered adequate, with an evaluation of over 70%.
As for the evaluation of the sub-characteristics, at least one in each characteristic obtained 100% adequacy. Only the sub-characteristic accessibility, of the usability characteristic, obtained an evaluation lower than 70% among IT professionals (Table 2).
www.eerp.usp.br/rlae 4 Rev. Latino-Am. Enfermagem 2020;28:e3330.    among IT professionals. The suitability of these characteristics corresponds to the satisfaction of the users, in order to achieve the objectives of using the system with precision and integrity. Performance efficiency is linked to system response time, considered in this study as satisfactory for both categories of SACR-O in these items (14,16) .
Reliability represents how much the system can deliver the level of performance when exposed for a given time and circumstances, such as being able to perform its tasks reliably with limited resources. For the evaluators, this characteristic was over 90%, which reflects users' confidence in the system. We stress that the reliability of computerized systems has shown higher rates when compared to manual systems (9,14) .
The usability characteristic has been extensively studied in health information system evaluations, as it evaluates the accuracy of the system and the efficiency to users. It often identifies complaints such as slow system response time, too much information on the screen, and changing the workflow (17)(18) .
As for the sub-characteristic accessibility, in the evaluation by IT professionals, the expected value was not reached, being considered as regular. However, it was considered excellent by the nurses. The suggestion pointed out by IT professionals to accessibility was to insert a magnifying glass resource, for users with visual difficulty.
The assessment below expected values reflects the need to reassess system requirements. Due to the time for the development of the study, it was not possible to reevaluate this sub-characteristic after adjustments. Other studies (18) also describe results with fragility in the evaluation of accessibility, related to the resolution and visibility of the screen. Some authors (19) recommend that in order to serve people with disabilities, especially visually impaired ones, health information systems should present visually clear screens, with organized information and easyto-view images.
Also regarding usability, the sub-characteristic error protection obtained 100% adequacy among nurses and IT professionals, which demonstrates that SACR-O offers security in the registration of information.
In the compatibility characteristic, which includes the sub-characteristics interoperability and coexistence, IT professionals evaluated 100% adequacy in both sub-characteristics, being the only characteristic with the maximum consensus value; nurses indicated 90% adequacy for interoperability and 100% for coexistence. This characteristic reflects the ability to exchange information with other systems, for example, the exchange of information with the PER (14,17) .
Interoperability is a concern for health information systems since the lack of integration between systems can compromise patient care (11,14,17) .
The safety characteristic obtained the criterion of excellence in three of its sub-characteristics by the evaluators. This is one of the main features to be considered today since health information systems store a large amount of patients' information and are essential for continuous care. Login and password authentication methods are crucial, as well as the need for effective backup systems, for the ability to store information and reuse it in case of loss or downtime of the main system (20-21) .
The maintainability characteristic, which aims to show the necessary efforts to make specific modifications, and the portability characteristic, that demonstrates the capacity of the system to be transferred to another operational environment, presented rates of adequacy corresponding to data from previous studies, with adequacy higher than 70%, considered satisfactory in this requirement (14,22) .  (23) .
We consider as a study limitation the inadequacy of the system in terms of the sub-characteristic of accessibility in time to be re-evaluated by IT professionals.
The limited number of publications on health information systems evaluation is noteworthy. This review (24) demonstrated the need for further research and dissemination, with emphasis on the use of information systems in nursing.
We highlight the importance of this study to support the decision of managers and developers in building and acquiring reliable systems that improve the quality of health care.

Conclusion
The use of information and communication technologies for decision support has the potential to improve and enhance health care. This study aimed to evaluate the technical quality of SACR-O and the results obtained showed adequacy higher than 87% in all characteristics evaluated, obtaining an excellent criterion.
In the evaluation of the system, the software quality metrics indicated by ISO/IEC were used, with a set of six to eight characteristics, composed by subcharacteristics.
The results achieved from the evaluation indicate that SACR-O has proved adequate in all characteristics.
We stress that at least one sub-characteristic, of each characteristic evaluated, reached 100% of adequacy: