Evaluation of digital vaccine card in nursing practice in vaccination room

Objective: develop and evaluate a vaccine application for mobile devices, with update integrated with the National Immunization Program Information System, for care in vaccination rooms. Method: methodological research based on the Pressman System Development Life Cycle theory developed in three stages: integrative literature review, computational development, and application evaluation. The product was evaluated as to satisfaction, using a validated questionnaire, and as to usability by the System Usability Scale. Results: the application functionalities were based on the survey of technological Innovations on immunization, published in the scientific literature. It displays user vaccines directly from the National Immunization Program Information System, notifies about upcoming vaccines, and enables the inclusion of vaccine cards of dependents. The evaluation resulted in users’ mean score of 90.5 ± 11.1 and health professionals’ mean score of 84.2 ± 19.4. Conclusion: the application is a technological tool with potential to improve the work process in vaccination rooms and to reach the goals of vaccine coverage. It synchronizes data with the National Immunization Program Information System, thus enabling the maintenance of people’s vaccination history.


Introduction
Vaccination is an important public health instrument in the prevention and control of immunopreventable diseases and, therefore, keeping the vaccine card updated and accessible is essential to enable its benefits. However, keeping this card updated is hindered by many barriers, such as lacking knowledge about the importance of immunobiological administration, forgetting scheduled doses, and fearing possible complications related to the vaccine (1)(2) .
In the routine of health services, the vaccine card is presented as a paper document maintained in various types of formats and contents. This may result in problems such as loss of card and difficulty in providing the health professional with access to consistent and reliable information, as these cards are vulnerable to damage, which compromises their validity (3) . One of the ways to solve the problems concerning the maintenance of vaccination records would be the use of mobile devices (4) .
International studies highlight the use of mobile devices comprising electronic and reliable records of people's vaccination history, providing updated vaccination schedule and reminders of future vaccines, thus contributing to improve vaccination coverage rates and organization of the immunization schedule (4)(5)(6)(7)(8) . In Brazil, a mobile device developed with the objective of permanent education of professionals and health education in vaccination was traced in the scientific literature (9) .
Vaccination applications function as a digital vaccination card, registering vaccines and providing information to people (10) . However, a limitation in the use of applications is the reliability of the information provided. Most applications do not automatically update vaccination card records directly from the Immunization Information System (IIS), which makes it difficult to maintain the card because the person needs to manually register their vaccine records, and may compromise the validity of the information. Accordingly, as a criterion for validating this information, applications should be "linked" to the IIS (8,(10)(11) . Integration with the IIS enables the reliability of vaccine information for the health team and guarantees the digital vaccine card as a document to prove the person's vaccination history (8) .
In Brazil, traditionally, nursing assumes the entire work process in the vaccination room. Thus, a digital vaccine card, synchronized with the National Immunization Program IIS, will enable the management of care in vaccination rooms, with decision making about the vaccination situation of the person resulting in safe care, both for the nursing staff and for the person to be vaccinated. Moreover, it may enable greater involvement of the population with issues related to vaccination.  Initially, to enable the application to update with SIPNI vaccine records, we developed the synchronization system called VPM-Sinc, which is composed of two modules ( Figure 1). The first module was developed in desktop environment -for selection of SIPNI vaccine records and storage in an online database -, in Java, due to its portability, which enables installing the software in vaccination rooms with any computer operating system.
The second module, developed in web environment, enables the mobile application to update with SIPNI vaccine records stored in the online database.
The application was evaluated in two stages and with three different groups: professors/researchers, nursing professionals working in the vaccination room, and primary health care users. For selection of professionals, we included all those of the nursing team who worked in the vaccination room and were present in the unit at the time of data collection.
Of the total 16 professionals of the municipality, 13 were present in the health unit and only one nurse did not agree to participate in the study.
In the evaluation with the professors, we When evaluating the final version of the application, participants used the application on a mobile device, exploring its functionality more than once. After becoming familiar with the presented content and its structure, they answered the System Usability Scale -SUS (15) questionnaire, validated in Portuguese in 2010 (16) .
The SUS questionnaire contains ten questions, totaling 100 points, which allow obtaining people's overview about the system. The SUS measurement scale is of the Likert type, whose score ranges from 1 to 5 points.
Participants are asked to answer whether or not they agree with the statements using the options: totally of all scores is multiplied by 2.5, thus obtaining the total SUS value that classifies the system's usability (15) .
After assigning and calculating the score, it is possible to classify the evaluated system: 0 to 50 (not around 85.5 is excellent; around 90.9 is considered best imaginable (17) . After the recognition of the five quality components, the amplitude from 0 to 4 was calculated based on the answers related to each component (16) .

Results
In the application construction stage, the functionalities were based on 9 ITs for mobile devices traced in the literature. Of these, two were developed in the United States of America, two in China, one in Canada, Austria, Kenya, Thailand and Brazil respectively.
In the stage of application evaluation by professors/ researchers, seven judges participated in the study, with a mean age of 44.4 years, the majority of whom were female (57.1%); as for professional training, they were all nurses and had a doctoral degree.
In the evaluation of the satisfaction questionnaire (13) almost all the evaluated criteria obtained a mean equal to or greater than 4.0. Only the "System messages" criterion was evaluated with a 3.43 score. In the specific field for suggestions, they indicated some improvements in the application layout, such as inclusion of background image, harmonization of components with more rounded    The mean SUS score obtained (90.5 ± 11.1) in the evaluation of PHCU users indicated best imaginable usability, which means that the application usability was accepted by the respondents. Based on the users' general information, the SUS score was estimated for each characteristic such as age, sex, smartphone usage frequency, familiarity with smartphone, smartphone operating system, Internet access on the smartphone, Internet usage frequency, and whether they use any vaccination application ( Table 1).
As for age group, the mean SUS score of PHCU

Indications
Newborns preferably in the first 12 hours of life; pregnant women of any age group and gestational age, and individuals at any age who have never received the vaccine.

Contraindications
The vaccine is contraindicated in cases of allergic reactions to the components of the vaccine, manifested after the administration of any dose.

In case there are reactions
You should take note of signs and symptoms, time/duration, intensity, and go to the closest care and safety (20)(21) .
It is important to highlight the use of applications for immunization in other countries. The ImmunizeCA application, developed in the Hospital of Ottawa (Canada), became official with functionalities of information integration and management by the country's immunization information system (4)(5) . The ImmunizeCA reminder function was evaluated in a study and it was found that 36% of the mothers evaluated resorted to this application functionality to monitor the vaccination records of their children, minimizing delays and, consequently, improving vaccine coverage (22) .
A study conducted in San Diego found that reminder system associated with immunization records can be successful in improving vaccine coverage (23) .
A randomized clinical trial, conducted to evaluate vaccine coverage in a rural province of China, using an IT in immunization, enabled tracing children vaccination delays by generating alerts, resulting in a 17% increase in vaccination coverage in the region.
Health professionals reported that the information available through the application favored the continuity of immunization activities (24) .
In  (25) . A frequently reported factor for decreased vaccination is the parents' forgetfulness as to subsequent doses of multi-dose regimens (26) . The technological innovation built enables parents/guardians to monitor the vaccination status of the children/dependents through the functionality of adding dependents' cards. Thus, parents or guardians will be reminded of applied and scheduled vaccines for maintenance of the dependents' vaccine card.
By using the application, individuals are able to track their own vaccinations, being aware of their vaccination status, solving problems in the maintenance of records (4) , and thus avoiding double vaccination and outbreaks of immunopreventable diseases (4,10) . In addition, the users' access to information about their vaccination status contributes to the co-responsibility of care and the development of their autonomy aiming at disease prevention and health promotion. The use of information technologies has transformed the relations between health professionals and patients, mainly expanding access to information and sharing of information related to the health/disease/care process (27) .
For health professionals, information technologies provide greater resolution in vaccine management, and these factors facilitate the promotion of health surveillance actions (5,8) . Furthermore, they support the nursing team in decision making on the administration of vaccines, which avoids waste of doses administered unnecessarily (10) . Accordingly, the decision-making process that is involved in the management of care is favored by the innovation Vacinação na Palma da Mão, because it promotes the organization and systematization of the information that will be the basis for qualified decisions about the care provided to users.
Some applications do not ensure security of the data provided, raising concerns about the truthfulness and quality of the information. One of the mechanisms to overcome this is to have a system that validates the information provided in the applications (8) . In the developed application, the personal data security and network through interoperability (28) . The objective is linking information technologies, both public or private, through a standard of information transmission. Thus, user vaccination-related information may contribute to interventions at the various points of the care network, following the user's path through public and private health services, strengthening the process of referral and counterreferral (29) .
Among the limitations of the study, it is necessary The usability evaluation proved to be satisfactory, and the application can be considered easy to use by people and nursing professionals, with best imaginable and excellent classification, respectively. The impact of this study consists in the use of this innovation as a strategy in improving vaccine coverage and consequently in the control of immunopreventable diseases.