Challenges in the usability of the e-SUS Território application in Brazil’s Digital Health StrategyABSTRACT
The usability of health information systems is highly relevant, especially in Primary Health Care (PHC), a scenario of increasing digitalization and technological integration. In this context, the objective of this study was to evaluate the usability of the e-SUS Território application implemented by the Brazilian Ministry of Health. A sequential explanatory mixed-method approach was used. The System Usability Scale (SUS) was applied in the quantitative phase. An exploratory descriptive study based on the Technology Acceptance Model was conducted in the qualitative phase. The participants were Community Health Workers (CHWs). The SUS score was calculated using Stata statistical software. Qualitative data were organized using Open Logos software, and Thematic-Categorical Analysis was performed. The average SUS score was 55.3 points, indicating marginal acceptance of the application. The results also highlight the application’s usefulness and ease of use. However, synchronization failures with the citizens’ Electronic Health Record (EHR) system were identified as issues, leading to errors in household and individual records, duplication of citizens, and loss of recorded visits. The usability of e-SUS Território was below the expected average, indicating acceptance issues. It is emphasized that usability can be improved through managerial efforts to enhance CHWs’ training, particularly in addressing the needs of users who consider the application to have unacceptable usability.
KEYWORDS
eHealth policies, Electronic health records; Electronic health records; Community Health Workers; Primary Health Care; Digital health.
RESUMO
A usabilidade de sistemas informacionais em saúde apresenta grande relevância, especialmente na Atenção Primária à Saúde (APS), contexto marcado por crescente digitalização e integração de tecnologias. Nesse sentido, objetivou-se avaliar a usabilidade do aplicativo e-SUS Território implementado pelo Ministério da Saúde. Utilizou-se método misto explanatório sequencial. Na abordagem quantitativa, usou-se a Escala de Usabilidade de Sistemas (SUS). Na abordagem qualitativa, empregou-se estudo descritivo exploratório baseado no Modelo de Aceitação de Tecnologia. Os participantes foram Agentes Comunitários de Saúde (ACS). Calculouse o SUS-score com auxílio do software estatístico Stata. Os dados qualitativos foram organizados utilizando-se o software Open Logos, e realizou-se Análise Temática-Categorial. O SUS-score médio encontrado foi 55,3 pontos, indicando aceitação marginal do aplicativo. Os resultados destacam utilidades e facilidades de uso do aplicativo. Falhas na sincronização com o sistema Prontuário Eletrônico do Cidadão foram apontadas como problemas, gerando erros nos cadastros domiciliares e individuais, duplicação de cidadãos e perda de registro de visitas realizadas. A usabilidade do e-SUS Território mostrou-se abaixo da média esperada, apontando problemas em sua aceitação. Ressalta-se que a usabilidade pode ser melhorada com esforços dos gestores na qualificação dos ACS, especialmente no atendimento às necessidades dos usuários que consideram o aplicativo com usabilidade inaceitável.
PALAVRAS-CHAVE
Políticas de eSaúde; Registros eletrônicos de saúde; Agentes Comunitários de Saúde; Atenção Primária à; Saúde; Saúde digital.
Introduction
The e-SUS Primary Care Strategy (e-SUS APS), implemented by the Ministry of Health (MS), has restructured information management in Primary Health Care (PHC). This initiative is part of Brazil’s Digital Health Strategy, which aims to restructure the resources of Digital Information and Communication Technologies (DICTs)1,2. It is important to highlight that the World Health Organization (WHO) encourages the Digital Health movement, which has defined a set of actions and technological resources in the Global Digital Health Strategy3. Integrating digital technologies, especially in PHC, is essential for increasing the Brazilian Unified Health System’s (SUS) efficiency, optimizing processes, expanding access to services, and improving health information management.
In Brazil, a range of initiatives has been undertaken, including the revision of the National Health Information and Informatics Policy, the implementation of the e-Health strategy for Brazil, and the development of the Digital Health Action, Monitoring, and Evaluation Plan, among others. Strengthening information management has significant potential to improve the quality of healthcare service delivery1,2.
The software systems provided through the e-SUS Primary Care (e-SUS APS) strategy enable individualized data collection, clinical record-keeping, and the integration of various Primary Health Care Information Systems (SIS). They also help eliminate redundant data entry, contributing to the digitalization of Primary Health Centers (PHC)1. At the core of this software suite is the Primary Care Information System (SISAB), which is fed by tools made available by the Ministry of Health, such as the Simplified Data Collection system and the citizens’ Electronic Health Record (EHR), as well as other existing or municipally contracted systems1.
The e-SUS APS strategy also provides applications such as e-SUS Território, which is the focus of this study. This application was developed to digitize the recording of activities carried out by Community Health Workers (CHWs). Integrated with the citizens’ Electronic Health Record (EHR), e-SUS Território was designed for mobile devices to facilitate the CHWs’ workflow.
The adoption of mobile applications in healthcare, particularly in the context of Primary Health Care, enables real-time, location-independent access to information, thereby supporting the evolution of a more dynamic and patient-centered model of care5,6. Nevertheless, a range of factors may impact the usability of these technologies, including age, educational attainment, prior digital experience, and the availability of technical support - all of which can shape user perceptions and potentially hinder the effective use of these tools7-10.
It is important to note that usability is a quality attribute that assesses how easy it is for users to interact with interfaces, considering aspects such as learning capacity, efficiency, memorization, error frequency, and user satisfaction with the system11. The Community Health Worker, a key professional within PHC, performs the majority of their duties outside the Primary Health Centers. Therefore, the use of this application has the potential to positively influence their work processes. In this context, it is essential to examine the factors associated with the usability and acceptance of this technological innovation by these professionals.
Local initiatives involving the integration of mobile devices into the work of Community Health Workers have shown that these professionals report high levels of satisfaction, along with improved working conditions and greater reliability of health information12. However, the nationwide digitalization of CHWs’ work is a recent initiative in Brazil, producing significant changes in daily work and justifying the need to evaluate the usability and acceptance of the system.
Therefore, the present study aims to conduct a targeted assessment to better understand the usability and acceptance of the e-SUS Território application by the Community Health Workers. Specifically, it investigates the usability challenges associated with the app, observing how its interface may influence the efficiency of CHWs’ workflows and the quality of health data recording. The study’s findings are intended to contribute to the ongoing discourse on the adoption of Digital Information and Communication Technologies (DICTs) in the health sector, with particular emphasis on the digital transformation of CHWs’ activities and the improvement of health information systems within the framework of the Brazilian Unified Health System (SUS).
Material and methods
This study employed a sequential explanatory mixed-methods design13, in which the quantitative approach was conducted using a cross-sectional design14. The qualitative approach followed an exploratory-descriptive approach, guided by the Technology Acceptance Model (TAM)15 (box 1) as the theoretical framework. This model is widely recognized for its validity, applicability, and replicability across diverse contexts and is considered a robust model for describing the influence of both external and internal factors on users of information systems15,16.
TAM is grounded in two core constructs: perceived usefulness and perceived ease of use. Perceived usefulness refers to the extent to which a user believes that using a system will enhance their performance. In contrast, perceived ease of use relates to the degree to which a user believes that interacting with an information system will be effortless15.
The qualitative data collected in this study aim to enrich the interpretation of the quantitative results, offering stronger evidence on the usability of the e-SUS Território application by CHWs. This complementary approach facilitates a more comprehensive understanding of the perceptions, experiences, and contextual factors that influence the acceptance and use of the system among PHC professionals.
The quantitative study was conducted with Community Health Workers using the e-SUS Território application in a health macro-region of the state of Minas Gerais, comprising 53 municipalities, of which 27 (50.9%) were already using the application. According to the National Registry of Health Establishments, 611 CHWs worked in these 27 municipalities. The sample size was calculated assuming maximum variability for the frequency of the studied events (p = 0.5), a 95% confidence level, and a 5% sampling error. Therefore, the study population was required to include at least 237 Community Health Workers, with an inclusion criterion of a minimum of three months’ experience using the application, excluding those on vacation, maternity leave, or leave for other reasons during the research period.
Quantitative data collection was conducted online (web survey). The instrument used was the System Usability Scale (SUS), a version adapted and validated for the Brazilian context17,18. The scale consists of ten items, five of which positively worded and five negatively worded, rated on a five-point Likert scale, where 1 corresponds to ‘strongly disagree’, 2 to ‘disagree’, 3 to ‘neither agree nor disagree,’ 4 to ‘agree’, and 5 to ‘strongly agree’17,18. The SUS scale is widely recognized and frequently used to assess the usability of information systems11,20.
Participants were contacted through the Municipal Health Secretaries, using email and social media messages, which included information about the study and an invitation to participate. Three consecutive rounds of invitations were sent, with a seven-day interval between each one. Access to the data collection instruments was provided electronically via the Google Forms platform. A total of 254 Community Health Workers responded to the survey.
The data collected were stored and processed using Stata 14.0 statistical software. Initially, a descriptive analysis of the variables was conducted. Subsequently, the SUS score was calculated - a composite measure derived from the individual contributions of each item on the instrument. For the odd-numbered items, 1 was subtracted from the participant’s response; for even-numbered items, the participant’s response was subtracted from 5. After this calculation, the sum of the ten items was multiplied by 2.5. The resulting SUS score can range from 0 to 100 points, which is used to classify the usability level of the system under evaluation17.
According to the SUS score scale, the usability of a technology is classified as ‘not acceptable’ if the score falls between 0 and < 50 points. A score between 50 and < 70 points is considered ‘marginally acceptable’, while a SUS score ≥ 70 indicates ‘acceptable’ usability17. A mean score of 68 points or higher is often used as a benchmark for adequate usability, representing the midpoint at which half of all applications score below and half score above this value11,21.
In the qualitative approach of this study, data were collected from the Google Play Store platform, where users of the e-SUS Território application had posted comments and reviews. This space is particularly valuable as it directly reflects user acceptance or rejection of the application, while also revealing their perceptions of use, including reported difficulties and facilitating factors.
The dataset for this study consisted of 1,066 user comments on the e-SUS Território application, covering versions 1.0 through 3.2. All comments were initially reviewed, with illegible, blank, or duplicate entries excluded from the analysis. Subsequently, the Categorical Thematic Analysis technique of Content Analysis was applied. This process was carried out in three distinct phases, as outlined in the methodology section19.
In the first phase, a preliminary reading of the data was conducted to identify emerging hypotheses and theories related to the topic. The quality of this process was ensured through the application of criteria such as exhaustiveness, homogeneity, exclusivity, objectivity, and the appropriateness of the collected data19.
In the second phase, the analytical categories were established, based on the theoretical framework of the Technology Acceptance Model15. The predefined categories included perceived usefulness and perceived ease of use. Perceived usefulness assesses the extent to which users believe the application can enhance their job performance, while perceived ease of use explores users’ perceptions regarding how easy or difficult the technology is to use15,16.
In the third and final phase, data segments from the analyzed material were systematically grouped to deepen the exploration of the emerging hypotheses. The categorization was guided by the presence or absence of perceived ease of use and perceived usefulness to interpret the meaning of the user comments through a reflective and critical lens19.
It is important to highlight that this study adhered to strict ethical standards involving research with human subjects and was approved by the Research Ethics Committee of the Federal University of São João del-Rei, under the Presentation Certificate for Ethical Appraisal (CAAE) No. 37555620.9.0000.5545 and Opinion No. 4.523.507. All ethical considerations, including the confidentiality and privacy of participants, were ensured following Resolution No. 466/201222 of the National Health Council.
Results
The majority of participants in the quantitative phase were female (92.5%), married or in a stable relationship (58.7%), identified as white (52.4%), had completed secondary education (76.0%), and reported a household income of up to three minimum wages (86.6%) (table 1). The age variable showed a normal distribution (Shapiro-Wilk normality test), with a median of 38 years, ranging from 30 to 45 years old. Regarding professional experience as Community Health Workers, 74% had more than two years of experience, and the majority (75%) had been using the application for over one year (table 1).
Participation in computer training was reported by 68.5% of the participants, and 70.9% stated that they had received training to use the application and felt confident in their ability to operate it. The most commonly used device to access the application was the tablet (table 1).
The average SUS score was 55.3, falling below the commonly referenced benchmark of 68 points typically used in the literature to evaluate application usability. This suggests that, on average, participants perceived the usability of the e-SUS Território app as sub-optimal.
When examined individually, the SUS scores revealed that 49.4% of participants rated the app’s usability as ‘marginally acceptable’, with scores between 50 and < 70 points; 31.1% rated it as ‘not acceptable’, with scores below 50; and only 19.5% considered the usability ‘acceptable’, with scores equal to or greater than 70 points (table 2).
These findings reveal that a significant portion of users perceived the usability of the e-SUS Território app as marginal or below expectations, highlighting areas that may require improvement to better meet the needs and expectations of Community Health Workers.
Graph 1 presents the average score for each of the evaluated variables. Variable 6 - ‘I thought e-SUS Território had many inconsistencies’ - received the lowest average score. Qualitative analysis further supported this perception, with comments emphasizing this issue. The most frequently reported challenges were related to inconsistencies within the app, particularly those involving synchronization with the Picture Exchange Communication System (PECS).
Failures in the app’s synchronization with the PEC system are cited as a cause of operational issues, such as errors in household and individual records, duplication of citizen entries, and loss of visit records:
[...] the app is experiencing a malfunction: during synchronization with the PEC system, some families are missing. While everything appears correct in the system, various errors appear in the app [...]. (C423).
[...] all visits and registrations have disappeared [...]. (C692).
Such issues result in additional workload for Community Health Workers, leading to user dissatisfaction and potentially undermining the overall usability of the app, as evidenced by the findings.:
[...] even though I’ve followed the synchronization instructions step by step, I’ve lost count of how many times I’ve had to redo my entries. [...] unfortunately, I’m left with no choice but to return to using paper forms, as they’re more reliable and practical. (C62).
In some cases, synchronization problems are linked to app updates: “It won’t let me sync because it asks for an update, but when I try to update, it says the app is already updated” (C105).
Another variable that received a low average score was “I found the e-SUS Território app more complex than necessary” (graph 1). Users identify a certain disorganization of the family and citizen registers, indicating the complexity of the app’s use. They point out that the records are not organized by micro-areas: “[...] families are not separated by CHW, everything is mixed with other CHWs” (C67).
Another key concern raised by users regarding the complexity of the application is the restriction placed on Community Health Workers from merging records or resolving issues related to updates and duplicate entries. When these inconsistencies are identified, CHWs must rely on nurses to address them, which diminishes their perception of the application’s ease of use.
[…] since duplicate records and users are a recurring issue, I suggest that we, as Community Health Workers, be allowed to merge duplicate records to prevent inconsistencies in the system […]. (C282).
Users also noted that completing certain fields during registration makes the app more difficult to use. One example is the mandatory requirement to enter a mobile phone number: “It’s wrong to require users’ phone numbers, because many won’t give them and others don’t even have one” (C261). Another commonly cited issue is the occupation field: “[...] searching for the citizens’ occupation can be time-consuming at times [...]” (C343). Users also reported difficulties in accessing information or routine reports frequently used by the Community Health Workers:
[…] when a CHW needs access to individual-level data, the system does not provide it, forcing them to rely on manual, paper-based counts. (C202).
There is a lack of a prioritized summary for each worker by micro-area, highlighting key groups such as children under one year old, children aged two to five, individuals with hypertension or diabetes, pregnant women, and postpartum women [...]. (C947).
There were also comments reinforcing the perception that the use of the e-SUS Território system is complex, with some users questioning its usefulness: “Terrible - it only gets in the way of the Community Health Workers” (C469).
On the other hand, the analyzed comments show that some users had a positive assessment of the app’s use. Statements such as “[...] It was everything I wanted as a work tool” (C507), “[...] The app is a great tool for CHWs” (C152), and “[...] Very useful app for me as a health worker” (C214) highlight the app as a facilitator in their daily work. There were also reports indicating that the app’s layout contributes to its ease of use.
The comments also show that the CHWs recognize that using the app can speed up and better organize their work: “Very useful app, it made the work much more organized and productive” (C459).
Another perceived advantage is the reduction in paper use, since registrations and visits are entered directly into the app: “No paper or pen, everything is done directly” (C114). The app’s usefulness was also noted in areas with limited internet access: “Very useful app for me as a CHW working in a rural area” (C214).
Discussion
The results show that, despite some recognized utilities and conveniences in using e-SUS Território, its usability was found to be inadequate, which may compromise its acceptance by CHWs. It is noteworthy that the significant increase in the digitization of health services through the widespread adoption of Digital Information and Communication Technologies (DICTs) has brought about major changes in the production, recording, and dissemination of data and information in the health sector2,7.
Thus, the use of technological resources to optimize and improve service quality is already a reality in many areas; however, these changes had not yet reached the CHWs until 2016, when e-SUS Território was made available throughout Brazil. Overall, e-SUS Território has the potential to facilitate the work processes of CHWs by simplifying data collection4.
The usability analysis of this application, based on the SUS score, identified that this technological innovation scored below the expected average. It is understood that even the best applications do not guarantee satisfactory usability for all users, especially considering this is the first time this technology has been adopted on a wide scale. It is also noteworthy that a SUS score of 68 points or higher has proven to be an excellent benchmark for the quality of digital health applications11.
The use of mobile devices has already proven to be more challenging for CHWs, mainly due to their professional profiles and limited experience with technological tools23. There is evidence that CHWs generally have limited knowledge in using mobile devices and require basic computer training, in addition to facing difficulties accessing computers and the internet12. Therefore, it is natural for problems to arise during this initial phase, highlighting the need to invest in digital inclusion processes.
The Ministry of Health recommends starting the implementation of this technology with professional training following a step-by-step approach, covering preparation for use, simulation, and supervised on-site operation. Continuous training and technical support are necessary, given that the systems frequently undergo updates. Therefore, CHWs need to be well-prepared and capable of using technologies throughout the entire process, from data capture to data analysis23.
Studies indicate that limited familiarity with the use of DICTs hinders the usability of mobile devices12,24. In this context, the use of mobile devices is perceived as unusual and disconnected from the reality of some CHWs. They feel uncomfortable approaching people while carrying these devices due to their reactions. This contributes to greater resistance and distrust toward the use of this technology25.
Overall, participants reported limited improvements in performance and ease of use. This is likely because the technology is new to the daily workflow of Community Health Workers. It is worth noting that there is evidence that individuals are more inclined to adopt technologies when they perceive them as both useful and user-friendly15,26. The acceptance of mobile technology also depends on confidence in handling it, voluntary use, comfort and ease, as well as speed, all of which contribute positively to productivity29.
Community Health Workers play a crucial role in data collection within PHC, where accurate information is fundamental to their work. They manage a large volume of data that must be exclusively entered by them, and a frequent complaint is the excessive burden of form-filling25. Thus, the effective implementation of e-SUS Território is essential not only to optimize their work time but also to enhance the quality of the data collected.
It is essential to recognize that innovation should not be exclusive to technical professionals. The users of the application in their daily work must be involved in the process and have the opportunity to experience the innovation before its implementation. Additionally, it is crucial to allocate time to organize teams and work processes. Implementing a new DICT, such as e-SUS Território, abruptly and without involving the Community Health Workers, tends to increase resistance to its adoption15,25,30.
Conclusions
The usability of the e-SUS Território app, as measured by the SUS score, fell below the expected average, indicating issues with its acceptance. It is important to stress that usability must be improved through management efforts focused on training the CHWs in using the app, especially to better meet the needs of users who consider the app’s usability unacceptable. There is also a need to enhance these professionals’ perception of the app’s usefulness and ease of use. Given the diverse contexts of use, the adoption and acceptance of DICTs is not an easy task and requires significant effort from management to succeed.
The findings of this study offer important guidance for enhancing both the e-SUS Território app and the work practices of Community Health Workers. Nonetheless, it is crucial to address identified challenges by implementing necessary adjustments, such as increasing user autonomy, improving synchronization to minimize data loss, and sustaining ongoing training programs alongside effective support channels to resolve user concerns.
Finally, it is necessary to reassess the app’s usability in the future, considering that it may improve as users gain more experience and as system updates are implemented. Longer periods of use tend to enhance user familiarity, thereby reducing initial difficulties. Furthermore, updates can fix errors and incorporate features that make e-SUS Território more intuitive and efficient.
Data availability:
Research data are contained in the manuscript itself
-
Financial support:
Non-existent
References
-
1 Ministério da Saúde (BR). Estratégia de saúde digital para o Brasil 2020-2028 [Internet]. Brasília, DF: Ministério da Saúde; 2020 [acesso em 2023 fev 9]. 129 p. Disponível em: https://bvsms.saude.gov.br/bvs/publicacoes/estrategia_saude_digital_Brasil.pdf
» https://bvsms.saude.gov.br/bvs/publicacoes/estrategia_saude_digital_Brasil.pdf -
2 Gontijo TL, Lima PKM, Guimarães EAA, et al. Computerization of primary health care: the manager as a change agent. Rev Bras Enferm. 2021;74(2):e20180855. DOI: https://doi.org/10.1590/0034-7167-2018-0855
» https://doi.org/10.1590/0034-7167-2018-0855 -
3 World Health Organization. Global strategy on digital health 2020-2025 [Internet]. Geneva: WHO; 2021 [acesso em 2023 fev 2]. 60 p. Disponível em: https://www.who.int/docs/default-source/documents/gs4dhdaa2a9f352b0445bafbc79ca799dce4d.pdf
» https://www.who.int/docs/default-source/documents/gs4dhdaa2a9f352b0445bafbc79ca799dce4d.pdf -
4 Ministério da Saúde (BR). e-SUS atenção primária à saúde: Manual do aplicativo e-SUS território [Internet]. Brasília, DF: Ministério da Saúde; 2022 [acesso em 2023 fev 2]. 56 p. Disponível em: https://sisaps.saude.gov.br/esus/upload/docs/manual_utilizacao_aplicativo_esus_aps_territorio.pdf
» https://sisaps.saude.gov.br/esus/upload/docs/manual_utilizacao_aplicativo_esus_aps_territorio.pdf -
5 Zhou L, Bao J, Setiawan IMA, et al. The mHealth app usability questionnaire (MAUQ): development and validation study. JMIR Mhealth e Uhealth. 2019;7(4):e11500. DOI: https://doi.org/10.2196/11500
» https://doi.org/10.2196/11500 -
6 Netto JT, Rodrigues NCP, Souza BNP, et al. Tecnologia digital para o enfrentamento da Covid-19: um estudo de caso na atenção primária. Saúde debate. 2022;45:56-67. DOI: https://doi.org/10.1590/0103-11042021E204
» https://doi.org/10.1590/0103-11042021E204 -
7 Cavalcante RB, Vasconcelos DD, Gontijo TL, et al. Computerization of primary health care information systems: advances and challenges. Cogitare Enferm. 2018;23(3):e54297. DOI: https://doi.org/10.5380/ce.v23i3.54297
» https://doi.org/10.5380/ce.v23i3.54297 -
8 Melnick ER, Dyrbye LN, Sinsky CA, et al. The Association Between Perceived Electronic Health Record Usability and Professional Burnout Among US Physicians. Mayo Clin Proc Dígito Saúde. 2020;95(3):476-87. DOI: https://doi.org/10.1016/j.mayocp.2019.09.024
» https://doi.org/10.1016/j.mayocp.2019.09.024 -
9 Tajirian T, Stergiopoulos V, Strudwick G, et al. The Influence of Electronic Health Record Use on Physician Burnout: Cross-Sectional Survey. J Med Internet Res. 2020;22(7):e19274. DOI: https://doi.org/10.2196/19274
» https://doi.org/10.2196/19274 -
10 Vehko T, Hypponen H, Puttonen S, et al. Experienced time pressure and stress: electronic health records usability and information technology competence play a role. BMC Med Inform Decis Mak. 2019;19(1):1-9. DOI: https://doi.org/10.1186/s12911-019-0891-z
» https://doi.org/10.1186/s12911-019-0891-z -
11 Hyzy M, Bond R, Mulvenna M, et al. Escala de usabilidade do sistema Benchmarking para aplicativos de saúde digital: meta-análise. JMIR Mhealth Uhealth. 2022;10(8):e37290. DOI: https://doi.org/10.2196/37290
» https://doi.org/10.2196/37290 -
12 Sousa PHL, Pelogi APS. Use of mobile by community health workers. Rev Enferm Atenção Saúde. 2018;7(1):225-33. DOI: https://doi.org/10.18554/reas.v7i1.2126
» https://doi.org/10.18554/reas.v7i1.2126 - 13 Creswell JW, Creswell JD. Projeto de pesquisa: métodos qualitativo, quantitativo e misto. Porto Alegre: Penso Editora; 2021. 248 p.
- 14 Gordis L. Epidemiologia. Rio de Janeiro: Editora Revinter; 2017. 404 p.
-
15 Davis FD. Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly. 1989;13(3):319-40. DOI: https://doi.org/10.2307/249008
» https://doi.org/10.2307/249008 - 16 Parreira P, Proença S, Sousa LB, et al. Technology assessment model (TAM): modelos percursores e modelos evolutivos. In: Parreira P, Alves L, Mónico L, et al., organizadores. Competências empreendedoras no ensino superior politécnico: motivos, influências, serviços de apoio e educação. Guarda: Instituto Politécnico da Guarda; 2018. p. 143-63.
- 17 Brooke J. A “quick and dirty” usability scale. In: Jordan PW, Thomas B, Weerdmeester BA, et al., organizadores. Usability evaluation in industry. Londres: Taylor and Francis; 1996. p. 107-114.
-
18 Tenório JM, Cohrs FM, Sdepanian VL, et al. Desenvolvimento e avaliação de um protocolo eletrônico para atendimento e monitoramento do paciente com doença celíaca. Rev Inform Teór Apl. 2010;17(2):210-20. DOI: https://doi.org/10.22456/2175-2745.12119
» https://doi.org/10.22456/2175-2745.12119 - 19 Bardin L. Análise de conteúdo. 4ª ed. Lisboa: Ed. 70; 2018. 288 p.
-
20 Maramba I, Chatterjee A, Newman C. Métodos de teste de usabilidade no desenvolvimento de aplicativos de eSaúde: uma revisão de escopo. Int J Behav Med. 2019;126:95-104. DOI: https://doi.org/10.1016/j.ijmedinf.2019.03.018
» https://doi.org/10.1016/j.ijmedinf.2019.03.018 - 21 Sauro J, Lewis J. Quantificando a experiência do usuário: estatísticas práticas para pesquisa do usuário. Burlington: Elsevier/Morgan Kaufmann; 2016. 295 p.
-
22 Ministério da Saúde (BR); Conselho Nacional de Saúde. Resolução nº 466, de 12 dezembro de 2012. Aprova as diretrizes e normas regulamentadoras de pesquisas envolvendo seres humanos. Diário Oficial da União [Internet], Brasília, DF. 2013 jun 13 [acesso em 2025 fev 26]; Edição 112; Seção I:59-62. Disponível em: http://conselho.saude.gov.br/resolucoes/2012/Reso466.pdf
» http://conselho.saude.gov.br/resolucoes/2012/Reso466.pdf -
23 Abreu FDL, Zanin NB, Bissaco MAS, et al. Percepções dos agentes comunitários de saúde sobre as tecnologias de informação e comunicação na atenção primária à saúde: uma pesquisa exploratória. Rev Humanid Inov [Internet]. 2020 [acesso em 2023 mar 21];8(5):32-45. Disponível em: https://revista.unitins.br/index.php/humanidadeseinovacao/article/view/2720
» https://revista.unitins.br/index.php/humanidadeseinovacao/article/view/2720 -
24 Weber GA, Frantz MI, Rocha PFA. Uso dos tablets como recurso para melhorar a captação de dados pelas agentes de saúde da unidade sede de Cunhata [monografia na Internet]. Florianópolis: Instituto Federal de Santa Catarina; 2019 [acesso em 2023 mar 21]. 17 p. Disponível em: https://repositorio.ifsc.edu.br/handle/123456789/1893
» https://repositorio.ifsc.edu.br/handle/123456789/1893 -
25 Vasconcelos DD, Braz PR, Gontijo TL, et al. Implantação e utilização de dispositivo móvel na Atenção Primária à Saúde no Brasil. Rev Cub Inf Cienc Salud [Internet]. 2021 [acesso em 2023 mar 21];32(4):e1838. Disponível em: https://pesquisa.bvsalud.org/portal/resource/pt/biblio-1408099
» https://pesquisa.bvsalud.org/portal/resource/pt/biblio-1408099 -
26 Youself AMF, Sumner T. Reflexões sobre a última década de pesquisa MOOC. Computer Appl Eng Educ. 2021;29(4):645-7. DOI: https://doi.org/10.1002/cae.22334
» https://doi.org/10.1002/cae.22334 -
27 Ammenwerth E. Technology acceptance models in health informatics: TAM and UTAUT. Stud Health Technol Informe. 2019;263:64-71. DOI: https://doi.org/10.3233/SHTI190111
» https://doi.org/10.3233/SHTI190111 -
28 Thies K, Anderson D, Cramer B. Lack of adoption of a mobile app to support patient self-management of diabetes and hypertension in a federally qualified health center: interview analysis of staff and patients in a failed randomized trial. JMIR Hum Factors. 2017;4(4):e24. DOI: https://doi.org/10.2196/humanfactors.7709
» https://doi.org/10.2196/humanfactors.7709 - 29 Mangkunegara C, Azzahro F, Handayani P. Analysis of factors afecting user’s intention in using mobile health application: a case study of halodoc. In: Anais do International Conference on Advanced Computer Science and Information Systems; 2018; Yogyakarta. Indonésia. Yogyakarta, Indonésia: IEEE; 2018. p. 87-92.
-
30 Silva TIM, Cavalcante RB, Santos RC, et al. Diffusion of the e-SUS Primary Care innovation in family health teams. Rev Bras Enferm. 2018;71(6):2945-52. DOI: https://doi.org/10.1590/0034-7167-2018-0053
» https://doi.org/10.1590/0034-7167-2018-0053
-
Editor in charge:
Mônica Ferreira da Silva
Publication Dates
-
Publication in this collection
01 Sept 2025 -
Date of issue
2025
History
-
Received
28 Aug 2024 -
Accepted
29 Apr 2025
Source: Own elaboration.1 - I thought I would like to use the e-SUS Território frequently; 2 - I found the e-SUS Território was more complex than necessary; 3 - I thought the e-SUS Território was easy to use; 4 - I thought I would need help from a technician to be able to use it; 5 - I thought the various functionalities of the e-SUS Território were well integrated; 6 - I thought the e-SUS Território had a lot of inconsistencies; 7 - I suppose most people would learn how to use it; 8 - I considered the e-SUS Território very complicated to use; 9 - I felt confident using the e-SUS Território; 10 - I had to learn a lot before I could get to grips with the e-SUS Território.