Development and evaluation of an application for syphilis control

Desarrollo y evaluación de un aplicación para el control de la sífilis en gestantes

ABSTRACT

Objective:

to develop and evaluate an application for syphilis control in pregnant women.

Method:

methodological research developed between March and November of 2016 in two phases: bibliographic survey of the years 2012 to 2016 in the databases PubMed, CAPES and Scopus and application development. Eight users participated in the usability test and five doctors and five nurses working in prenatal care participated in the evaluation.

Results:

the application contains informative video, information about the disease, map of health clinics, agenda function and anonymous notification. The evaluation of the objective, function and relevance was considered adequate with value higher than 0.80 in all items of the Content Validity Index.

Final considerations:

the application makes easier the routine of health services in the context of health promotion, in the convocation and treatment of pregnant women and their partners.

Descriptors:
Development; Evaluation; Syphilis; Technology; Contact Tracing

RESUMEN

Objetivo:

desarrollar y evaluar una aplicación para el control de la sífilis en gestantes.

Método:

Investigación metodológica desarrollada en el período de marzo a noviembre de 2016, en dos fases: levantamiento bibliográfico de los años 2012 a 2016 en las bases de datos PubMed, CAPES y Scopus y desarrollo de la aplicación. Participaron de la prueba de usabilidad ocho usuarios y de la evaluación cinco médicos y cinco enfermeras que actuaban en el prenatal.

Resultados:

la aplicación contiene vídeo informativo, información sobre la enfermedad, mapa de los puestos de salud, función de agenda y notificación anónima. La evaluación del objetivo, función y relevancia se consideró adecuada con un valor superior a 0,80 en todos los ítems del Índice de Validación de Contenido.

Consideraciones finales:

la aplicación facilita la rutina de los servicios de salud en el contexto de promoción de la salud, en la convocatoria y tratamiento de gestantes y sus parejas.

Descriptores:
Desarrollo; Evaluación; Sífilis; Tecnología; Trazado de Contacto

RESUMO

Objetivo:

desenvolver e avaliar um aplicativo para o controle da sífilis em gestantes.

Método:

pesquisa metodológica desenvolvida no período de março a novembro de 2016, em duas fases: levantamento bibliográfico dos anos de 2012 a 2016 nas bases de dados PubMed, CAPES e Scopus e desenvolvimento do aplicativo. Participaram do teste de usabilidade: oito usuários e da avaliação cinco médicas e cinco enfermeiras que atuavam no pré-natal.

Resultados:

o aplicativo contém vídeo informativo, informações sobre a doença, mapa dos postos de saúde, função de agenda e notificação anônima. A avaliação do objetivo, função e relevância foi considerada adequada com valor superior a 0,80 em todos os itens do Índice de Validação de Conteúdo.

Considerações finais:

o aplicativo facilita a rotina dos serviços de saúde no contexto de promoção da saúde, na convocação e tratamento de gestantes e seus parceiros.

Descritores:
Desenvolvimento; Avaliação; Sífilis; Tecnologia; Notificação do Parceiro

INTRODUCTION

Syphilis is a Public Health problem because of the difficulties in accessing proper treatment⁽¹⁾, of limited resources, stigma, poor quality of health services and unsatisfactory follow-up of the sexual partners⁽²⁾. Pregnant women with syphilis, untreated or inadequately treated, transmit the infection to the child, which born with Congenital Syphilis (CS)⁽¹⁾. Compulsory notification of syphilis has been adopted in 84 countries around the world. In 2017, only 26 countries reported that 95% of pregnant women underwent testing for syphilis during prenatal consultations⁽³⁾.

In Brazil, according to data from the Syphilis Epidemiological Bulletin, from January 2010 to June 2016, 227,663 cases of Acquired Syphilis (AS) were registered. The incidence of CS in children under one year increased from 1.7 cases/1,000 Live Birth (LB) in 2004 to 6.5 cases/1,000 LB in 2015^(1,4).

In view of the above, health professionals use new strategies and technologies to reduce these rates. Studies show that the technology mHealth can assist HIV-infected women⁽⁵⁾, to adherence to antiretroviral therapy⁽⁶⁾ and to prevention of vertical transmission of the virus⁽⁷⁾.

MHealth is defined by the World Health Organization (WHO) as a component of the electronic health (eHealth) which includes public health medical practices supported by mobile devices such as cell phones, patient monitoring devices, personal digital assistants, among others wireless equipment⁽⁸⁾.

MHealth has the potential to address many of the challenges that developing countries face, including labor shortages, lack of information on health by the population, limited training of health professionals and difficulty in tracking patients⁽⁹⁾.

During prenatal, studies show the utility of mHealth for health education, appointments reminders, and communication with the health worker by sending text messages^(10-11).

Due to the high number of cases of SA and SC, the failures of the Primary Health Care network and the potentialities of mHealth technology with contextualized information, it has a clear and didactic language that will help to expand the knowledge about syphilis for the pregnant women and will support the search for their partner(s) for the treatment. This paper describes, however, the development and evaluation of an application to optimize the care provided to pregnant women with syphilis during prenatal consultations.

OBJECTIVE

To develop and evaluate an application for syphilis control in pregnant women.

METHOD

Ethical aspects

This research was approved by the Research Ethics Committee (CEP) of the Universidade de Fortaleza (UNIFOR), in compliance with the provision of Resolution 466 of December 12, 2012, of the Brazilian Health Board (Conselho Nacional de Saúde), which regulates research involving human beings⁽¹²⁾.

Type of study

This is a methodological research aimed at the verification of methods and procedures adopted as scientific and require knowledge of psychometric techniques. This technique deals with the theory and development of instruments such as questionnaires or measurement techniques⁽¹³⁾.

Methodological procedures

The application called SELP – a term that originated from the junction of the syphilis S and ELP letters of the word help –, was elaborated in the Laboratory of Technological Innovation of a university in the city of Fortaleza, Ceará, Brazil.

The application was developed in two stages: In the first, was carried out a bibliographical study of the articles on the subject and a survey of syphilis applications available in stores Apple Store and Google Play; In the second, the conception, development and evaluation with the health experts, on the basis of the Participatory Design of Interaction⁽¹⁴⁾, which considers the epistemological approach of the, design process in which users act as experts according to work practice, language refinement, and the development process⁽¹⁵⁾.

First stage: Bibliographical survey and of the available applications

In the first stage, a research was carried out in the PubMed, Periodic of CAPES and Scopus databases. 15 articles were selected, published between 2012 and 2016, using the descriptors “sexually transmitted infection” and “applications”, combined by the Boolean connector AND, in English, Portuguese and Spanish. To identify syphilis-related applications available in the market, a survey was conducted at two online stores of applications – Google Play and Apple Store – using the following terms: Syphilis, Sífilis, Sexually Transmitted Diseases and Doenças Sexualmente Transmissíveis.

Second stage: Development and evaluation of the application

In the second stage, meetings of brainstorms were held with a multidisciplinary team that aimed to the design of the technological artifact called SELP. In this procedure, it was used the Human-Computer Interaction (HCI) related to the Participatory Design of Interaction⁽¹⁴⁾, to guide and identify the necessary requirements of the tool.

The Participatory Design of Interaction is subdivided into four stages: 1- to identify the users’ needs and establish the requirements to develop the technological artifact; 2- propose the design (color palette, typography, iconography and elements of interaction); 3- to redesign (discussion and review by design team proposed before beginning the codification of the technological artifact) and to formulate an interactive version (functional prototype); 4- to evaluate the version developed in the laboratory.

Since the first stage of the design process, it was identified the needs of the users and established the requirements of the technological artifact to be coded.

In the second stage, it was started the design activity in which teams of computer engineering, computer science and audiovisual and new media worked together to create interface (system screen) designs (interface drafts) of low-fidelity (interface drafts) and high fidelity (drawings already with typography, icons and palette of colors), as a possible solution for the final development of the artifact. Still in this stage, was made a video content that addresses the main concepts about syphilis.

The third stage refers to the redesign of the application, in which the usability test is applied with representatives of the target population, which aims to understand the interaction of the user with the product and to correct possible flaws.

The eight participants were selected for convenience, invited by direct call, all males, aged 25 to 39 years, in a stable or consensual union, with high school education, of low middle class, users of the Unified Health System (UHS) and general service workers of a higher education institution.

It was considered this number of participants that, according to Nielsen (1994)⁽¹⁶⁾, must be at least three and at most five to evaluate the authenticity, functionality and user satisfaction with the technological artifact.

The usability test was conducted by a nurse and a computer engineer, who delivered participants a smartphone and a form with five activities evaluated from a Likert scale⁽¹⁷⁾.

The activities contained in the form guided the assessment team and established the user’s level of difficulty in completing personal data, watching the video and finding syphilis information, identifying the health clinics and scheduling treatment, and notifying a sexual partner.

After interaction with the application, participants described their opinion on five open questions. Users were asked if they would use the app to notify a sexual partner, what they liked and disliked about the device, and what they could improve.

To protect the identity of the participants, the letter “H” was used, followed by the numbers one through eight. All the participants agreed to participate in the study and signed an informed consent form.

In the fourth stage, it was measured the content, functionality and relevance of the application with five doctors and five expert nurses, masters and doctors with employment contracts for more than one year in Primary Health Care and that assisted the pregnant women. The test was performed in October 2016 at BlueLab, Laboratory of the UNIFOR.

It was visited the three Basic Health Units and two hospitals, reference in the care of pregnant women in the Municipality of Fortaleza, Ceará. The professionals were personally invited in their workplace to participate in the evaluation of the application and a letter was sent with the title of the research, objectives, time, place and date of data collection.

To carry out the evaluation, each expert received a smartphone with the application installed for free handling and an evaluation form with objective questions corresponding to the content, objective and relevance. It was calculated the Content Validity Index (CVI), in which the Likert scale value ranged from 1 to 5 (1, no relevance, up to 5, totally relevant) and the calculation was performed by the sum of agreement of the answers marked by “4 or 5”. For the concordance rate of the evaluated items, it was adopted the value from 0.78 to 1⁽¹⁸⁾.

Data analysis

Quantitative data from the usability test with users and experts were analyzed using simple descriptive statistics using the software Microsoft Excel^®. To the qualitative approach with the users, a content analysis⁽¹⁹⁾ was performed based on the participants’ answers, division into units, categorization and empirical analysis of the categories.

RESULTS

Evidence found in the literature

The analysis of the 12 articles found shows that the study population is interested in using applications that clarify their doubts about the treatment, symptoms and prevention of Sexually Transmitted Infections (STIs). Among the possible functions to compose an application, it is highlighted information on the treatment, transmission of STI, history of risky sexual behavior, information about condom use, location of health service, treatment associated with the use of alcohol and drugs.

It was found 31 applications (22 in the Google Play and nine in the Apple Store). Of these, 12 are in both stores and were not registered in duplicates. As for the language, 13 are available in English, two in Spanish, three in Portuguese and one in Italian. It is deserved to emphasize the main functions of the applications, which are focused on informing about symptoms, treatment, safe sex practices, medications, examinations, risk assessment of contracting an STI and location of health service.

SELP application design

According to the survey of scientific publications and research in the stores of Apple Store and Google Play developed in the first stage, a SELP high fidelity prototype (Figure 1) was obtained with 29 screens and the following functions:

Figure 1
High-fidelity prototype screen of the application, Fortaleza, Ceará, Brazil, May 2016

• Video - information on transmission, symptoms and treatment of syphilis;

• Questionnaire - the user is classified in medium and high risk of exposure for syphilis;

• System Options - the menu screen displays a list that contains the video, information about syphilis, the list of health clinics, the option “my treatment” and “notify my partners”;

• Information on syphilis - shows what is the disease, the stages (primary, secondary, tertiary, in pregnancy), transmission, examinations and treatment;

• Health clinics - brings the map of health clinics in the municipality of Fortaleza;

• Treatment - the tool locates a health clinic, the user schedules a day and time to start or continue treatment;

• Partner notification - enables the user to notify one or more sexual partners in a confidential manner.

Application redesign (usability test)

The objective of this stage was to verify if the representatives of the target public are able to carry out the proposed tasks and to identify the difficulties in handling the tool (Table 1).

Table 1
Answers of the participants of the application usability test, Fortaleza, Ceará, Brazil, May 2016

Variables (n=8)	n	%	IVC
Filling in the test user information
Filed in quickly	6	75
Took a while to fill in	1	12.5
Complained to fill in	0	0	0.87
Asked for help to fill in	1	12.5
Did not fill in	0	0
Watching the video and finding information
Found quickly	7	87.5
Took a while to find	0	0
Complained about the menu icons	0	0	0.87
Asked for help to find	1	12.5
Did not find	0	0
Identifying the nearest health clinic
Identified quickly	2	25
Took a while to identify	1	12.5
Complained about the menu items	1	12.5	0.37
Asked for help to identify	0	0
Did not identify	4	50
Scheduling the treatment
Scheduled the treatment	1	12.5
Took a while to schedule the treatment	0	0
Complained about the button to schedule	3	37.5	0.12
Asked for help to schedule	4	50
Did not schedule	0	0
Notifying a partner, filling in e-mail and phone number
Notified quickly	7	87.5
Took a while to notify	0	0
Complained about the used space	0	0	0.87
Asked for help to notify	1	12.5
Did not notify	0	0

Satisfactory findings were divided into the tasks “filling in the test user information”, “watching the video and finding information” and “notifying a partner, filling in email and phone number.” All the tasks obtained the CVI equal to 0.87.

It is observed that six participants have achieved good performance in the task of “fill in the test user information”. This has been simplified by the intuitive layout, by objective questions, simple language and binary responses.

The item “watch the video and find information”, which addresses the complications of the disease in pregnant women infected with syphilis, obtained positive results, since seven participants had no difficulties performing the task. The video’s script emphasizes the treatment of the couple and possible sexual partners, mitigating stigma and discrimination, as well as encouraging the search for free tests available at health services.

The task related to a partner notification, proved to be easy to execute since seven participants notified a partner after filling in the spaces of e-mail and phone number. The ease of running this task is granted to the existence of a simple interface and intuitive icons.

The unsatisfactory results occurred in the tasks of “identify the nearest health clinic” and “schedule the treatment”. Both were extremely difficult, and two participants performed the tasks, which were evaluated by the team as unsatisfactory.

The results were contrary in the task in which the participants identified in the application the nearest health clinic where the usability test was being carried out. This activity showed that four participants had previous knowledge of the health clinic of the neighborhood where they lived and did not noticed the component that displays the name of the nearest post automatically.

The “scheduling the treatment” task, through which the user can record the first dose of syphilis treatment, showed that four participants had difficulty creating the registry and three had difficulty saving it. It was also verified that the participants demonstrated easier to understand the text buttons than the icons.

Participants were asked about the use of the application if they had syphilis. There was interest in using the application, as exemplified by the reports.

Yes, for the ease and practicality of being able to start the treatment, besides being very objective. (H8)

Yes! Because it gives us all the necessary information. (H6)

Concerning the anonymous call of a sexual partner, if confirmed the diagnosis of syphilis, many of the respondents stated that they would use the application, as the following excerpts show.

Yes. For reasons of not having embarrassment of both parties. (H3)

Yes, so that it does not infect other people. (H6)

The video and the accessible language were the potentialities of the application pointed out by the participants.

Video with information on the disease and the location of the nearest health clinic. (H1)

The information and ease of how you could be dealing with. (H5)

The language is very didactic and efficient for all groups of people, from the least enlightened to educated people. (H6)

The negative points of the application pointed out by the participants were:

Scheduling button. (H1)

A short video showing how the application works. (H2)

These answers show participants’ difficulties in handling the application, highlighting to the information technology team the necessary adjustments to the application layout.

Evaluation of the application by experts

This stage consisted in the evaluation of the SELP application by ten experts with experience in assisting STI patients. All are certificated in Infectology (2), Family Health (4), Collective Health (1), Adolescent Health (1), Gynecology and Obstetrics (2).

When asked about the completion of master and doctorate., concluded or in progress, they reported having Doctoral degree in Infectology (1), Collective/Public Health (4), Child Health (1); Master in Clinical Care in Nursing (1), Child and Adolescent Health (1), Collective/Public Health (6), Family Health (1) and Infectology (1).

The evaluation performed by the experts on the objectives of the application (Table 2) showed that in the six items there is a high level of satisfaction regarding the objectives of the application, since the CVI values are between 0.80 and 1.

Table 2
Answers of the experts regarding the validation of the objectives of the application, Fortaleza, Ceará, Brazil, October 2016

Variables (N = 10)	CVI
The text is compatible with the target public, taking into account the different profiles of syphilis carriers	0.80
The information/content is appropriate for guidance on the importance of syphilis treatment	1.00
It may promote behavioral changes in relation to the treatment of syphilis	1.00
The content and image are motivating and encourage them to continue browsing in the application	0.80
The content answers the questions, clarifies the ways of transmission, the signs and symptoms and the treatment of the disease	1.00
It can be used by health professionals during prenatal consultations	1.00
Total CVI	0.93

The second aspect evaluated refers to the structure and functionality of the application (Table 3). The findings of this question were considered applicable (CVI values from 0.80 to 1) with total CVI equal to 0.89.

Table 3
Answers of the experts regarding the validation of the structure and functionality of the application, Fortaleza, Ceará, Brazil, October 2016

Variables (N = 10)	CVI
The video is appropriate to guide/teach men and women about the prevention and treatment of syphilis	1.00
The video is compatible with the participants' level of knowledge	1.00
The information presented is scientifically correct	1.00
There is a logical sequence of proposed content	1.00
The information used is appropriate and sufficient for syphilis treatment.	1.00
Content and messages are appealing	0.80
Language is clear and objective	0.90
The scenes reflect discrimination	0.80
The illustrations (images, photos and videos) are pertinent	0.80
The illustrations (images, photos and videos) are clear and transmit easier comprehension	0.80
The number of illustrations is adequate	0.80
Text colors are relevant and easy to read	0.80
The letter used is easy to read	0.90
Total CVI	0.89

In the assessment of the relevance of the application (Table 4), a consensus was obtained among the evaluators, all of which assigned the maximum score for the items (CVI=1).

Table 4
Answers of the experts regarding the validation of the relevance of the application, Fortaleza, Ceará, Brazil, October 2016

Variables (N= 10)	CVI
The application proposes to pregnant women and their partners awareness that helps in the treatment of syphilis	1.00
The application addresses the issues necessary for treatment orientation	1.00
The application is suitable for use by health professionals	1.00
The application could improve healthcare in the health service	1.00
The application will improve interpersonal relationship between professional and user	1.00
Total CVI	1

DISCUSSION

According to the evidence found in the literature, the development of an application is a viable solution for health, provides faster information delivery, and contains attractive and dynamic sound resources and images, strategically created to catch the attention of the user.

Although mobile technologies represent powerful tools for engaging and transmitting information on syphilis prevention and treatment, little is known about the methodological rigor of the applications available in online stores. Due to the fast technological evolution, several applications are launched daily in stores, installed and uninstalled and few are evaluated.

Because of the difficulties identified during the usability test, the layouts of the interface of some screens were adjusted to make it easier the handling of the tool before making the application available in its final version for wide use. Human-Computer Interaction is an information-processing device in which the layout of the interface is designed to try to minimize the cognitive load of people, making the product intuitive and efficient⁽²⁰⁾.

The Participatory Design of Interaction emphasizes the importance of meeting users’ needs, adding necessary adjustments make it easier the use of the tool so that it has maximum effectiveness⁽¹⁴⁾.

Participant reports point out that the video, language, and localization feature are highlights in the SELP application. The Global Positioning System (GPS) can be used to identify the distribution of STIs in a specific geographical area⁽²¹⁾ and to locate the health services where the user obtains information about the treatment, examinations, medications and support groups^(22-23).

The video deserves to be emphasized, because was considered appropriate to guide the prevention and treatment of syphilis, being compatible with the level of knowledge of all and does not reflect discrimination. Confirming the research, other authors developed videos to encourage condom use, dialogue between the couple and HIV testing^(24-25).

Respondents said they would use the application because it is a quick and practical way to start treatment, but the application does not guarantee that users will get the right treatment. A study shows that, despite the free treatment for patients diagnosed with STI, only 44% of male partners had performed the treatment properly⁽²⁶⁾.

Regarding the anonymous call of a sexual partner, if confirmed the diagnosis of syphilis, seven respondents said they would use the application, so that their partnerships could carry out the treatment and adopt preventive measures, besides being a tool that avoids certain constraints. Only one participant preferred to speak face-to-face with their partners.

It is highlighted some suggestions made by the participants to improve the application: to insert more images, videos and address other STIs. Similar findings were seen in a study of applications for men with HIV, since for the participants, applications have to be visually appealing, contain videos with information about STI and how to use condom^(23,27).

In relation to the objectives, content and relevance of the technological device, the evaluators considered the texts adequate and compatible with the different profiles of syphilis carriers, with stimulating language and images to go through the other functions of the application.

The information was considered by the evaluators as scientifically correct, which has a logical sequence and is suitable for the treatment of syphilis. The illustrations and colors of the text were considered relevant. A study developed by Muessig et al⁽²⁸⁾ shows that participants want a useful technology that meets their needs, with few texts and attractive content.

Contributions to the Health field

The SELP application assists health professionals during prenatal consultations, conveying information about syphilis in a simple, interactive and dynamic way, mitigating stigma. It sensitizes the user to seek health service and, if necessary, perform the treatment and notify their possible sexual contacts. It is a proper tool to assist in interpersonal relationship between professional and user. The advantage of the mobile device is the provision of a low-cost service, a greater wideness in the disseminating of the intervention and access to information.

Study limitations

The limitations of the research refer to non-performance of the usability test with illiterate participants and those with syphilis. Future studies may expand usability testing, acceptability assessment, and cost-effectiveness feasibility of intervention using this application.

FINAL CONSIDERATIONS

The SELP application innovates the reality of Collective Health, since it can assist health professionals in the convening and treatment of pregnant women and their partners with syphilis, as well as being a strategy to promote health regarding care with the sexual health of the population. In order to do so, it was intended to identify the necessary requirements in the literature of the components required to be inserted in the application, together with the profile of the pregnant women with syphilis, and a methodology for the design of a software for the development and evaluation of the technology considering the objectives of the study.

Among the functions offered by the SELP application, it is highlighted information on symptoms, causes, risks and treatment, in text and video. These resources make the application accessible and direct the treatment by locating health clinics in the municipality of Fortaleza by means of digital maps, alerting the pregnant woman and the partner about the treatment dates and making it possible to map the network of contacts of syphilis carriers in an anonymous way.

When incorporated into the health services routine, the SELP application may provide epidemiological data, such as age, sex, number of sexual partners, use of condoms, and number of notifications sent. These data can be adopted by regional and national systems to support public policies for coping with syphilis. To that end, health professionals working in the Basic Health Units need to be able to use the technology in order to acquire the knowledge and skills necessary for users to adopt the technology.

The app has been well evaluated by users and experts with respect to information, layout, and relevance. The information was considered suitable, scientifically correct and easy to understand. In addition, the application was adhered as relevant, an assistance to doctors and nurses in the communication about the modalities of transmission, prevention and treatment of syphilis, besides contributing to the improvement of the interpersonal relationship between the professional and the user.

In another perspective, despite the good evaluation of the application, some necessary modifications were identified in some functions to make it more practical, didactic and intuitive. Future versions will add the proposed adjustments in the usability test and expert judgment.

Future developments should include a study with the population served in Primary Health Care to verify the efficacy and applicability of SELP, since the effectiveness of the application depends on the behavior and cultural factors of the users. Thus, understanding these factors will favor the wide dissemination and implementation of the tool.

REFERENCES

• ¹
  Ministério da Saúde (BR). Combate à Sífilis Congênita. Agenda de Ações Estratégicas para Redução da Sífilis Congênita no Brasil. Secretaria de Vigilância em Saúde Departamento de Vigilância, Prevenção e Controle das DST, Aids e Hepatites Virais. Brasília: MS; 2016.
• ²
  World Health Organization (WHO). Global health sector strategy on Sexually Transmitted Infections, 2016-2021 [Internet]. Geneva, Switzerland. 2017[cited 2018 Oct 20]. Available from: https://apps.who.int/iris/handle/10665/260232
  » https://apps.who.int/iris/handle/10665/260232
• ³
  World Health Organization (WHO). Global Health Observatory (GHO) data[Internet]. Geneva, Switzerland. 2017[cited 2018 Oct 20]. Available from: https://www.who.int/gho/sti/pregnancy/text/en/
  » https://www.who.int/gho/sti/pregnancy/text/en/
• ⁴
  Ministério da Saúde (BR). Boletim Epidemiológico da Sífilis. Secretaria de Vigilância em Saúde - Departamento de DST, Aids e Hepatites Virais. Brasília: MS; 2015.
• ⁵
  Reynolds NR, Satyanarayana V, Duggal M, Varghese M, Liberti L, Singh P, et al. MAHILA: a protocol for evaluating a nurse-delivered mHealth intervention for women with HIV and psychosocial risk factors in India. BMC Health Serv Res. 2016;16(a):352. doi: 10.1186/s12913-016-1605-1
  » https://doi.org/10.1186/s12913-016-1605-1
• ⁶
  Campbell AR, Kinvig K, Côté HC, Lester RT, Qiu AQ, Maan EJ, et al. Health care provider utilization and cost of an mhealth intervention in vulnerable people living with HIV in Vancouver, Canada: Prospective Study. JMIR Mhealth Uhealth. 2018;6(7):e152. doi: 10.2196/mhealth.9493
  » https://doi.org/10.2196/mhealth.9493
• ⁷
  Awiti PO, Grotta A, van der Kop M, Dusabe J, Thorson A, Mwangi J, et al. The effect of an interactive weekly mobile phone messaging on retention in prevention of mother to child transmission (PMTCT) of HIV program: study protocol for a randomized controlled trial (WELTEL PMTCT). BMC Med Inform Decis Mak. 2016;16:86. doi: 10.1186/s12911-016-0321-4
  » https://doi.org/10.1186/s12911-016-0321-4
• ⁸
  World Health Organization (WHO). New horizons for health through mobile technologies Based on the findings of the second global survey on eHealth[Internet]. (Global Observatory for eHealth Series, Volume 3). World Health Organization. 2011 [cited 2017 Apr 5]. Available from: http://apps.who.int/iris/bitstream/10665/44607/1/9789241564250_eng.pdf
  » http://apps.who.int/iris/bitstream/10665/44607/1/9789241564250_eng.pdf
• ⁹
  Watterson JL, Walsh J, Madeka I. Using mHealth to Improve Usage of Antenatal Care, Postnatal Care, and Immunization: a systematic review of the literature. Biomed Res Int. 2015;2015:153402. doi:10.1155/2015/153402
  » https://doi.org/10.1155/2015/153402
• ¹⁰
  Lund S, Nielsen BB, Hemed M, Boas IM, Said A, Said K et al. Mobile phones improve antenatal care attendance in Zanzibar: a cluster randomized controlled trial. BMC Pregnancy Childbirth. 2014;14(1, article 29). doi: 10.1186/1471-2393-14-29
  » https://doi.org/10.1186/1471-2393-14-29
• ¹¹
  Crawford J, Larsen-Cooper E, Jezman Z, Cunningham SC, Bancroft E. SMS versus voice messaging to deliver MNCH communication in rural Malawi: assessment of delivery success and user experience. Glob Health Sci Pract. 2014;2(1):35-46. doi: 10.9745/GHSP-D-13-00155
  » https://doi.org/10.9745/GHSP-D-13-00155
• ¹²
  Ministério da Saúde (BR). Conselho Nacional de Saúde. Diretrizes e normas regulamentadoras de pesquisas envolvendo seres humanos. Resolução 466 de 12 de dezembro de 2012. Diário Oficial da União, Brasília: MS; 2012.
• ¹³
  LoBiondo WG; Habber J. Confiabilidade e Validade In: LoBiondo WG; Habber J. Pesquisa em Enfermagem: métodos, avaliação crítica e utilização. 40 ed. Rio de Janeiro: Guanabara Koogan; 2001. p. 186-199.
• ¹⁴
  Rogers Y, Sharp H, Preece J. Design de interação: além da interação humano-computador. 30. ed. Porto Alegre: Bookman, 2013.
• ¹⁵
  Kensing F, Blomberg J. Participatory design: issues and concerns. Comput Support Coop Work 1998;7(3):167-85. doi: https://doi.org/10.1023/A:1008689307411
  » https://doi.org/10.1023/A:1008689307411
• ¹⁶
  Nielsen J. Heuristic evaluation. In: Nielsen J, Mack RL. Usability Inspection Methods. New York: John Wiley & Sons Inc., 1994.
• ¹⁷
  Likert R. A technique for the measurement of attitudes. Arch Psychol [Internet]. 1932[cited 2017 Apr 5];140:44-53. Available from: https://psycnet.apa.org/record/1933-01885-001
  » https://psycnet.apa.org/record/1933-01885-001
• ¹⁸
  Alexandre NMC, Coluci MZO. Content validity in the development and adaptation processes of measurement instruments Ciênc Saúde Coletiva. 2011;16(7):3061-8. doi: 10.1590/S1413-81232011000800006
  » https://doi.org/10.1590/S1413-81232011000800006
• ¹⁹
  Bardin L. Análise de Conteúdo. 20 ed. Lisboa: Edições 70; 2011.
• ²⁰
  Deng L, Wang G, Yu S. Layout design of human-machine interaction interface of cabin based on cognitive ergonomics and GA-ACA. Comput Intell Neurosci. 2016;2016:1032139. doi: 10.1155/2016/1032139
  » https://doi.org/10.1155/2016/1032139
• ²¹
  Besoain F, Perez-Navarro A, Caylà JA, Aviñó CJ, Olalla PG. Prevention of sexually transmitted infections using mobile devices and ubiquitous computing. Int J Health Geogr. 2015;14:18. doi: 10.1186/s12942-015-0010-z
  » https://doi.org/10.1186/s12942-015-0010-z
• ²²
  Sun CJ, Stowers J, Miller C, Bachmann LH, Rhodes SD. Acceptability and feasibility of using established geosocial and sexual networking mobile applications to promote HIV and STD testing among men who have sex with men. AIDS Behav. 2015;19(3):543-52. doi: 10.1007/s10461-014-0942-5
  » https://doi.org/10.1007/s10461-014-0942-5
• ²³
  Levy ME, Watson CC, Wilton L, Criss V, Kuo I, Glick SN, et al. Acceptability of a mobile smartphone application intervention to improve access to HIV prevention and care services for black men who have sex with men in the District of Columbia. Digit Cult Educ [Internet]. 2015[cited 2017 Apr 5];7(2):169-91. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/pmid/26594251/
  » https://www.ncbi.nlm.nih.gov/pmc/articles/pmid/26594251/
• ²⁴
  Jones R, Hoover DR, Lacroix L J. A randomized controlled trial of soap opera videos streamed to smartphones to reduce risk of sexually transmitted human immunodeficiency virus (HIV) in young urban African American women. Nurs Outlook. 2013;61(4):205-15. doi: 10.1016/j.outlook.2013.03.006
  » https://doi.org/10.1016/j.outlook.2013.03.006
• ²⁵
  Myint-U A, Bull S, Greenwood GL, Patterson J, Rietmeijer CA, Vrungos S, et al. Safe in the City: developing an effective video based intervention for STD clinic waiting. Health Promot Pract. Health Promot Pract. 2010;11(3):408-17. doi: 10.1177/1524839908318830
  » https://doi.org/10.1177/1524839908318830
• ²⁶
  Secura GM, Desir FA, Mullersman JL, Madden T, Allsworth JE, Peipert JF. Predictors of male partner treatment for sexually transmitted infection. Sex Transm Dis. 2012;39(10):769-75. doi: 10.1097/OLQ.0b013e31825ec611
  » https://doi.org/10.1097/OLQ.0b013e31825ec611
• ²⁷
  Aliabadi N, Carballo-Dieguez A, Bakken S, Rojas M, Brown W 3rd, Carry M, et al. Using the information-motivation-behavioral skills model to guide the development of an HIV prevention smartphone application for high-risk MSM. AIDS Educ Prev. 2015;27(6):522-37. doi: 10.1521/aeap.2015.27.6.522
  » https://doi.org/10.1521/aeap.2015.27.6.522
• ²⁸
  Muessig KE, Pike EC, Fowler B, LeGrand S, Parsons JT, Bull SS, et al. Putting prevention in their pockets: developing mobile phone-based HIV interventions for black men who have sex with men. AIDS Patient Care STDS. 2013;27(4):211–22. doi: 10.1089/apc.2012.0404
  » https://doi.org/10.1089/apc.2012.0404

Publication Dates

• Publication in this collection
  16 Sept 2019
• Date of issue
  Sep-Oct 2019

History

• Received
  08 Nov 2018
• Accepted
  21 Mar 2019