SARA application for treating people with tuberculosis: a methodological study

Abstract

Objective

To develop an application for monitoring the treatment of individuals with tuberculosis.

Methods

A methodological study with three phases: preparation of the concept map according to the manuals of the Brazilian Ministry of Health and the World Health Organization; development of the application, with navigation and prototyping design; and based on Pasquale’s method, content validation was performed by health experts in an online environment with semantic validation by the target audience, individuals with tuberculosis treated in a health unit in Natal, Rio Grande do Norte, Brazil, using the Delphi technique, analyzed using the Content Validity Index.

Results

The final version of the application was developed after three rounds of the Delphi technique. In the third Delphi round, the overall Content Validity Index was 0.92, as rated by seven expert evaluators in the North, Northeast, Central-West, South, and Southeast of Brazil, and validated by six patients being treated for tuberculosis, obtaining an overall Content Validity Index of 0.98.

Conclusion

The application was validated regarding content and semantics by expert evaluators working in the area of tuberculosis in more than one region of Brazil, from a multidisciplinary perspective, and by the target audience for which it is intended. It was considered an important tool to contribute to control strategies for the treatment of tuberculosis.

Tuberculosis; Medical informatics applications; Technology; Software validation

Resumo

Objetivo

Construir um aplicativo para o acompanhamento do tratamento de pessoas com tuberculose.

Métodos

Estudo metodológico com três fases: preparação do mapa conceitual por meio de manuais do Ministério da Saúde do Brasil e da Organização Mundial da Saúde; a construção do aplicativo com a elaboração do projeto de navegação e a prototipagem; e validação por conteúdo e semântica guiada pelo método de Pasquale com validação de conteúdo por juízes especialistas em saúde em ambiente online e validação de semântica pelo público-alvo, pessoas com tuberculose atendidas em uma unidade de saúde de Natal, no Rio Grande do Norte, Brasil, por meio da Técnica Delphi analisada a partir do Índice de Validade de Conteúdo.

Resultados

Construiu-se o aplicativo em sua versão final após três etapas da Técnica Delphi. Na etapa Delphi 3, o Índice de Validade de Conteúdo global foi de 0,92, sendo avaliado por sete juízes especialistas nas regiões Norte, Nordeste, Centro-Oeste, Sul e Sudeste e validado por seis pessoas com tuberculose atendidas, alcançando um Índice de Validade de Conteúdo global de 0,98.

Conclusão

O aplicativo foi validado quanto ao conteúdo e à semântica por juízes especialistas, em uma perspectiva multiprofissional, e atuantes na área de tuberculose de mais de uma região do Brasil e pelo público-alvo a que se destina, sendo considerado uma ferramenta importante para somar às estratégias de controle para o fim da tuberculose.

Tuberculose; Aplicações da informática médica; Tecnologia; Validação de programas de computador

Resumen

Objetivo

Elaborar una aplicación para el seguimiento del tratamiento de personas con tuberculosis.

Métodos

Estudio metodológico de tres fases: preparación del mapa conceptual por medio de manuales del Ministerio de Salud de Brasil y de la Organización Mundial de la Salud; desarrollo de la aplicación con la elaboración del proyecto de navegación y creación del prototipo; y validación de contenido y semántica guiada por el método de Pasquale, en la cual la validación de contenido fue realizada por jueces especialistas en salud en ambiente virtual y la validación semántica por el público destinatario, personas con tuberculosis atendidas en una unidad de salud de Natal, estado de Rio Grande do Norte, Brasil, mediante el método Delphi analizado a partir del Índice de Validez de Contenido.

Resultados

Se elaboró la aplicación en su versión final luego de tres etapas del método Delphi. En la etapa Delphi 3, el Índice de Validez de Contenido global fue 0,92, evaluado por siete jueces especialistas de las regiones Norte, Nordeste, Centro-oeste, Sur y Sudeste, y validado por seis personas con tuberculosis atendidas, con un Índice de Validez de Contenido global de 0,98.

Conclusión

La aplicación fue validada respecto al contenido y a la semántica por jueces especialistas, bajo una perspectiva multiprofesional y del área de tuberculosis con presencia en más de una región de Brasil, y por el público destinatario. Fue considerada una herramienta importante para sumar a las estrategias de control para el fin de la tuberculosis.

Tuberculosis; Aplicaciones de la informática médica; Tecnología; Validación de programas de computación

Introduction

Mobile devices are tools used for worldwide communication, which is no different in healthcare, where they play the role of informing, preventing, monitoring, and guiding users about diseases and illnesses.( ¹ )The term “Digital Health” incorporates recent technological advances, unifying all the concepts of Information and Communication Technology (ICT) applications.( ² - ⁴ )

The new Coronavirus pandemic has reinforced the importance of digital health, which has been increasing in importance, as it facilitates adherence to and treatment of diseases.( ⁵ )This is not new technology, such as the Internet of Things (IoT), however its use is growing substantially in health research from several areas, as observed in other studies, using validation regarding its content for the quality of the technology delivered.( ⁶ - ⁸ )

Those technologies related to applied knowledge, when specific to health, are described as Health Technologies (HT), which support the care offered. Among these is the use of mobile telephony, called Mobile Health, the use of which is increasing in health research( ⁹ )

Mobile devices have been used to monitor treatment of many diseases, and it would be no different for tuberculosis (TB), as the use of technologies is already considered a strategy for confronting the disease, while supporting and strengthening its treatment.( ⁶ , ¹⁰ )

Tuberculosis is a global epidemic, and Brazil ranks as the 20th country in the world in disease incidence. An aggravating factor is that, even though it is curable, adherence to treatment is still one of the greatest challenges faced in reaching the goals of eradicating this disease.( ¹⁰ - ¹² )

The strategies to combat TB have historically included the Stop TB strategy, which was established as part of the Millennium Development Goals in 1990, achieving significant reductions in the infection and mortality rates of the disease due to investment initiatives. However, TB remains the infectious disease with the highest mortality rate, especially when it affects people with HIV.( ¹³ )More recently, Brazil joined the World Health Assembly by WHO, which launched the strategy for TB control in 2014, proposing the TB Strategy, aiming to eliminate the epidemic by 2050, boosting the National Plan for TB termination in the country. The goal is to reduce the incidence coefficient to less than 10 cases per 100,000 population and one death per 100,000 population, by 2035.( ¹⁴ )

In 2017, treatment interruption was one of the greatest obstacles encountered in the treatment of TB, accounting in Brazil for a rate of 10.4% in the period from 2012 to 2018, according to the Information System on Diseases of Compulsory Declaration .( ¹⁵ )In the current SARS-CoV-2 pandemic environment, the individual with TB is considered vulnerable to this infection, with a risk of developing more severe forms of the disease, resulting in the discontinuation of treatment because of the need for social distancing, which requires complementary support for this monitoring by means of HT.( ¹⁶ )

Considering the need to search for mechanisms that support the Stop TB strategy, and because of the advances achieved by health technologies, this study aimed to develop an application to monitor the treatment of individuals with TB.

Methods

This was a methodological study designed to develop and validate content and semantics of the mobile application, Self-reported Follow-up for Treatment Support (SARA), aimed at people undergoing TB treatment. The application enables the patient to control the dose, organize medication taking, and provides reminders when to take it. The development of the application had three stages: 1- Preparation of the Conceptual Map; 2- Conception of the application (Navigation Design and Prototyping); and 3- Application validation (Validation of the application and semantics).

In the first step, the updated TB manuals of the Ministry of Health and the WHO were used as a theoretical basis for the mobile device.( ³ , ⁷ , ¹⁰ )

The second step consisted of developing the application, which was supported by a computer and design professional, who created the navigation project and the prototyping. The navigation project involves defining the user’s navigation formats, for example, mobile or otherwise. The prototyping helps with understanding the purpose of the application. In this phase, some requirements needed to be ascertained to ensure a quality final product.

Beginning with a first meeting between the researcher and the professional in the area, contributions were made in subsequent meetings to develop the application, such as functionality, usability, reliability, efficiency, and sustainability. These criteria evaluate the quality of the software internally and externally, adapting the product according to the user’s needs, as well as from the perspective of its construction engineering.( ⁹ , ¹⁷ )

The complete mobile application and web application formed the candidate release of the final product. This entire process was conducted by specialists in the area of information and communication technology, in particular multimedia mobile communications, and can be accessed through the link of the initial version, also named Betha: https://github.com/maxmonteiro2008/Pronex-Paciente.git.

The third step consisted of the application validation phase. This step aimed to validate the application for content and semantics, guided by Pasquale’s method using the Delphi technique.( ¹⁷ - ¹⁹ )

The research site followed the study methodology that allows multiple sites and times. Content validation was received from expert evaluators in an online environment using an e-mail tool with snowball sampling; semantic validation was achieved using a convenience sample from the unit that at the time of collection had the most TB cases, during home visits in the city of Natal, in the state of Rio Grande do Norte. Pasquale indicates that a sample for methodological study should have between six and 20 participants; this method was used for both content and semantic validation.( ¹⁸ )

Content validation was performed using evaluation of expert health care reviewers. The non-probability convenience sample used the snowball technique, in which initial study participants indicated new participants until the proposed objective was reached.( ²⁰ )

The first professional selected was from the city of Manaus/Amazonas, Brazil, because of his experience in the area, working in the TB coordination of that city. He was identified by means of the Lattes Platform, using a subject search with the word “tuberculosis”, in the databases “doctors”, with filters “academic education”, “professional activity”, and “professional activity in the institution”, and was asked to indicate another professional from another region of Brazil. Nine researchers were contacted; seven answered the invitation, six nurses and one biologist, comprising specialists from five regions of the country.

The inclusion criteria for this step were, healthcare professionals who worked directly with TB patients, and/or on the subject under study. For the first professional, experience in caring for TB patients over five years, and a position or activity related to the area was required, to consolidate significant experience. Failure to return the application evaluation was considered an exclusion criterion.

Data collection was conducted via e-mail, with an invitation letter issued for participation in this study. After accepting participation in the study, and agreement was obtained via the Terms of Free and Informed Consent Form (TFIC) available at the link stored in Google Forms, the professional expert could access the online survey questionnaire with the characterization of this professional, and complete the step, “Content Validation”.

This step was conducted in the city of Natal, state of Rio Grande do Norte, Brazil, in a Basic Health Unit (BHU) indicated by the TB coordination of the city, as it was the health unit with the largest number of TB patients at the time of data collection. The incidence rate of TB in the city of Natal was 44.1 per 100,000 inhabitants.

The target audience consisted of individuals with TB treated at the health care facility in the Natal city. The sample was by convenience. In June 2021, the UBS was treating 12 patients for at least 14 days, who were contacted by telephone to introduce the project. Six patients declined to participate in the study. The final sample consisted of six participants who received a home visit. Personal Protective Equipment and apparel according to NR32 were used. Both the researchers involved and the Community Health Worker (CHW), who participated in the data collection, were vaccinated.

The inclusion criteria for this group were: more than 18 years of age, enrolled in the program, undergoing treatment at the research site, with a minimum of 14 days of treatment because of TB transmission during this period, and with any level of education. The exclusion criteria were: a diagnosis of mental disorder, or a communication difficulty that would prevent participation in the study.

Data collection was performed by the authors, individually, during the home visit. After the application was demonstrated to the participant on the researcher’s mobile device, and after the participants themselves handled it, they answered the validation questionnaire in the presence of the researchers, to clarify any doubts about its completion. The researchers remained present to answer any questions they had about how to complete the questionnaire.

The data collection instrument for the two steps used the Delphi technique, and the evaluation was obtained by means of a Likert scale: 1 (totally disagree), 2 (disagree), 3 (neither agree nor disagree), 4 (agree), and 5 (totally agree). In cases where “totally disagree” or “disagree” was selected, a space was available to justify this reason, by means of numerical codes, such as usefulness/ relevance (1), consistency (2), clarity (3), objectivity (4), simplicity (5), executable (6), currentness (7), vocabulary (8), accuracy (9), and institutional sequence of topics (10).( ²⁰ , ²¹ )

For data analysis, the Content Validity Index (CVI) was used for both validations, by means of the formula below, which contains the sum of agreement of the items marked in the questionnaire with numbers 4 and 5, as they represent the agreement in the questionnaire:( ²¹ , ²² )

$CVI=\frac{\text{ Number of responses }4\text{ or }5}{\text{ Total number of responses }}$

A recommended value of at least CVI=0.80 was established as the criterion for deciding whether to accept the item. The suggested changes were made and sent again for the conclusion of the content validation. Finally, the data from the descriptive analysis were entered into the R software global library, 4.05 version.

This study was approved by the Research Ethics Committee, Health Sciences Center of the Federal University of Espírito Santo, protocol number CAAE#05675618,4,0000,5060. In accordance with Resolution No. 466/2012 of the National Health Council, all requirements for the protection of participants in scientific research involving human beings were met. All participants were informed about the purpose of the study and were signed the TFIC form.

Results

Figure 1 presents the concept map that was the basis for the development of the mobile application.

Figure 1
Concept map for development of the mobile application

The application was created with the objective of intuitiveness and didactics, to facilitate the user’s understanding. The patient records the medication that has been taken, which enables the control of doses without any breaks or loss in follow-up, and how he or she is feeling at that moment, enabling the control of side effects and symptoms during treatment. The patient can also send a message, but the professional does not have the option to answer via the application, and must contact the patient by phone to request a visit to the health unit to solve the question. Important information about the disease and medication was provided in the application, and all functions were able to be audiotaped to facilitate the use by illiterate individuals.

Seven TB specialists were invited and agreed to participate in this step of the study, six women and one man. The evaluators were from five regions of Brazil: Northeast (2), North (1), Midwest (2), South (1), and Southeast (1). Three were master’s degree holders, two had doctoral degrees, and two were post-doctoral; four of them had complementary education in TB. All of them had worked more than three years with TB patients, and had more than 10 years of professional training, 57.2% were working in Health Education at the Primary Health Care level, 28.3% in hospitals, and 14.3% in the Health Secretariat.

In the first Delphi round, the overall CVI was 0.6, indicating the need for adjustments in the steps of the application, and resending with suggestions for a new evaluation. The modifications derived from suggestions as to the wording, such as changing the term “perception of the patients followed”, because they do not like labels, to the objectivity, such as “organize the sequence of the questions”, and the usefulness, with “a place for information with appointment dates”. These changes were made by changing the term to, perception of TB patients, organizing the sequence of questions, and adding a place for the appointment dates.

After these modifications, the second Delphi round achieved the expected CVI, however, the item “The main information about TB enables the individual’s understanding about the disease” did not achieve the necessary CVI, and a new order of presentation of information about TB was suggested, requiring a new round.

The overall CVI in the third Delphi round was 0.92, with all items reaching a higher score than 0.80, validating the content. Table 1 shows the evaluation of the experts in the TB area about the “objective”, “writing”, “proposal”, and “organization”. About the name of the application, 57.14% totally agreed with the name SARA, and considered that the application focuses the attention of the individual with TB and provides information about the disease. Regarding the writing, 71.42% totally agreed with the text, the images, and the format of the written content of the application. For the proposal, 100% totally agreed with the “talk to a professional” option in the application, and with the information about treatment duration and follow-up. Considering the organization of the application, 85.71% totally agreed with the space to inform, what is the feeling at the moment, administration time, and important information about TB.

Table 1
Frequency of scores obtained in the content validation of the application according to the expert evaluators

Items	D	PD	PA	A	TA	CVI
Purpose
The name of the application is appropriate for the purpose.	-	1	-	2	4	0.85
The name of the application gets the patients’ attention.	-	-	1	2	4	0.85
The application provides information about TB.	-	-	-	3	4	1
Writing
The information about TB will include short informative text, with images, and the patient can choose the option to activate the explanatory audio.	-	-	1	1	5	0.85
Proposal
The application will have information about the treatment duration and its follow-up.	-	-	-	-	7	1
The application will enable the patient to take a picture while taking the medication.	-	-	-	2	5	1
The application will have a “talk to a professional” option.	-	-	-	-	7	1
Organization
There will be a space for the patient to report how he/she is feeling at the moment.	-	-	-	1	6	1
The application will have a place to report the time of medication administration.	-	1	-	-	6	0.85
The main information about TB will enable the patient’s understanding about the disease.	-	1	-	-	6	0.85

• *D - Disagree; PD - Partially Disagree; PA - Partially Agree; A - Agree; TA - Totally Agree; CVI - Content Validity Index

The semantic validation was performed by six individuals with TB, male (3) and female (3), predominantly brown skin color (4), followed by black (2) and white (1), most with incomplete elementary school education (3), complete college education (2), and only literate (1). The occupation was, household (1), garbage collector (1), para-athlete (1), plastic artist (1), recycling (1), and school teacher (1). The first Delphi round, obtained a total CVI of 0.98 and all with values greater than 0.80, as shown in table 2 . The evaluators of the application suggested that no reference to TB should appear on the initial screen, due to the social stigma carried by the disease, and to the difficulties some patients have in reporting that they are in treatment. Therefore, the name SARA was chosen, with no specific reference to the disease, facilitating patient compliance.

Table 2
Application validation by the target group of people with TB

Items	D	PD	PA	A	TA	CVI
Purpose
The name of the application is appropriate for the purpose.	-	-	-	4	2	1
The name of the application gets the patients’ attention.			1	2	3	0.83
The application provides information about TB.	-	-	-	-	6	1
Writing
The information about TB will include short informative text, with images, and the patient can choose the option to activate the explanatory audio.	-	-	-	1	5	1
Proposal
The application will have information about the treatment duration and its follow-up.	-	-	-	1	5	1
The application will enable the patient to take a picture while taking the medication.	-	-	-	1	5	1
The application will have a “talk to a professional” option.	-	-	-	1	6	1
Organization
There will be a space for the patient to report how he/she is feeling at the moment.	-	-	-	2	4	1
The application will have a place to report the time of medication administration.	-	-	-	-	6	1
The main information about TB will enable the patient’s understanding about the disease.	-	-	-	-	6	1

• *D - Disagree; PD - Partially Disagree; PA - Partially Agree; A - Agree; TA - Totally Agree; CVI - Content Validity Index

Discussion

The development of an application that enables treatment follow-up of individuals with TB is a tool that can improve treatment adherence, which is still one of the greatest challenges in the fight against the disease. To achieve this, it is necessary to disseminate health education and information technologies that can be applied to health.( ⁹ , ²³ )

The creation of the SARA application focused on easy comprehension by its target population, with images and simple options for access, such as the possibility of audio, with the material validated, aiming to address different regional realities to support the monitoring of this group, achieving a CVI above 80%, which is necessary to be considered valid, as in other studies.( ⁹ , ²² , ²³ )

When comparing to other TB research addressing the validation of HT, most are aimed at health professionals, such as the validation of an instrument to assess Directly Observed Treatment (DOT), and another validating an instrument to evaluate the structures of primary care services for TB treatment( ⁸ , ⁹ )As shown in both studies, the focus is on care, the health care service, and the quality of adherence and treatment of individuals with TB, which is of great value in providing collaborative strategies with the health care professionals who work in this area.

Although incipient, some production of technologies for health education and/or treatment in TB was identified. As the proposal of an application, developed for use in a reference clinic for people with TB in Russia, aiming to improve the adherence of patients. Of the 46 participants, an increase of 52% in adherence was noticed, a percentage calculated from the proportion of doses taken over the number of doses prescribed for these patients. This reinforces that the use of HT, such as the SARA application, is fundamental to improve participation and adherence of patients to treatment.( ²⁴ )

Semantic validation was also observed using a technology designed for children with TB, a printed version, which was validated and, as in this study, obtained a CVI higher than 0.80 in a first step. However, by recommendation of the participants, changes were made, such as enlarging the images, reinforcing the need to understand that the HT are for an intended audience, ratifying the reliability so that the proposed technology could reach its objective.( ²⁵ )

This statement complements what was exposed in another study, in which a new educational material for TB was developed by means of existing productions in Brazil, identifying that most of the existing material was not validated for content, and that there is a need for the development of evaluation of the methodology used for technological advances with quality.( ²⁶ )Developed in the context of the pandemic of COVID-19, even though this was not the focus of this study, its contribution to social isolation in this population is significant. This must be analyzed by researchers, as it reduces exposure to the virus, reducing the need for contact to find or deliver medication, and enables the patient to stay informed or inform his providers about any treatment needs.

In this context, accessible technologies are important, as is this mobile application and a social media dissemination developed and validated in another study, with educational material for patients with TB, and guidelines on COVID-19, finding that both technologies exercise creativity and clinical reasoning. These authors agree with the possibility of the individual with TB assuming the leading role in his/her care.( ²⁷ )

Validated health applications are a current alternative that can contribute both to health education, in its informative aspect, and to treatment monitoring, as proposed by SARA, and can result in desired adherence to TB care. The target group validation was an opportunity to understand the use of the application from the perspective of the population that will use it, in the possibility of the application becoming an additional strategy for adherence to treatment.( ²⁸ , ²⁹ )

The use of technologies, such as Apps, shows a possibility of improvement in the relationship between health professionals and people with TB. The advantages are innumerable, as it eliminates inconsistencies in the information provided, as well as the process of validation by expert health professionals, and by those for whom it is intended to be used.( ²⁵ - ²⁹ )

As a limitation of the study, the number of evaluators, although predicted in the literature, was reduced, and for this reason, the choice of including different regions of Brazil with specialists who met the selection criteria, facilitated a more thorough evaluation of the application’s validation.

Conclusion

The SARA mobile application for people with TB was developed and content validated by expert evaluators from a multidisciplinary perspective, working in the area of TB, from different regions of Brazil, by means of three Delphi Technique rounds. The suggestions from both populations were accepted and changes were made in an application that actually met the needs of the target population, which can be accessed using the link: https://github.com/maxmonteiro2008/paciente-TB.git. The differential of being non-print technology, with audio for accessibility, and enabling communication with health professionals were identified potentials. All items achieved content validation index values. Applications for treatment, which enable the person with TB to participate in the care process and be instructed about the issue, were not identified, but are considered an important tool to contribute to the strategies of control and eradication of the disease. The use of technologies in the health sector is necessary in the current times, as most of the population already uses mobile devices that facilitate this kind of access. The proposal for the continuity of this study would be the development of an integrated information platform for TB monitoring, which would also allow the development of several educational aspects, such as the publication of e-books, generation of videos, publication of articles in a more popular language, and the availability of anonymized data for research purposes.

Acknowledgments

To the Foundation for Research Support and Innovation of Espírito Santo (FAPES) - CNPq/FAPES No. 24/2018 - SUPPORT PROGRAM TO EXCELLENCE CENTERS - PRONEX.

Referências

• ¹
  Digital Health & Care Scotland. Scotland’s digital health and care strategy: enabling, connecting and empowering. Edinburgh: Scottish Government; 2018. 20 p.
• ²
  Bousquet J, Anto JM, Bachert C, Haahtela T, Zuberbier T, Czarlewski W, et al. ARIA digital anamorphosis: Digital transformation of health and care in airway diseases from research to practice. Allergy. 2021;76(1):168-90. Review.
• ³
  Brasil. Ministério da Saúde. Secretaria-Executiva. Departamento de Informática do SUS. Estratégia de Saúde Digital para o Brasil 2020-2028. Brasília (DF): Ministério da Saúde; 2020 [citado 2022 Mar 13]. Disponível em: https://bvsms.saude.gov.br/bvs/publicacoes/estrategia_saude_digital_Brasil.pdf
  » https://bvsms.saude.gov.br/bvs/publicacoes/estrategia_saude_digital_Brasil.pdf
• ⁴
  World Health Organization (WHO). WHO guideline: recommendations on digital interventions for health system strengthening. Geneva: WHO; 2019 [cited 2022 Mar 13]. Available from: https://www.who.int/publications/i/item/9789241550505
  » https://www.who.int/publications/i/item/9789241550505
• ⁵
  World Health Organization (WHO). Handbook for the use of digital technologies to support tuberculosis medication adherence (No. WHO/HTM/TB/2017.30). Geneva: WHO; 2017 [cited 2022 Mar 13]. Available from: https://apps.who.int/iris/handle/10665/259832
  » https://apps.who.int/iris/handle/10665/259832
• ⁶
  Oliveira RS, Oliveira JL. A internet das coisas (IOT) com enfoque na saúde. Tecnol Projeção. 2017;8(1):77-85.
• ⁷
  Silva SS, Sipolatti WG, Fiorin BH, Massaroni L, Lopes AB, Fioresi M, et al. Content validation and development of a software for hemodialysis. Acta Paul Enferm. 2021;34:eAPE02571.
• ⁸
  Silva AV, Silva AC, Camelo EM, Coelho AA, Santos AH, Torres AL. Validation of an evaluation instrument of adherence to tuberculosis prevention measures in primary care. Res Soc Development. 2022;11(1):e101112414.
• ⁹
  Silva LM, Surniche CA, Sicsú AN, Mitano F, Nogueira JA, Santos CB, et al. Elaboração e validação semântica de um instrumento de avaliação da transferência do tratamento diretamente observado como política de controle da tuberculose. Rev Panam Salud Publica. 2015;38(2):129-35.
• ¹⁰
  Brasil. Ministério da Saúde. Secretaria de Vigilância em Saúde. Ministério da Saúde. Manual de Recomendações para o Controle da Tuberculose no Brasil. 2ª ed. Brasília (DF): Ministério da Saúde; 2019 [citado 2022 Mar 13]. Disponível em: https://bvsms.saude.gov.br/bvs/publicacoes/manual_recomendacoes_controle_tuberculose_brasil_2_ed.pdf
  » https://bvsms.saude.gov.br/bvs/publicacoes/manual_recomendacoes_controle_tuberculose_brasil_2_ed.pdf
• ¹¹
  World Health Organization (WHO). Global tuberculosis report 2019. Geneva: WHO; 2019 [cited 2022 Mar 13]. Available from: https://www.who.int/publications/i/item/9789241565714
  » https://www.who.int/publications/i/item/9789241565714
• ¹²
  Cazabon D, Alsdurf H, Satyanarayana S, Nathavitharana R, Subbaraman R, Daftary A, et al. Quality of tuberculosis care in high burden countries: the urgent need to address gaps in the care cascade. Int J Infect Dis. 2017;56:111-6. Review.
• ¹³
  Barreira D. The challenges to eliminating tuberculosis in Brazil. Epidemiol Serv Saude. 2018;27(1):e00100009.
• ¹⁴
  Brasil. Ministério da Saúde. Secretaria de Vigilância em Saúde. Brasil Livre da Tuberculose: plano nacional pelo fim da tuberculose como problema de saúde pública. Brasília (DF): Ministério da Saúde; 2017 [citado 2022 Mar 13]. Disponível em: https://bvsms.saude.gov.br/bvs/publicacoes/brasil_livre_tuberculose_plano_nacional.pdf
  » https://bvsms.saude.gov.br/bvs/publicacoes/brasil_livre_tuberculose_plano_nacional.pdf
• ¹⁵
  Soeiro VM, Caldas AJ, Ferreira TF. Abandono do tratamento da tuberculose no Brasil, 2012-2018: tendência e distribuição espaço-temporal. Cien Saude Colet. 2022;27(3):825-36.
• ¹⁶
  Hino P, Yamamoto TT, Magnabosco GT, Bertolozzi MR, Taminato M, Fornari LF. Impacto da Covid-19 no controle e reorganização da atenção à tuberculose. Acta Paul Enferm. 2021;34:eAPE002115.
• ¹⁷
  Oliveira AR, Alencar MS. The use of health applications for mobile devices as sources of information and education in healthcare. Rev Dig Bibliot Cien. Inf. 2017;15(1):234-45.
• ¹⁸
  Pasquali, L. Psicometria: teoria e aplicações. Brasília (DF): Editora UnB; 1997. p.161-200.
• ¹⁹
  Marques JB, Freitas D. The DELPHI method: characterization and potentialities for educational research. Pro-Posições. 2018;29(2):389-415.
• ²⁰
  Costa BR. Bola de neve virtual: o uso das redes sociais virtuais no processo de coleta de dados de uma Pesquisa Científica. Rev Interd Gestão Social. 2018;7(1):15-37.
• ²¹
  Pasquali L. Instrumentação psicológica: fundamentos e práticas. Porto alegre: Artmed; 2010. 568 p.
• ²²
  Souza AC, Alexandre NM, Guirardello EB. Psychometric properties in instruments evaluation of reliability and validity. Epidemiol Serv Saude. 2017;26(3):649-59.
• ²³
  Valença MS, Cezar-Vaz MR, Brum CB, Silva PE. O processo de detecção e tratamento de casos de tuberculose em um presídio. Cien Saude Colet. 2016;21(7):2111-22.
• ²⁴
  Gelmanova IY, Taran DV, Mishustin SP, Golubkov AA, Solovyova AV, Keshavjee S. ‘Sputnik’: a programmatic approach to improve tuberculosis treatment adherence and outcome among defaulters. Int J Tuberc Lung Dis. 2011;15(10):1373-9.
• ²⁵
  Rodrigues IL, Nogueira LM, Pereira AA. Learning through play: semantic validation of educational technology on tuberculosis for school children. Esc Anna Nery. 2021;25(4):e20200492.
• ²⁶
  Carvalho AC, Trajano VS, Oliveira LM, Costa VM, La Rocque L, Martins M, et al. Promoção da adesão ao tratamento da tuberculose (TB): experiência de avaliação e produção de material educativo sobre TB. Ensino Saúde Ambient. 2019;11(3):149-66.
• ²⁷
  Rocha NL, Araujo KF, Nakano AR, Lima HS, Marques JL. Material educativo para pacientes com tuberculose pulmonar diante da pandemia de COVID-19. Rev Saúde Colet UEFS. 2020;10:18-22.
• ²⁸
  Oliveira GM, Santos LF. Uso de aplicativos para dispositivos móveis no processo de educação em saúde: reflexões da contemporaneidade. Rev Observatório. 2018;4(6):826-44.
• ²⁹
  Barra DC, Paim SM, Dal Sasso GT, Colla GW. Methods for developing mobile apps in health: an integrative review of the literature. Texto Contexto Enferm. 2017; 26(4):e2260017. Review.

Edited by

Associate Editor (Peer review process): Rafaela Gessner Lourenço. ( https://orcid.org/0000-0002-3855-0003 ). Universidade Federal do Paraná, Curitiba, PR, Brazil

Publication Dates

• Publication in this collection
  06 Feb 2023
• Date of issue
  2023

History

• Received
  22 Nov 2021
• Accepted
  14 June 2022