Construction and validation of an e-book about cardiovascular risk in people living with the human immunodeficiency virus

Abstract

Objective

To build and validate an e-book about cardiovascular risk in people living with the human immunodeficiency virus.

Methods

Methodological study based on the evaluation research theory with analysis of outcome. It involved technological production comprising the phases of analysis and planning, modeling, implementation, and evaluation. Cardiovascular risk factors and strategies to reduce them were discussed. The e-book was validated with experts from all over the country between October 2017 and August 2018 using the Content Validity Index.

Results

As the e-book was written to meet the needs of the population, accessible language was used for an easier understanding of the content. Several aspects of the e-book were evaluated by experts. Afterwards, the most frequent criticisms and/or suggestions were analyzed. A concordance rate of 80.5% was observed after analysis of the global CVI. All items had satisfactory agreement rates, with a CVI of 86.6% for general impression, 96.8% for objective, 92.0% for content and 90.4% for relevance.

Conclusion

The e-book was evaluated in several aspects by experts and a global agreement index of 80.5% was found. Suggestions were accepted aiming at ensuring a more complete, cohesive, easy to read and updated material. According to the evaluation between experts, the material proved to be valid for use by people living with HIV in order to understand their cardiovascular risk and to know healthier habits that may help in prevention of cardiovascular diseases.

HIV Infections; Cardiovascular diseases; Health education; Educational technology

Resumo

Objetivo

Construir e validar um e-book sobre risco cardiovascular em pessoas vivendo com o vírus da imunodeficiência humana.

Métodos

Estudo metodológico fundamentado na teoria da pesquisa avaliativa com análise de resultado. Envolveu a produção tecnológica compreendendo as fases de análise e planejamento, modelagem, implementação e avaliação. Foram discutidos os fatores de risco cardiovascular e as estratégias para reduzi-los. O e-book foi validado por especialistas de todo o país entre outubro de 2017 e agosto de 2018 por meio do Índice de Validade de Conteúdo.

Resultados

Como o e-book foi escrito para atender as necessidades da população, adotou-se uma linguagem acessível para facilitar a compreensão do conteúdo. Vários aspectos do e-book foram avaliados por especialistas. Em seguida, foram analisadas as críticas e/ou sugestões mais frequentes. Após análise do IVC global, foi encontrada uma taxa de concordância de 80,5%. Todos os itens tiveram índices de concordância satisfatórios, com IVC de 86,6% para impressão geral, 96,8% para objetivo, 92,0% para conteúdo e 90,4% para relevância.

Conclusão

O e-book foi avaliado em diversos aspectos pelos especialistas, encontrando um índice de concordância global de 80,5%. As sugestões foram acatadas com o objetivo de garantir um material mais completo, coeso, de fácil leitura e atualizado. De acordo com a avaliação dos especialistas, o material se mostrou válido para utilização por pessoas vivendo com HIV, no entendimento de seu risco cardiovascular e conhecimento de hábitos mais saudáveis que ajudem na prevenção de doenças cardiovasculares.

Infecções por HIV; Doenças cardiovasculares; Educação em saúde; Tecnologia educacional

Resumen

Objetivo

Elaborar y validar un libro electrónico sobre riesgo cardiovascular en personas que viven con el virus de la inmunodeficiencia humana.

Métodos

Estudio metodológico fundamentado en la teoría de la investigación evaluativa con análisis de resultado. Incluyó la producción tecnológica, que comprendió las fases de análisis y planificación, creación de modelo, implementación y evaluación. Se discutieron los factores de riesgo cardiovascular y las estrategias para reducirlos. El libro electrónico fue validado por especialistas de todo el país entre octubre de 2017 y agosto de 2018, mediante el Índice de Validez de Contenido.

Resultados

Como el libro electrónico se escribió para cubrir las necesidades de la población, se adoptó un lenguaje accesible para facilitar la comprensión del contenido. Varios aspectos del libro electrónico fueron evaluados por especialistas. Luego, se analizaron las críticas o sugerencias más frecuentes. Después de analizar el IVC global, se observó un índice de concordancia del 80,5 %. Todos los ítems tuvieron índices de concordancia satisfactorios, con un IVC del 86,6 % en impresión general, 96,8 % en objetivo, 92,0 % en contenido y 90,4 % en relevancia.

Conclusión

Se evaluaron diversos aspectos del libro electrónico por parte de especialistas y se observó un índice de concordancia global del 80,5 %. Se acataron las sugerencias con el objetivo de garantizar un material más completo, cohesivo, de fácil lectura y actualizado. De acuerdo con la evaluación de los especialistas, el material demostró ser válido para ser utilizado por personas que viven con el VIH, para la comprensión de su riesgo cardiovascular y el conocimiento de hábitos más saludables que ayuden a prevenir enfermedades cardiovasculares.

Infecciones por VIH; Enfermedades cardiovasculares; Educación en salud; Tecnología educacional

Introduction

Antiretroviral therapy adherence has provided better rates of morbidity and mortality in people living with human immunodeficiency virus (PLHIV), as the infection has gone from a progressive fatal disease to a chronic disease with a significant improvement in life expectancy.(¹,²)

According to the Joint United Nations Program on HIV/AIDS (UNAIDS), there were 37.7 million PLHIV and around 28.2 million with access to treatment by 2020, which reflects the success of every policy effort towards access to antiretroviral therapy that contributes to reducing the rate of AIDS-related deaths.(¹)

However, the rates of chronic noncommunicable diseases in this population - including cardiovascular diseases - have been higher than those in the general population, signaling the need to rethink healthcare strategies for this population from the perspective of preventive care.(²,³)

The pathophysiology of cardiovascular diseases associated with HIV is complex, multifactorial and involves the interaction between traditional risk factors, the presence of HIV infection markers, CD4 T-cell and viral load counts, and prolonged exposure to antiretroviral therapy. These factors enable the occurrence of inflammation and immune activation responsible for triggering a persistent endothelial inflammatory process, which is a precursor to diseases such as myocardial infarction and stroke.(³,⁴)

In fact, PLHIV still have a higher prevalence of classic risk factors for cardiovascular diseases than that of people who do not live with the infection. Thus, the management of modifiable risk factors for cardiovascular diseases has become an essential aspect for the care of PLHIV.(²,³)

Educational interventions encouraging changes in lifestyle and the management of established risk factors for cardiovascular diseases are considered the first step towards primary and secondary prevention and have an association with decreased cardiovascular risk and better health outcomes.(³,⁵)

Health education is a care technology and a tool that values the knowledge, practices and cultural context of everyone involved in the educational process. Manuals, booklets, games, workshops, e-books, educational programs and software are examples of educational technologies that can be used.(⁶-⁹)

In the context of HIV infection, digital information and communication technologies have been used for interventions related to the prevention of virus transmission and to monitor patients, with a view to improving accessibility and quality of care.(¹⁰,¹¹)

However, no educational material addressing the prevention of cardiovascular diseases in this population has been identified. There are gaps in the literature on the construction and validation of educational material to improve the provision of care aimed at preventing cardiovascular diseases in PLHIV.

Therefore, the aim of this project was to develop innovative digital information and communication technologies that could meet the needs of PLHIV and assist health professionals in the context of cardiovascular disease prevention. Thus, this study aimed to build and validate an e-book about cardiovascular risk in PLHIV.

Methods

This study was part of a larger project entitled “Development, validation and effectiveness of educational technologies focused on the behavior, preventive practices and lifestyle of people living with HIV/aids”. This methodological study was based on the theory of evaluation research, outcome analysis type, which involves technological production.(¹²)

It comprised the phases of: analysis and planning (organization of the script and selection of content); modeling (construction of the material, preparation and editing of the layout, editing of images and videos); implementation (final construction of the e-book and availability for download); and evaluation (evaluation by experts in the field). Specific procedures must be followed in each phase to guarantee the quality of the material.(¹³)

The building process took place from February to August 2017, and the validation by experts took place between October 2017 and August 2018. The term e-book refers to the English abbreviation of the term “electronic book”. It is a digital book that can be read on electronic devices such as computers, tablets or even cell phones that support this feature.(¹³)

The first version of the e-book was entitled “Take care of your heart: strategies to reduce cardiovascular risk in people living with HIV/aids”. Part of the theoretical content on the topic was gathered for this purpose.

The script was developed by the authors. Several studies and the original articles identified were used for the selection of the content, in addition to the Update of the Brazilian Dyslipidemia and Atherosclerosis Prevention Directive(¹⁴)and the Clinical Protocol and Therapeutic Guidelines for the Management of HIV infection in adults in Brazil.(¹⁵)

The topics of the e-book were related to modifiable risk factors for cardiovascular diseases, cardiovascular risk among PLHIV, and strategies that can be adopted to reduce such risks.

In the modeling phase, all selected material was used to prepare the content of the first version of the e-book. The text editing software Microsoft Word 2016 was used in the construction before sending it to the Virtual Learning Environment.

In this phase, images, videos, photographs and links that could assist in reading the material were also selected. In addition to stimulating reading, these resources make the process of building knowledge more dynamic and attractive.

Initially, videos made available on public domain sites (YouTube) were used, which were evaluated by experts as to the veracity and quality of content.

However, videos with more specific guidelines were needed. The script of the videos was completed, specialists were invited, and the new videos were treated and edited by an audiovisual technical team. After the final version was approved, the videos were hosted on the Ribeirão Preto College of Nursing page on YouTube for later sharing in the e-book.

With the prepared material in hand, the content was transferred to a file in the Electronic Publication format, which allows the use of several tools that guarantee the principles of usability and accessibility, providing the user with an easy-to-use, dynamic, and interactive tool.

After testing all tools and layout approval, the e-book was available for free download on digital platforms for iOS and Android operating systems.

After the creation and modeling of the e-book, it was evaluated by experts in the area. This improved quality through evaluation throughout the development process.

A reference that recommends between six and 20 specialists in the selection of experts was followed.(¹⁶)It is also recommended that the number is odd to avoid possible ties.(¹⁷,¹⁸)

For a safe and reliable validation of the submitted content, judges should be experts in the subject area, which can be proven by both professional experience and academic career.(¹⁹,²⁰)

The choice of clinical specialists was made by checking the Lattes Platform, which makes available the curriculum of all researchers in Brazil. The option for “Advanced Search” was selected and in the “Subject” tab were used the keywords “HIV”, “Cardiovascular Diseases”, “Nursing”.

After a systematic search, 54 researchers were found. However, when evaluating their resumes, many did not work in the required expertise area, and 36 specialists in the field of cardiology and infectiology were invited to participate. The purpose of this evaluation was to analyze the content, possible errors, and interface or layout problems that could reduce the quality of the experience in the virtual learning environment. Twenty-one of these specialists completed the analysis.

Criteria proposing a score calculation were used in the selection of these experts. A minimum score of four points was necessary for inclusion of a specialist. In case of professionals with titles, a point was added as if the expert had all the necessary degrees.(¹⁹)

The evaluation of instruments should be carried out in a maximum of 30 days. After completing the evaluation and signing the informed consent, the evaluation instruments should be sent to the researcher. A reminder with a new deadline of seven days was sent to those who did not respond within 30 days. After the deadline, in the absence of a response, they were considered dropouts.

An instrument adapted from a model was used in the e-book validation by health experts. It assessed the following aspects: general impression, objective, content, relevance, verbal language and inclusion of topics.

Another instrument was used for audiovisual specialists, which included the following aspects: interface, aesthetic and audiovisual quality, and space for notes on necessary but absent content, unnecessary content and other comments.

The response options in both instruments were on a five-point Likert scale; “strongly agree” (5), “agree” (4), “disagree” (3), “strongly disagree” (2) and “don’t know” (1).

In addition, each specialist completed a sociodemographic characterization questionnaire containing information about sex, age, time of academic training, field, length of professional experience, and academic degree.

In the validation of the e-book by judges, the Content Validity Index (CVI) per item and global CVI were used, and these measure the degree of agreement of experts on aspects of the material.(²¹)

The Content Validity Index per item is calculated based on the number of experts who rated the items with answers of 3 or 4 (representative or very representative), divided by the total number of specialists.(²²) In the calculation of global CVI, the number of responses 4 and 5 (I agree and totally agree) was divided by the total number of questions.(¹,²³)The value of 80% was adopted as a limit for approval or disapproval of items covered.(²²)

After assessing all the criticism and suggestions raised by experts, the material was restructured according to their recommendations. Subsequently, the material was sent for publication on digital platforms.

The e-book is available for free download on the Apple Books for the iOS platform, and on Google Play for the Android platform.(²⁴)

All guidelines of the Declaration of Helsinki were followed in the development of this study. It was approved by the Research Ethics Committee of the proponent research institution under number 76868517.7.0000.5393.

Results

The script was organized and the content that should compose the e-book was selected based on findings of the literature, as seen in table 1. As the content was written to meet the needs of the population, accessible language was used to help them understand the content. Images, videos, photographs and links were selected to facilitate the interpretation of the content and make the virtual learning environment more dynamic and interactive. In addition, scripts were created for recording videos created for the project; one on opening the material, another on nutrition, and another on mental health.

Table 1
Script prepared for version 1 of the e-book - Take care of your heart: strategies to reduce cardiovascular risk in people living with HIV

eBook Content
Part 1	Know how to prevent
	Risk factors for cardiovascular diseases Tobacco Use Sedentary Lifestyle Stress Obesity Diabetes Mellitus Hypertension Dyslipidemia
	Cardiovascular risk in people living with HIV
Part 2	Strategies for care of yourself and your health
	1. Interventions for eating 2. Smoking cessation interventions 3. Stimulate the practice of physical activity 4. Strategies to relieve stress

The e-book interface was built using HTML5 and Java Script. A file was generated in Electronic Publication format, which has features that promote usability and accessibility, and provides the user with an easy-to-use, dynamic, and interactive tool. As described in the previous section, after conducting the systematic search, 36 professionals were invited to participate in the study; 34 of these accepted, but only 21 completed the e-book evaluation and validation process. For assessing their expertise, the specialists were classified according to Fehring score, which ranged from 7 to 19 points, and a mean score of 12.76 was achieved, as seen in table 2.

Table 2
Fehring classification score of specialists

Identification	Area of Specialization	Fehring Score
P*1	Service to people living with HIV	8
P2	Service to people living with HIV	7
P3	Nursing Service to people living with HIV	11
P4	Nursing Care in Cardiology	10
P5	Nursing Service to people living with HIV	13
P6	Nursing Service to people living with HIV	11
P7	Nutrition	18
P8	Mental health	19
P9	Service to people living with HIV	14
P10	Nursing Service to people living with HIV	16
P11	Nursing Service in the prevention of cardiovascular diseases	7
P12	Nursing Service to people living with HIV	11
P13	Nursing Service to people living with HIV	16
P14	Nursing Service to people living with HIV	11
P15	Nursing Service in the prevention of cardiovascular diseases	16
P16	Nursing Service in the prevention of cardiovascular diseases	12
P17	Nursing Care in Cardiology	11
P18	Nursing Service in the prevention of cardiovascular diseases	16
P19	Nursing Service to people living with HIV	16
P20	Informatics in Education	16
P21	Informatics in Education	9

• *p-judges

Regarding the characterization of specialists, 52.4% were female, age range of 26-55 years, average of 36.1 years, and an average academic education time of 12.86 years (SD ± 7.79). In relation to degree, 48% had a master degree, 33% held a PhD, 14% had post-doctorate, and 5% were specialists. The e-book was evaluated in several aspects by specialists in the field of health and information technology. After the evaluation, an analysis of the most frequent criticisms and/or suggestions was performed, from which the authors worked on readjustment of the material. According to the analysis of the global CVI of specialists in the field of health and information technology, an agreement rate of 80.5% was observed. All items had satisfactory indexes of agreement, with a CVI of 86.6% for overall impression, 96.8% for objective, 92.0% for content, 90.4% for relevance, 88.7% for verbal language, and 92.3% for inclusion of topics. Regarding the analysis performed by information technology specialists, an agreement index of 95.0% was identified for the quality of the interface and 86.0% for aesthetics and audiovisual. In order to respond to the criticisms and/or suggestions of experts, the goal was to create new images, update the Hypertension and Dyslipidemia Guidelines, record new videos and use new colors in the layout, as displayed in table 3.

Table 3
Suggestions from the expert committee after e-book evaluation

Suggestions	Verdict
Give more emphasis on the layout by using more colors	Accepted
Make clear the meaning of acronyms appearing throughout the text	Accepted
Include more playful images before each modifiable risk factor	Accepted
Exchange some technical terms for synonyms more understandable to the population	Accepted
Add abdominal obesity as a cardiovascular risk factor on the topic of obesity	Accepted
Replace text with figures and tables to make it easier to read and draw reader’s attention	Accepted
Tips should draw more attention	Accepted
Insert the content of how antiretrovirals interfere with cardiovascular diseases	Accepted
Review the second figure, as it is difficult to visualize	Accepted
Do not use abbreviations and/or technical terms	Accepted
Place references at the end of the book, not each chapter	Accepted
Update the reference values for Hypertension, following the 7th Brazilian Guideline for Hypertension (2016)	Accepted
Justify the text	Not accepted
More cold colors in titles / subtitles	Accepted
Add the difference between physical activity and exercise	Accepted
Add a final message to the patient	Accepted
Update the reference values for Diabetes Mellitus, following the Guidelines of the Brazilian Diabetes Society (2017-2018)	Accepted
Add who the material is intended for	Accepted

Discussion

The educational e-book is an innovative, accessible, low-cost digital tool that can be adopted in health services as a strategy for promoting cardiovascular health in PLHIV, since it was considered valid by experts, and followed the construction and validation steps recommended in the literature.(²⁴)

With the increasing access to mobile devices, new digital interventions and digitization of healthcare products are perceived as a unique opportunity for health interventions.

The use of digital strategies creates interesting opportunities for health promotion throughout the care and prevention process, especially in light of the rapid expansion of access to this technology, as its emergence diversifies learning methods and makes it possible to acquire knowledge not only for general purposes, but also to solve real problems.(²⁵,²⁶)

The evolution of books stands out in this context, represented by the emergence of e-books. Following the evolution of man and the emergence of new technologies, the book production process has changed, culminating in digital publications.(²⁶)

In the health area, the use of varied information and communication technologies goes beyond the transmission of information, as it supports self-care, behavioral changes, and exchange of information and emotional support among peers, in addition to providing benefits in the screening of people with chronic diseases.(²⁷,²⁸)

The rapid growth of mobile communication technologies (cell phone, smartphone) is being used to complement traditional public health programs, to promote health and healthy behaviors, to raise awareness of health risks and to manage treatment and adherence to medication.(²⁹)

The preparation of the e-book starts from this assumption, since this material provides knowledge for decision making based on self-care and promotion of behavioral changes with a view to reducing cardiovascular risk in PLHIV.

In this sense, the e-book appears an important tool linked to the Theories of Health Behavior Change, since it aims to promote subjects’ knowledge, self-efficacy, and motivation, skills that are the basis for behavioral change.(²⁹)

It can be inferred that the use of technology associated with quality information can promote greater interest of subjects, and foster knowledge and motivation. The human functioning is inherent to a wide network of influences mediated by cognitive processes in the adaptation to human changes.(²⁹)

In the context of PLHIV, studies show that technologies have been increasingly used in interventions aimed at preventing transmission of the virus or monitoring patients, promoting improved accessibility and quality of care, evidencing the originality of this study. Other advantages include its ability to impact hard-to-reach populations, including those with typically stigmatized behaviors within health services.(¹⁰,¹¹,²⁵,³⁰)

Therefore, the use of the e-book ensures access to secure information, regardless of where patients are, since it can be accessed from anywhere at any time and assist them in the process of knowing the risk factors for cardiovascular diseases, providing a reflection on their habits.

Mobile health technology has been a focus of growing interest as a way to improve cardiovascular prevention by combining modifiable risk factors, which represent most global risk factors for cardiovascular diseases, in a scalable and accessible way with the potential to assist in lifestyle modification.(³¹)

Even though the use of technology in health has the potential to improve the efficiency and effectiveness of care, studies have shown that acceptance among health professionals is still limited.(³²,³³)

In this context, the possibility of using the e-book during healthcare is highlighted as a means of passing the guidelines to PLHIV with a view to reducing cardiovascular risk.

In order to ensure that this instrument is presented in a safe and effective manner, the satisfactory levels of agreement indices presented during the validation process are highlighted. This factor demonstrates the importance of validation with a committee of experts to ensure the clarity, adequacy and relevance of the content, as well as the language of the educational material.

A study showed a variety of factors listed by PLHIV to face the challenges of having a healthy lifestyle. Although healthy eating and physical activity were recognized as important components of a good lifestyle, socioeconomic and/or financial reasons were the main barriers to the adoption of this behavior.(³³)

For the success of lifestyle interventions in the long run, recommendations must include much more than a list of foods to eat and/or avoid.(³³)For this reason, the material was designed and built in such a way as to provide possible changes in habits, regardless of financial issues.

A study indicated that participants were willing to learn about healthier food choices and enthusiastic to share knowledge with each other. The e-book tool can meet these needs since it brings information and enables the sharing of ideas.(³³)

Understanding the positive aspects from the perspective of patients provides the basis for patient centered counseling approaches that motivate people towards changing their behaviors.³³ In this context, modifiable cardiovascular risk factors should be a significant component of care, and part of the care routine established by health professionals.

Consequently, knowing the effects of dyslipidemia, stress, poor diet, obesity, smoking, hypertension, and diabetes is the first step to perceiving cardiovascular risk and determining the changes that must be made.

This material was developed to meet the needs of this population and make them aware of the need for prevention, adequate management of cardiovascular risk factors, and of their role as agents for changing their own habits, able to make decisions by themselves and move forward with necessary changes to a better quality of life.

We emphasize that this work is an example of how nurses can use new technologies and innovations to improve care and look at the patient in a holistic way, enabling a new way of doing health, using technology in their favor.

Our study had some limitations. As we had low feedback from the experts, the study took a broader time to be completed, although this did not interfere in the methodological rigor or in the quality of our findings. Furthermore, we could not validate the e-book with the population given the timeframe of the study.

Conclusion

This study presented the steps for the construction and validation of educational material in digital format (e-book) for the Brazilian population with guidance on knowledge of the risk factors for cardiovascular diseases. The production followed the instructional design development phases through which the content was selected, the material was built, and the layout was prepared, followed by the creation of images and recording of videos. An evaluation was performed by experts and the content was made available for download. A global agreement index of 80.5% was found. Furthermore, suggestions were accepted aiming to ensure a more complete, cohesive, easy to read and updated material. According to the evaluation between specialists, the material proved to be valid for use by PLHIV with the purpose to understand their cardiovascular risk and know healthier habits that may help in prevention of cardiovascular diseases.

Acknowlegdments

Financial Support from the Conselho Nacional de Ciência e Tecnologia (CNPq), Universal announcement number 428304/2018-4, Brazil.

Referências

• ¹
  Joint United Nations Programme on HIV/AIDS (UNAIDS). Global HIV & AIDS statistics — World AIDS Day 2021. Genebra: UNAIDS; 2021 [cited 2023 Mar 21]. Available from: https://www.unaids.org/en/resources/fact-sheet
  » https://www.unaids.org/en/resources/fact-sheet
• ²
  Wollner G, Zimpfer D, Manduric M, Laufer G, Rieger A, Sandner SE. Outcomes of coronary artery bypass grafting in patients with human immunodeficiency virus infection. J Card Surg. 2020;35(10):2543-9.
• ³
  Feinstein MJ, Hsue PY, Benjamin LA, Bloomfield GS, Currier JS, Freiberg MS, et al. Characteristics, prevention, and management of cardiovascular disease in people living with hiv: a scientific statement from the american heart association. Circulation. 2019;140(2):e98-124.
• ⁴
  Sinha A, Feinstein MJ. Coronary artery disease manifestations in HIV: what, how, and why. Can J Cardiol. 2019;35:270-9.
• ⁵
  Touloumi G, Kalpourtzi N, Papastamopoulos V, Paparizos V, Adamis G, Antoniadou A, Chini M, Karakosta A, Makrilakis K, Gavana M, Vantarakis A, Psichogiou M, Metallidis S, Sipsas NV, Sambatakou H, Hadjichristodoulou C, Voulgari PV, Chrysos G, Gogos C, Chlouverakis G, Tripsianis G, Alamanos Y, Stergiou G; AMACS and EMENO. Cardiovascular risk factors in HIV infected individuals: Comparison with general adult control population in Greece. PLoS One. 2020;15(3):e0230730.
• ⁶
  Nunes FC, Caixeta CC, Pinho ES, Souza AC, Barbosa MA. Group technology in psychosocial care: a dialogue between action research and permanent health education. Texto Contexto Nurs. 2019;28:E20180161.
• ⁷
  Hadaya A, Sunardi A. Perception of junior high school students about the use of E-books as learning source. J Sci Educ Technol. 2018;4(1):e2477-3840.
• ⁸
  Huang RH, Liu DJ, Tlili A, Yang JF, Wang HH, Jemni M, et al. Handbook on facilitating flexible learning during educational disruption: the chinese experience in maintaining undisrupted learning in COVID-19 outbreak. 2020; Beijing: Smart Learning Institute of Beijing Normal University [cited 2023 Mar 21]. Available from: https://iite.unesco.org/wp-content/uploads/2020/03/Handbook-on-Facilitating-Flexible-Learning-in-COVID-19-Outbreak-SLIBNU-V1.2-20200315.pdf
  » https://iite.unesco.org/wp-content/uploads/2020/03/Handbook-on-Facilitating-Flexible-Learning-in-COVID-19-Outbreak-SLIBNU-V1.2-20200315.pdf
• ⁹
  Demir KA. Smart education framework. Demir Smart Learn. Environ. 2021;8:29.
• ¹⁰
  Veronese V, Ryan KE, Hughes C, Lim MS, Pedrana A, Stoové M. Using digital communication technology to increase HIV testing among men who have sex with men and transgender women: systematic review and meta-analysis. J Med Internet Res. 2020;22(7):e14230.
• ¹¹
  Polit DF, Beck CT. Fundamentos de pesquisa em enfermagem: avaliação de evidências para a prática da enfermagem. Artmed Editora. 2011. pp. 7-576.
• ¹²
  Prasetya DD, Wibawa AP, Hirashima T, Hayashi Y. Designing rich interactive content for blended learning: a casestudy from Indonesia. Electron J e-Learn. 2020;18(4):277-87.
• ¹³
  Manby L, Aicken C, Delgrange M, Bailey JV. Effectiveness of ehealth interventions for HIV prevention and management in sub-saharan africa: systematic review and meta-analyses. AIDS Behav. 2022;26(2):457-69. Review.
• ¹⁴
  Faludi AA, Izar MC, Saraiva JF, Chacra AP, Bianco HT, Afiune Neto A, et al. Atualização da Diretriz Brasileira de Dislipidemias e Prevenção da Aterosclerose-2017. Arch. Braz Cardiol. 2017;109(2 Supl1):1-76.
• ¹⁵
  Brasil. Ministério da Saúde. Secretaria de Vigilância em Saúde. Protocolo Clínico e Diretrizes Terapêuticas para Manejo da Infecção pelo HIV em Adultos. Brasília (DF): Ministério da Saúde; 2018.
• ¹⁶
  Pasquali L. Psicometria: teoria e aplicações. Brasília (DF): Editora Universidade de Brasília; 1997.
• ¹⁷
  Vianna HM. Testes em educação. São Paulo: Ibrasa; 1982.
• ¹⁸
  Fehring RJ. The fering model. In: Carroll-Johnson RM, editor. Classification of nursing diagnoses: proceedings of the tenth conference. Philadelphia: Lippincott; 1994.
• ¹⁹
  Jasper MA. Expert: a discussion of the implications of the concept as used in nursing. J Adv Nurs. 1994;20(4):769-76.
• ²⁰
  Waltz CF, Bausell RB. Nursing research: design, statistics, and computer analysis. United States: F.A. Davis Company; 1981. p. 380.
• ²¹
  Polit DF, Beck CT. The content validity index: are you sure you know what’s being reported? Critique and recommendations. Res Nurs Health. 2006;29(5):489-97.
• ²²
  Alexandre NM, Coluci MZ. Validade de conteúdo nos processos de construção e adaptação de instrumentos de medidas. Cien Saude Colet. 2011;16(7):3061-8. Review.
• ²³
  Melo ES. Cuide do seu coração: estratégias para redução do risco cardiovascular em pessoas que vivem com HIV. São Paulo: Bibliomundi Serviços Digitais Ltda; 2020. p. 63.
• ²⁴
  Churi P, Pawar A, Moreno-Guerrero AJ. A comprehensive survey on data utility and privacy: taking indian healthcare system as a potential case study. Inventions. 2021;6(3):45. Review.
• ²⁵
  Sung TW, Wu TT. Learning with e-books and project-based strategy in a community health nursing course. Comput Inform Nurs. 2018;36(3):140-6.
• ²⁶
  Burbules N, Fan G, Repp P. Five trends of education and technology in a sustainable future. Geogr Environ Sustain. 2020;1(5):93-7.
• ²⁷
  Mirzaei T, Esmaeilzadeh P. Engagement in online health communities: Channel expansion and social exchanges. Inf Manag. 2021;58(1):103404.
• ²⁸
  Shrestha R, Huedo-Medina TB, Altice FL, Krishnan A, Copenhaver M. Examining the acceptability of mhealth technology in HIV prevention among high-risk drug users in treatment. AIDS Behav. 2017;21(11):3100-10.
• ²⁹
  Bandura A, Azzi RG, Polydoro, S. Teoria social cognitiva: conceitos básicos. Artmed; 2008. p. 177.
• ³⁰
  Klimis H, Thakkar J, Chow CK. Breaking barriers: mobile health interventions for cardiovascular disease. Can J Cardiol. 2018;34(7):905-13. Review.
• ³¹
  Maddison R, Rawstorn JC, Shariful Islam SM, Ball K, Tighe S, Gant N, et al. mHealth Interventions for Exercise and Risk Factor Modification in Cardiovascular Disease. Exerc Sport Sci Rev. 2019;47(2):86-90.
• ³²
  Krick T, Huter K, Domhoff D, Schmidt A, Rothgang H, Wolf-Ostermann K. Digital technology and nursing care: a scoping review on acceptance, effectiveness and efficiency studies of informal and formal care technologies. BMC Health Serv Res. 2019;19(1):400. Review.
• ³³
  Yan ZH, Lin J, Xiao WJ, Lin KM, McFarland W, Yan HJ, et al. Identity, stigma, and HIV risk among transgender women: a qualitative study in Jiangsu Province, China. Infect Dis Poverty. 2019;8(1):94.

Edited by

Associate Editor (Peer review process): Juliana de Lima Lopes (https://orcid.org/0000-0001-6915-6781) Escola Paulista de Enfermagem, Universidade Federal de São Paulo, São Paulo, SP, Brazil.

Publication Dates

• Publication in this collection
  18 Aug 2023
• Date of issue
  2023

History

• Received
  19 Apr 2022
• Accepted
  16 May 2023