Open-access VALIDATION OF A GAMIFIED EDUCATIONAL TECHNOLOGY FOR ARBOVIRUS PREVENTION IN HIGH SCHOOL

VALIDACIÓN DE UNA TECNOLOGÍA EDUCATIVA GAMIFICADA PARA EL CONTROL DE ARBOVIROSES EN ESCUELA SECUNDARIA

ABSTRACT

Objectives:   Arboviruses pose a significant threat to public health, necessitating active community involvement in their prevention and control. Health education is a crucial component of health promotion, and when integrated with technology in school environments, it can effectively motivate behaviour changes in the population. This study aimed to validate the Arboedu technology (a mobile application, web platform, and teacher’s guide) through expert review by professionals in the fields of education and health. The goal was to assess its potential as a resource for promoting preventive practices against arboviruses among high school students.

Method:  This is study employed a quantitative methodological validation approach. The Arboedu prototype was evaluated by 25 highly qualified judges. Online data collection was conducted from August to September 2021. The Content Validity Index (CVI) was used with an 80% cut-off point to validate the following domains: Objectives, Presentation, User-Friendliness, Content, Relevance, and Pedagogical Assumptions.

Results:  Arboedu received a General Agreement Index of 99%, with a lower individual index being 0.90. This validates its effectiveness as a gamified technology for teacher-mediated educational interventions in the prevention of arboviruses.

Conclusion:  The validation of Arboedu by experts in education and health supports its potential to enhance adherence to educational processes and health practices, suggesting its likely acceptance indicating potential acceptance by its target audience. Future research should explore its impact on knowledge acquisition, the adoption of protective practices, and the reduction of arbovirus cases, as well as its contribution to improving students’ self-efficacy levels.

DESCRIPTORS:
Arboviruses; Health education; Adolescent health; Mobile applications; Gamification; Software validation

RESUMO

Objetivos:   As arboviroses são uma ameaça à saúde pública e requerem o engajamento da população no seu enfrentamento. Ações de educação em saúde são essenciais para a promoção da saúde, e quando atreladas a tecnologias no ambiente escolar têm a condição de motivar mudanças de práticas da população. Este estudo teve como objetivo validar com juízes das áreas de educação e saúde, a tecnologia Arboedu (aplicativo móvel, plataforma web e guia do professor), como recurso para promoção de mudanças de práticas na prevenção de arboviroses de estudantes do ensino médio.

Método:   Estudo metodológico quantitativo de validação. O protótipo do Arboedu foi avaliado por 25 juízes com ampla qualificação profissional. A coleta de dados online aconteceu no período de agosto a setembro de 2021. Foi utilizado o Índice de Validade de Conteúdo, com um ponto de corte de 80%, para validação dos domínios: Objetivos, Apresentação, Amigabilidade, Conteúdo, Relevância e Pressupostos Pedagógicos.

Resultados:   O Arboedu foi avaliado com um Índice de Concordância Geral de 99%, com menor Índice individual de 0,90, considerando a sua proposta como tecnologia gamificada para a mediação do professor em intervenções educativas na prevenção de arboviroses.

Conclusão:   A validação do Arboedu por juízes de educação e saúde assegura uma maior aderência a processos educacionais e práticas de saúde, indicando uma aceitação potencial pelo seu público-alvo. Pesquisas futuras precisam investigar seus impactos em conhecimento, adoção de práticas protetivas e redução de casos de arboviroses, além de estudar sua contribuição para o aumento dos níveis de autoeficácia dos estudantes.

DESCRITORES:
Arboviroses; Educação em saúde; Saúde do adolescente; Aplicativos móveis; Gamificação; Validação de software

RESUMEN

Objetivos:   Los arbovirus son una amenaza para la salud pública y requieren del compromiso de la población para combatirlos. Las acciones de educación en salud son esenciales para la promoción de la salud y, cuando se vinculan a las tecnologías en el ambiente escolar, son capaces de motivar cambios en las prácticas de la población. Este estudio tuvo como objetivo validar, con jueces de las áreas de educación y salud, la tecnología Arboedu (aplicación móvil, plataforma web y guía docente), como recurso para promover cambios en las prácticas en la prevención de arbovirus en estudiantes de secundaria.

Método:  Estudio de validación metodológica cuantitativa. El prototipo Arboedu fue evaluado por 25 jueces con amplia cualificación profesional. La recolección de datos en línea se realizó de agosto a septiembre de 2021. Se utilizó el Índice de Validez de Contenido, con un punto de corte del 80%, para validar los dominios: Objetivos, Presentación, Amabilidad, Contenido, Relevancia y Supuestos Pedagógicos.

Resultados:  Arboedu fue evaluado con un Índice de Acuerdo General de 99%, con un Índice individual más bajo de 0,90, considerando su propuesta como una tecnología gamificada para la mediación docente en intervenciones educativas para la prevención de arbovirus.

Conclusión:  La validación de Arboedu por jueces de educación y salud asegura mayor adherencia a los procesos educativos y prácticas de salud, indicando una potencial aceptación por parte de su público objetivo. Es necesario que futuras investigaciones investiguen sus impactos en el conocimiento, la adopción de prácticas protectoras y la reducción de casos de arbovirus, además de estudiar su contribución al aumento de los niveles de autoeficacia de los estudiantes.

DESCRIPTORES:
Arbovirus; Educación sanitaria; Salud de los adolescentes; Aplicaciones móviles; Gamificación; Validación de software

INTRODUCTION

Arboviruses, including dengue virus, Zika virus, chikungunya, and yellow fever, pose a significant global health threat. Among these, dengue is the most prevalent, particularly in the Americas. The proliferation of vector mosquitoes in urban environments has become a persistent challenge for public health in Brazil. Historically, the country has invested in various strategies (educational, mechanical, and biological) to combat mosquito vectors. However, epidemiological data reveal the ineffectiveness of these interventions, largely due to inadequate urban sanitation and low public engagement in mosquito control efforts1,2. These challenges have been exacerbated by climate conditions driven by global warming. By April 2024, Brazil saw a significant rise in dengue cases, with 2,624,300 probable cases and 991 deaths reported3.

Considering this, intensifying health education efforts that promote social participation and encourage changes in habits, attitudes, and practices is essential for sustainable health promotion4. Educational technologies have been used to implement innovative strategies, particularly targeting children and adolescents. These tools contribute to comprehensive approaches that integrate communication, knowledge-sharing, and cross-sectoral collaboration, aligning with the core principles of Brazil’s Unified Health System (Sistema Único de Saúde - SUS)5.

The use of gamification and playful activities in developing educational technologies offers the potential to motivate and engage adolescents in adopting protective practices for improving their quality of life, while also spreading these practices within their families6-9. Gamification involves applying game design elements and techniques in non-game contexts to engage, motivate, and influence behaviors10.

Despite progress in the field, there is still a lack of studies that effectively explore gamification and playful strategies for teaching and social mobilization around the issue of arboviruses. Several initiatives exist to support children and adolescents in combating arboviruses11-13, but most solely focus on providing information, without motivating sustainable behaviour changes. Arboedu was developed in response to this gap, constituting an educational technology designed for students, teachers, and health professionals to interact and collaborate in implementing actions for arbovirus prevention.

This study aims to validate Arboedu (a mobile application, web platform, and teacher's guide) with expert judges from the education and health sectors, assessing its effectiveness in promoting behaviour changes for arbovirus prevention among high school students. This represents the first stage of Arboedu’s validation process, where health and education professionals evaluate whether the educational model aligns with its objectives. Validation by expert judges is a crucial step to ensure the technology's suitability before its deployment with the target audience. This study evaluates Arboedu in terms of its objectives, presentation, user-friendliness, content, relevance, and adherence to pedagogical principles.

As an educational tool, Arboedu can play a key role in health promotion, facilitating knowledge construction about arboviruses and encouraging changes in socio-environmental practices among children and adolescents, in line with the UN 2030 Agenda14.

METHOD

This study used a methodological development approach to validate an educational technology (Arboedu) through expert judges. The goal of this stage was to validate the prototype, allowing necessary adjustments before its application to the target audience.

Arboedu is an educational technology designed to support the teaching-learning process about arboviruses for high school students aged 14 to 19. It comprises a mobile application, a web platform, and a teacher’s guide, aiming to foster behavioural changes among adolescents and their families by reflecting on socio-environmental attitudes in the fight against arboviruses. The technology employs gamification and playful elements aligned with the National Common Curricular Base (BNCC)15, mobile learning16, and the self-efficacy concept from Social Cognitive Theory (SCT) 17.

Games are instrumental in developing various skills, understanding reality, and promoting social roles18, while enhancing engagement and making learning more enjoyable19. Gamification elements like rewards and feedback are used alongside games to foster interaction, cooperation, collaboration, and knowledge sharing among students9. Arboedu mobilizes nine BNCC competencies: knowledge; scientific, critical, and creative thinking; cultural repertoire; communication; digital literacy; argumentation; self-knowledge and self-care; empathy and cooperation; and responsibility and citizenship15.

Self-efficacy, a key component of SCT, refers to “people’s judgments about their capabilities to carry out actions necessary to achieve specific performance levels” 17. This concept is used to motivate students and build skills and knowledge essential for preventing arboviruses17. Throughout the activities, questions and messages prompt reflection and reinforce self-efficacy beliefs.

High school teachers act as mediators in the scavenger hunt-style educational process, guiding students through “missions.” Teachers can communicate with students, manage activities, and monitor progress via the web platform and the teacher’s guide. Health professionals also contribute by clarifying questions and presenting the role of public authorities in arbovirus prevention. The web platform highlights knowledge gaps and the absence of localized practices, which can inform local policies and health education (Figure 1).

Figure 1-
Screenshots of the Arboedu application and web platform.

The app includes five challenges (QuizArbo, Patrulha em Foco, Janelas & Tanques, Coleta & Reciclagem, and ArboTurma) structured around knowledge assessments and attitudinal and motivational activities presented as missions. “QuizArbo” deepens students’ knowledge of arboviruses and vector biology. “Patrulha em Foco” focuses on vector control, incorporating preventive practices into students' daily lives and family routines, with students addressing risk situations using mobile phone geolocation. In “Janelas & Tanques,” students assess risks related to windows and water storage, and are encouraged to use protective screens at home. “Coleta & Reciclagem” emphasizes the importance of selective waste collection, proper disposal, and sustainability, while also addressing entomological surveillance and the water cycle. Finally, “ArboTurma” is a collaborative, creative challenge in which teams produce an audiovisual presentation on the subject, forming the basis for a competition.

Students are guided by videos and infographics, fostering an engaging and light-hearted interaction, with minimal text. Activities are scored through the app, generating rankings for students, classes, and schools to motivate participation.

A non-probabilistic, quantitative approach was employed to gather objective, measurable data on the perceptions of health and education professionals regarding the technology20. The study took place in Recife between August and September 2021. Evaluators were selected based on convenience and required qualifications. A total of 25 judges participated: 10 from the health field and 15 from education21.

The education group included high school teachers, support educators, pedagogical coordinators, managers, assistant managers from Recife’s municipal and state education networks, and undergraduates in Biological Sciences at the Federal University of Pernambuco (UFPE). The health group consisted of professionals from the Executive Board of Health Surveillance and the Centre for Environmental Surveillance and Zoonosis Control in Recife, who were either involved in education or responsible for controlling mosquito vectors.

Eligibility criteria for judges followed previous studies, requiring a minimum score of five points based on experience and qualifications22. Criteria included thesis or dissertation completion, specialized courses, publications, professional experience, researcher roles, use of educational software, and enrolment in a biological sciences degree. Judges who failed to complete the form within the set deadline were excluded.

An assessment tool based on previous research23 was adopted to systematically evaluate the pedagogical quality of the technology. The sections related to hardware and software assessment were excluded, considering the diversity of technological scenarios. Instead, the instrument focused on pedagogical domains such as Objectives (Chart 1), Presentation and User-Friendliness, Content, Relevance, and Pedagogical Assumptions (Chart 2). The health professionals did not evaluate the Pedagogical Assumptions domain, which was only reviewed by education professionals.

The instrument used a Likert scale with five response options: Completely Agree (A), Partially Agree (PA), Neither Agree nor Disagree (NAND), Partially Disagree (PD), and Completely Disagree (CD). An open-ended section was included for comments and suggestions in each domain.

Chart 1-
Objectives Domain. Recife, PE, Brazil, 2022.
Chart 2-
Pedagogical assumptions Domain. Recife, PE, Brazil, 2022.

Data collection was conducted online through a web platform, where participants were introduced to the prototype, user guide, and assessment tools. After exploring the technology, participants completed the judge’s profile form (health or education).

The results were analysed using the Content Validity Index (CVI), which measures the degree of agreement among the judges. A cut-off points of 80% (0.80) was set as the threshold for validity20. Data were organized into absolute numbers, averages, medians, and percentages, using Microsoft Office Excel 2016 and BioEstat software. Qualitative analysis was also performed on the comments and suggestions provided by the judges.

This study was conducted during the COVID-19 pandemic, following approval by the university’s Research Ethics Committee. Judges gave their consent online. Due to pandemic restrictions, data collection was exclusively remote. Despite engagement strategies and personalized communication, participants' readiness may have been affected by emotional and mental challenges posed by the pandemic. Support materials were provided to clarify the technology's use and validation process to mitigate these limitations, enhancing the study’s quality.

RESULTS

The study involved 25 judges, with 10 (40%) from the health sector and 15 (60%) from the education sector (Table 1). The participants from the health sector were all female, with an average age of 34 years (SD ± 8.04), ranging from 24 to 49 years. Four participants (40%) had a master’s or doctorate in collective health, family health, or public health. Of the health professionals, 40% (n = 4) had published articles on arboviruses (10%) or educational software validation (30%), with an average of 3.25 published articles (SD ± 6.74). In terms of professional experience, 70% had teaching or clinical experience in family health, collective health, public health, or child and adolescent health, averaging 5.5 years (SD ± 6.16). All health judges reported using educational software in their professional practice.

The education sector was represented by school managers, educators, and undergraduate students in biological sciences. The group included seven women (63.63%) and four men (36.37%), with an average age of 42.45 years (SD ± 7.79), ranging from 30 to 55 years. Seven participants (63.63%) had at least one postgraduate degree, averaging two per person. All had teaching experience, with an average of 17 years, and 10 had worked specifically with adolescents, averaging 15.54 years. Only one judge was a researcher in educational software validation or arboviruses. Nine of the 11 participants used educational software in their professional practice. Additionally, the profile included four biological sciences undergraduates, three female and one male, with an average age of 24 years (SD ± 1.82). All were involved in research or extension projects in education and had teaching internship experience with children and adolescents.

Judges scored between five and 90 points, with a median of 24 (Table 1). Given the high scores, the judges were considered qualified to evaluate and offer meaningful feedback on Arboedu.

Table 1
Characterization of experts according to the adopted requirements. Recife, PE, Brazil, 2022.

The educational technology was validated with an overall Content Validity Index (CVI) of 0.99 in both profiles, and an overall agreement percentage of 99.33%, exceeding the cutoff point in each domain evaluated. As shown in Table 2, the results confirm the educational technology's adequacy, with the high CVI and agreement percentages reflecting both compliments and suggestions recorded in the open-ended responses in each domain.

Table 2
Overall Content Validation Index and Agreement Percentage. Recife, PE, Brazil, 2022.

Objectives Domain

The 10 health judges assigned a CVI for each item in the objectives domain ranging from 0.90 to 1.00, resulting in a CVI of 0.99 and an agreement percentage of 98.57%. Item 4 (Chart 1), which assesses whether the application promotes reflection on surveillance, prevention, and control networks for mosquito vector breeding sites, received the only neutral (NAND) evaluation, with the justification: “The reflection is still in its infancy; perhaps the application should encourage collective debate within the challenges by proposing a statement or problem.” Nonetheless, the item still had an agreement index of 0.90.

All items in the objectives’ domain in the education profile received the maximum CVI-I (1.00). Item 3 (Chart 1), which assesses the application’s ability to foster critical reflection on socio-environmental habit changes, had the most “Partially Agree” evaluations, but no justifications were provided.

Presentation and User-Friendliness Domain

All items in the health profile scored a CVI-I of 1.00, yielding a CVI-D of 1.00 and a 100% agreement percentage. The item on whether the language is appealing to adolescents received the highest rate of partial agreement (PA), justified by “grammatical errors, complex expressions, and quiz questions about incorrect answers.”

The CVI-I scores in the education profile ranged from 0.93 to 1.00, resulting in a CVI-D of 0.99 and an agreement percentage of 98.83%. The items with the lowest validation indices were item 2 (whether the application provides motivational stimuli) and item 4 (whether the language is appealing to adolescents). The high agreement percentage (99.41%) indicates positive feedback on user interaction and ease of use.

Content Domain

This domain received 100% agreement in the health profile, with CVI-I and CVI-D both equal to 1.00. Moreover, three items in the education profile scored slightly lower, with an agreement percentage of 99.18%. These items were related to i) the life cycle of Aedes and the use of screens for windows and water tanks; ii) strategies for preventing arboviruses and combating Aedes; and iii) the reproductive cycle of vectors.

The overall content domain was evaluated with a CVI-D of 0.99 and an agreement percentage of 99.09%.

Relevance Domain

The relevance of Arboedu was assessed by the health profile with a CVI-D of 0.99 and an agreement percentage of 98.57%. The lowest agreement index (0.93) in the education profile was for the item verifying “learning experiences that position the adolescent as an active participant in socio-environmental practices.” One judge provided a neutral (NAND) response, but did not offer any justifications. The high CVI-D indicates Arboedu’s relevance and potential for health education on arboviruses.

Pedagogical Assumptions Domain

This domain was only evaluated by education professionals, and all items (Chart 2) received the maximum CVI-I, CVI-D, and agreement percentage.

DISCUSSION

The evaluation results confirmed that Arboedu can achieve its objectives. However, its effectiveness significantly relies on teacher mediation to realize the desired outcomes. The literature emphasizes that the application of educational technologies should facilitate learning mediation, fostering collaboration between students and teachers as co-participants in the process24. Teacher mediation maximizes the impacts of technology, aiding in the achievement of its objectives.

The application’s content was developed based on formal references25. The judges suggested including symptoms and effects of self-medication, which were addressed in an introductory video in the new version. Additionally, the judges found the quiz questions to be complex. The question bank was designed to enhance adolescents’ argumentation, reading, writing, questioning, and critical thinking skills while increasing their knowledge on the subject. Although the quiz was intentionally challenging, it aimed to avoid discouraging student participation, as recommended in the literature26. Balancing difficulty and engagement are essential for gaining student buy-in, which should be evaluated with this audience in future studies.

The evaluations regarding presentation and user-friendliness highlighted the complexity of terms, aesthetics, and feedback. The mobile application aimed to utilize informal language tailored to high school students. However, some terms related to arboviruses (considered complex by the evaluators) were intentionally retained, as they are necessary vocabulary for college entrance exams. The grammatical errors reported were corrected.

The application interface received positive evaluations. A quality interface fosters better interaction and communication, facilitating the adoption of a technological solution24. The design incorporated videos, infographics, and graphic elements for this purpose. Another crucial aspect of communication and knowledge construction is the process of providing feedback on errors and successes27. Errors in Arboedu were used to stimulate reflection, transforming them into learning opportunities. Feedback was rated positively; however, the encouraging messages were enhanced in the new version at the evaluators' request.

Concerns were raised in the “Relevance” domain regarding the importance of Arboedu in combating arboviruses. The application aimed to facilitate the inclusion of practical activities in schools that stimulate creativity and experience sharing, in line with the BNCC15. Furthermore, Arboedu sought to encourage students to take an active role in the suggested actions. According to researchers4, community participation is vital for health education initiatives to be more effective, fostering changes in habits, attitudes, and practices within society.

The pedagogical assumptions domain is the most significant for the Arboedu evaluation, receiving the highest ratings. This domain demonstrated coherence between the technology and the pedagogical proposal on which it was based. Arboedu utilized various pedagogical resources, such as gamification and playfulness, to enhance collaborative learning, motivation, and engagement, as recommended in the literature28. Studies on the role of gamification as a resource that enhances user experience report its positive contributions to student engagement29. By aligning with the BNCC, Arboedu sought to connect technology to the school curriculum, enriching teaching practices on the topic of arboviruses30.

Beyond building knowledge, Arboedu aims to promote sustainable changes in health protection practices for children and adolescents, which is a considerable challenge4. Consequently, gamification methods were employed, such as encouraging the disposal of recyclable solid waste at collection points of electricity companies. This aspect received positive feedback from the judges, but its effectiveness requires evaluation in future research with the target audience.

In the context of public schools, leveraging technological resources is essential to enhance student engagement in their learning. This presents a challenge given the financial difficulties in education in Brazil. A technology like Arboedu necessitates public state resources (from high school management) to host the web platform, which will provide valuable inputs for education and health. Demonstrating its potential may help persuade public agencies of its benefits for both education and health, as well as its importance in combating arboviruses in the national context, especially with the recent increase in cases of Oropouche Fever.

CONCLUSION

Arboedu is an educational technology designed to engage adolescents and their families in adopting protective measures against arboviruses. This study demonstrated the suitability of Arboedu for this purpose through a validation process involving judges from health and education sectors, resulting in an Overall CVI of 0.99 across the following domains: objectives, presentation and user-friendliness, content, relevance, and pedagogical assumptions. However, the accessibility and efficiency criteria related to “usability” were not evaluated at this stage, which constitutes a limitation of this study. The initial version of the tool did not incorporate special access features for different types of students. The efficiency criterion will be assessed in future, more developed versions of the software.

The validation of an educational tool with a high degree of acceptance by teachers and health professionals ensures its alignment with educational processes and health practices, indicating potential acceptance by the target audience. Future research will focus on assessing acceptance by the target audience, identifying impacts on knowledge acquisition, sustainability of healthy practices, and reduction of arbovirus cases. Furthermore, it is equally important to investigate the impacts on students' levels of self-efficacy concerning arboviruses.

REFERENCES

  • ORIGIN OF THE ARTICLE
    Extracted from the dissertation - Validação de uma Tecnologia Educacional Baseada em Gamificação para o Enfrentamento de Arboviroses no Ensino Médio, presented to the Postgraduate Program in Child and Adolescent Health, of the Center of Medical Sciences of the Universidade Federal de Pernambuco, in 2022.
  • FUNDING INFORMATION
    The authors would like to acknowledge the support received from the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) through the PROPG Notice No. 02/2021, and the Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) through the PROPESQI Notice No. 02/2022.
  • APPROVAL OF ETHICS COMMITTEE IN RESEARCH
    Approved by the Ethics Committee in Research of the the Health Sciences Centre, Universidade Federal de Pernambuco, Opinion n.º 4,916,336, Certificate of Presentation for Ethical Assessment n.º 30742320.8.0000.5208.
  • TRANSLATED BY
    Christopher J. Quinn.

Edited by

  • EDITORS
    Associated Editors: Luciara Fabiane Sebold, Maria Lígia Bellaguarda.
    Editor-in-chief: Elisiane Lorenzini.

Publication Dates

  • Publication in this collection
    10 Jan 2025
  • Date of issue
    2024

History

  • Received
    18 May 2024
  • Accepted
    16 Sept 2024
location_on
Universidade Federal de Santa Catarina, Programa de Pós Graduação em Enfermagem Campus Universitário Trindade, 88040-970 Florianópolis - Santa Catarina - Brasil, Tel.: (55 48) 3721-4915 / (55 48) 3721-9043 - Florianópolis - SC - Brazil
E-mail: textoecontexto@contato.ufsc.br
rss_feed Acompanhe os números deste periódico no seu leitor de RSS
Ir para o topo Reportar erro