Content and semantics validation of an application for adolescents with diabetes mellitus

Abstract

Objective

To validate the content and semantics of the Glicado application for mobile devices aimed at adolescents with type 1 Diabetes Mellitus.

Methods

Methodological study based on the Product Development Process developed in three steps: pre-development, development and post-development. Content was validated by 16 judges and semantics by 14 adolescents.

Results

The content and semantic validation process of the application indicates its appropriateness in terms of functionality, reliability, utility and efficiency. The Content Validity Index was 0.93% and the Semantic Validation Index was 0.90%.

Conclusion

The Glicado application provides important and reliable information and can be used by adolescents as an auxiliary technology in self-control of the disease and health promotion.

Diabetes mellitus, type 1; Adolescent; Mobile applications Chronic disease; Software validation

Resumo

Objetivo

Validar o conteúdo e a semântica do aplicativo Glicado para dispositivos móveis voltado a adolescentes com Diabetes Mellitus tipo 1.

Métodos

Pesquisa Metodológica, baseada no Processo de Desenvolvimento de Produtos, tendo sido desenvolvido em três etapas: pré-desenvolvimento, desenvolvimento e pós-desenvolvimento. O conteúdo foi validado por 16 juízes e a semântica por 14 adolescentes.

Resultados

O processo de validação de conteúdo e semântica do aplicativo indica que ele está adequado em termos de funcionalidade, confiabilidade, utilidade e eficiência. O Índice de Validade de Conteúdo foi 0,93% o Índice de Validação Semântica foi 0,90%.

Conclusão

O aplicativo Glicado disponibiliza informações importantes e confiáveis, podendo ser usado por adolescentes como tecnologia auxiliar no autocontrole da doença e na promoção da saúde.

Diabetes mellitus tipo I; Adolescente; Aplicativos móveis; Doença crônica; Validação de programas de computador

Resumen

Objetivo

Validar el contenido y la semántica de la aplicación Glicado para dispositivos móviles direccionado a adolescentes con Diabetes Mellitus tipo 1.

Métodos

Investigación metodológica, basada en el Proceso de Desarrollo de Productos, desarrollada en tres etapas: predesarrollo, desarrollo y posdesarrollo. El contenido fue validado por 16 jueces y la semántica por 14 adolescentes.

Resultados

El proceso de validación de contenido y de semántica de la aplicación indica que es adecuada en lo que se refiere a funcionalidad, fiabilidad, utilidad y eficiencia. El Índice de Validez de Contenido fue del 0,93 % y el Índice de Validación Semántica del 0,90 %.

Conclusión

La aplicación Glicado provee información importante y confiable y puede ser usada por adolescentes como tecnología auxiliar para el autocontrol de la enfermedad y para la promoción de la salud.

Diabetes mellitus tipo 1; Adolescente; Aplicaciones móviles; Enfermedad crónica; Validación de programas de computación

Introduction

Diabetes mellitus (DM) is a chronic disease characterized by persistent hyperglycemia. It is estimated that 425 million people in the world have DM, a disease with the status of one of the most challenging problems of the 21^st century.(¹,²) Brazil ranks fourth among the ten countries with the greatest number of people with DM (16,780,800 Brazilians aged 20-79 years) and third in the world rank of children and adolescents with type 1 DM (95,846 young people aged 0-19 years).(³)

Diabetes mellitus is classified into type 1 (DM1), type 2 (DM2) and gestational diabetes. In DM1, people produce little or no insulin thus, their bodies are deficient in this hormone. Type 1 DM can be diagnosed at any age, and is generally discovered in childhood and adolescence.(³)

When DM1 affects adolescents (12 and 18 years old),(⁴) care needs to be intensified, requiring greater involvement of families and health professionals; the more knowledge they have about the disease, care and treatment, the easier it will be for them to maintain their glycemic levels and quality of life.(⁵) In this period of life, living with a chronic disease is even more complex, requiring that adolescents accept the disease, have maturity, responsibility and constant care with the diet, medication and restrictions.(⁶)

As adolescents experience many and rapid physical, psychological and social changes, adherence to treatment and maintaining the self-care plan may be more difficult. Thus, it is essential that health professionals consider the particularities of adolescence, plan care together with adolescents and adopt attractive technologies for disease control and treatment of these patients.(³,⁵) Among the technologies available, we highlight Information and Communication Technologies (ICTs), which are in increasing use among adolescents and nursing professionals. They motivate changes in behavior and help with self-monitoring of the disease when associated with health guidelines.(⁷,⁸) Applications (apps) for mobile devices (tablets and smartphones) are the ICTs with the greatest potential to attract adolescents given the ease of access and the possibility of accompanying them everywhere. These devices explore user preferences and promote adherence to the tool, creating new access facilities to various information.(⁹)

As teenagers are among those who use mobile devices and their apps the most, educational software programs are considered important allies in promoting health. Thus, it is essential to use them as a care strategy to control DM1, promote self-care and prevent injuries.(¹⁰,¹¹) In addition to the direct benefits for adolescents, this technology favors their approximation with the health team.(¹²)

However, a search in the literature(¹³) focusing on self-care showed a lack of studies on apps aimed at people living with DM1, especially adolescents. According to a scoping review study, apps aimed at people living with DM are limited in terms of “promoting care and preventing complications related to people with DM”.(¹⁴)

Thus, an emerging challenge is to expand adolescents’ access to safe, reliable and adequate information. The GLICADO [GLIC (blood glucose) + ADO (adolescent)] app for mobile devices was developed to meet this need, and validated after research via Facebook among Brazilian adolescents living with DM1.(⁶)

The aim of this study was to validate the content and semantics of the Glicado app for mobile devices focused on adolescents with DM1.

Methods

Methodological study conducted with 48 parents and 61 adolescents via the Facebook page “Bad type Diabetic” as part of the macro-research “Type 1 diabetes mellitus: perspective of parents and adolescents”.

The content validation judges (nurses, nutritionists and professionals in the field of computing/programming) were recruited from research groups of the National Council for Scientific and Technological Development (Research Directory). The following inclusion criteria were considered: knowing the topics involved (adolescents and/or diabetes and/or Information Technologies/app development); contemplating adolescent themes and/or diabetes and/or Information Technologies/app development in their work activities (teaching and/or research); having professional experience (minimum: two years) and academic training (minimum: master’s degree).

A total of 52 judges (27 nurses, five nutritionists, 20 programmers/computer science professionals) met these criteria. An invitation letter and the Informed Consent form were sent to them by email. Sixteen of them agreed to participate as judges and they received the link to download the app and the Google Forms questionnaire. We considered that the number of judges should be between six and 20.(¹⁵)

Semantic validation was performed by 14 adolescents who live with DM1. They were indicated by the Association of Diabetics and Hypertensives of the city where the study was carried out and selected based on two criteria: being 12-18 years old and having DM1. Although 16 adolescents met these criteria, two of them were excluded for not being available to participate in the study. Convenience sampling was used.

The development of the Glicado app was based on the Product Development Process (PDP) in three phases:(¹⁶,¹⁷) (I) Pre-Development: Strategic planning of the product; (II) Development: informational and conceptual and detailed projects; Production; product launch; (III) Post-Development: product monitoring. The app was developed for the Android operating system by a team consisting of a master’s student, two professors, a graphic designer, a system programmer and an undergraduate nursing student.

Phase I (Pre-development): data from a survey(⁶) conducted with 61 adolescents (15-18 years old) from several states in Brazil, followers of the “Bad Type Diabetic” Facebook page were analyzed. The results of this study supported the choice of contents for the app, mainly aimed at controlling blood glucose and promoting the health of adolescents living with DM.

An integrative review (IR) of the literature was also performed for identification of resources available in mobile apps that favor self-care and self-management of DM1. The inclusion criteria in the review were: studies published from January 2009 to June 2019 (time frame defined by the period of advancement of app development in the health area) available in full online; in Portuguese, English and Spanish; published and indexed in the VHL, PubMed and Scopus. Crossings of descriptors were performed by using the Boolean operator “AND”. In English: Diabetes Mellitus AND mobile applications; in Portuguese: Diabetes Mellitus AND aplicativos móveis; in Spanish: Diabetes Mellitus AND dispositivos móviles. Sixteen articles were analyzed using descriptive statistics with calculation of absolute and relative frequencies.(¹³)

Based on the results of the IR, the features that would be part of the app were defined as follows: insulin dose calculator, carbohydrate intake and counting, diabetes diary (blood glucose records, insulin, diet, exercises, medications, insulin doses, mood), graphs with glycemic averages, hypoglycemia and hyperglycemia notifications, profile customization, diary for food records, textual guidance on DM1 and videos. Afterwards, the interface of the app prototype was developed, followed by phase II (Development, contemplating its production) and phase III (Post-development), when the content and semantics of the app prototype were validated).

The content validation questionnaire was adapted.(¹⁸) The first part was intended to characterize the judges (age, sex, area of training, time and area of expertise, state of residence, highest degree) and the second consisted of 22 questions related to the functionality, reliability, utility and efficiency of the app.(¹⁸) Each question was answered using the Likert scale (values: 1-4; 1: totally appropriate; 2: appropriate; 3: partially appropriate and 4: inappropriate).(¹⁵) A field for comments and/or suggestions was provided for each item.

After analysis and adjustments, semantic validation was performed. The semantic validation questionnaire was also adapted.(¹⁸) It had nine questions related to language, utility, ease of use, content understanding and failures during use; the Likert scale (1-4) was also used.(¹⁵) After parents (and/or legal guardian) signed the informed consent and adolescents signed the Term of Assent, emails were sent to them with a link to download the app and test it for 15 days before answering the questionnaire. The content validation analysis was based on the Content Validity Index (CVI) and semantic validation analysis was based on the Semantic Agreement Index (SAI). Both analyzes considered 0.8 as the minimum coefficient; CVI and SAI values were obtained by adding the answers “Totally appropriate” and “Appropriate”, dividing the sum by the total number of responses.(¹⁹) The recommendations and suggestions of judges and adolescents were accepted and later, the Glicado app was made available for free download on the Play Store.

The study was approved by the local Research Ethics Committee (CAAE 18656819.4.0000.0118; opinion 3.562.709; 09/09/2019), in line with Resolutions (466/2012 and 510/2016) of the National Health Council, and subsidized with resources from the Foundation for Research and Innovation of the State of Santa Catarina (FAPESC; public notice 04/2018).

Results

A total of 16 judges characterized in table 1 validated the app’s content.

Table 1
Characterization of study judges

Characterization	n(%)
Age (mean)	41 years
Sex
Female	11(68.75)
Male	5(31.25)
State of residence
RS	10(62.5)
SC	4(25)
PB	1(6.25)
PR	1(6.25)
Training
Nursing	6(37.5)
Nutrition	2(12.5)
Information Technology	8(50)
Highest degree
Master	3(18.75)
Doctorate	10(62.5)
Postdoc	3(18.75)
Field of practice
Teaching	11(68.75)
Hospital	1(6.25)
Teaching and clinical	2(12.5)
Software development	1(6.25)
Information Technology	1(6.25)
Time of experience (mean)	16.56 years

The general CVI was 0.93% and most (14; 87.5%) items obtained agreement values > 0.80%; although items 4 (reliability) and 6 (utility) obtained CVI values of 0.62% and 0.75%, respectively, not reaching the minimum values of agreement between judges. The results are shown in table 2.

Table 2
Content validity index values of items evaluated by the judges

Items evaluated	Disagreement n(%)	Agreement n(%)	CVI (%)
1) The application contains the main information to help adolescents in the self-care of type 1 diabetes mellitus (functionality)	2(12.5)	14(87.5)	0.87
2) Performance of the application functions is accurate (functionality)	1(6.25)	15(93.75)	0.93
3) The application has access security through password (functionality)	0(0)	16(100)	1
4) The application has flaws (reliability)	0(0)	16(100)	0.62
5) The application reacts properly when failures occur (reliability)	0(0)	16(100)	1
6) It is easy to understand the content of the application (utility)	2(12.5)	14(87.5)	0.87
7) It is easy to learn to use the application (utility)	2(12.5)	14(87.5)	0.87
8) The running time of the application functions is appropriate (efficiency)	0(0)	16(100)	1
9) Application features are appropriate (efficiency)	0(0)	16(100)	1
10) It is easy to install the application (efficiency)	0(0)	16(100)	1
11) Did you find the application useful (utility)	0(0)	16(100)	1
12) The use of the application makes disease control more attractive (utility)	1(6.25)	15(93.75)	0.93
13) The application contributes to the understanding of contents (utility)	1(6.25)	15(93.75)	0.93
14) You would you recommend the app to others (reliability)	0(0)	16(100)	1
15) The application has a language of easy understanding (functionality)	0(0)	16(100)	1
16) The videos included in the application help to understand the information provided (functionality)	0(0)	16(100)	1

Regarding item 4 (reliability), the following adjustments were made: standardization of navigation and creation of an initial tour in the form of a carousel. In item 6 (utility), faults in the commands were corrected. Semantic validation was performed by 14 adolescents (8 females; 6 males). They were aged 12 (3; 21.4%), 13 (1; 7.1%), 15 (4; 28.6%), 16 (3; 21.4%) and 18 years (3; 21 .4%); 12 of them (85.7%) discovered DM1 in childhood (1-10 years) and two (14.2%) in adolescence. They were attending primary (4), secondary (8) and tertiary (2) education. The general SAI was 0.90%; eight out of nine items had SAI>0.80%. Item nine (related to failures presented while using the app) had SAI=0.57% (Table 3).

Table 3
Semantic agreement index values of items evaluated by adolescents

Items evaluated	Disagreement n(%)	Agreement n(%)	SAI (%)
1) The app is interesting and useful	0(0)	14(100)	1
2) Language is appropriate	0(0)	14(100)	1
3) The app is easy to use	1(7.1)	13(92.9)	0.92
4) The app content is easy to understand	1(7.1)	13(92.9)	0.92
5) Use of the application makes disease control more attractive	2(14.3)	12(85.7)	0.85
6) The application helps in self-care of the disease	0(0)	14(100)	1
7) You would you recommend the app to others	0(0)	14(100)	1
8) It is interesting for parents to receive an alert from the application when blood glucose data are inserted and if altered	2(14.3)	12(85.7)	0.85
9) The app presented failures during use	6(42.8)	8(57.2)	0.57

Based on semantic validation, the following adjustments were made: (1) the buttons for adding or subtracting values were standardized and replaced by numeric typing fields directly in the associated text box; (2) in the water intake item, measurement in liters was replaced by glasses; (3) all app commands were defaulted to “save” and “cancel” and (4) bug fixes. Figure 1 shows some screens of the Glicado app.

Figure 1
Glicado app screens

Discussion

The computational potential of smartphones is increasingly visible, and apps have become tools to promote the health of the population living with chronic diseases by helping them with their self-care and management.(²⁰) As most users of apps for mobile devices are adolescents, the use of this resource as an adjunct in disease management is emerging.(¹⁰) In the health area, interfaces of apps range from the simplest (such as informational texts ) to the most interactive (including texts, images, sounds and videos), allowing greater interaction of the user with the app and consequently, with information.(²⁰)

The IR conducted in the present study revealed that apps aimed at adolescents with DM1 are limited in number and resources. However, the studies recognize that this technology favors self-care and self-management of the disease, reinforcing the importance of exploring the needs and perspectives of users to develop effective and satisfactory apps.(¹³)

We believe the Glicado app brings together the most relevant information about DM1, allowing that daily situations experienced by adolescents are contextualized in the virtual world, expanding the reach of guidelines and favoring self-care and self-management of the disease.

The layout and organization of app elements were based on the principle of readability with visual and structural cohesion, allowing an easy understanding of functionality.(²¹) Content validation certified that the app has a simple, usual and pleasant interface, observing the principles of functionality, utility and reliability, and the easy and fast access improves its efficiency.

Regarding reliability (set of attributes that demonstrate the software’s ability to maintain a level of performance in the event of failures),(²¹) validation showed that the app required software adjustments, as both judges and adolescents identified failures during its use. The app was transferred from the Android smartphone database to an online server dedicated to running applications and storing and managing data in order to solve these failures. By using this server, it was possible to organize the software in a structured way and perform backups, which made processing easier on user commands, thereby avoiding failures.

In semantic validation, adolescents indicated the importance of the app having a resource for interaction between users, such as a chat (to share information and knowledge) or a feed (for posts about their daily lives). These considerations corroborated a study(²²) in which the interactivity function was considered important, as it helps users to share information and expand the support system.(²²)

The language, ease of use and content presented in the app were considered adequate by adolescents. In addition, most considered that it makes disease control more attractive. This result also confirms a study(¹⁰) that addresses mobile technologies as important allies in the health education of adolescents.

Item nine of the evaluation (failures) did not reach 0.80, but the app was semantically validated, as the general SAI value reached 0.90. In general, adolescents’ considerations and suggestions for improvements were accepted, but the inclusion of a chat or feed will be done in the future.

In a study(²³) conducted to validate an educational technology, the Likert Scale and CVI calculation with a minimum coefficient of 0.80 were also used. Even though one item resulted below the established coefficient, the overall mean CVI was greater than 0.80, so the technology content was considered validated.

The results allow us to state that the use of apps on mobile devices can help nurses to approach and share information with adolescents living with DM1. Such results reinforce the study(²²) according to which this technology can make assistance, access to information and resources more organized, fast and systematized, expediting and optimizing the time of these professionals in care and management of activities, in addition to narrowing the path of access to users.

Limitations of the present study include the small and convenience sampling, impossibility to include the chat and/or feed so far and the limited availability of the app to the Android platform.

Conclusion

The content of the Glicado app was defined after research in the national and international literature and with adolescents living with DM1, highlighting the potential of its resources that contemplate the needs of the target audience. Content and semantic validation of the Glicado app allows its classification as reliable and safe, since it promotes the health of adolescents living with DM1 and enhances the actions of health professionals.

Acknowledgements

Thanks to the Fundação de Amparo à Pesquisa de Santa Catarina (FAPESC), public call notice FAPESC number 04/2018.

Referências

• ¹
  Silva RH, Gatti MA, Marta SN, Marafon RG, Gatti Neto GG, Andrade EB, et al. Aplicativos de saúde para dispositivos móveis: uma revisão integrativa. Braz J Health Review. 2020;5:11754-65. Review.
• ²
  Sociedade Brasileira de Diabetes. Diretrizes da Sociedade Brasileira de Diabetes 2019-2020. São Paulo: Clannad; 2020 [citado 2021 Maio 15]. Disponível em: http://www.saude.ba.gov.br/wp-content/uploads/2020/02/Diretrizes-Sociedade-Brasileira-de-Diabetes-2019-2020.pdf
  » http://www.saude.ba.gov.br/wp-content/uploads/2020/02/Diretrizes-Sociedade-Brasileira-de-Diabetes-2019-2020.pdf
• ³
  International Diabetes Federation (IDF). IDF Diabetes Atlas: 10th edition. Bruxelas (BE): IDF; 2021 [cited 2021 May15]. Available from: https://diabetesatlas.org/atlas/tenth-edition/
  » https://diabetesatlas.org/atlas/tenth-edition/
• ⁴
  Brasil. Ministério da Mulher, da Família e dos Direitos Humanos. Estatuto da Criança e do Adolescente. Lei nº 8.069, 13 de julho de 1990. Dispõe sobre o Estatuto da Criança e do Adolescente e dá outras providências. Brasília (DF): Ministério da Mulher, da Família e dos Direitos Humanos; 2021. Disponível em: https://www.gov.br/mdh/pt-br/assuntos/noticias/2021/julho/trinta-e-um-anos-do-estatuto-da-crianca-e-do-adolescente-confira-as-novas-acoes-para-fortalecer-o-eca/ECA2021_Digital.pdf/@@download/file/ECA2021_Digital.pdf
  » https://www.gov.br/mdh/pt-br/assuntos/noticias/2021/julho/trinta-e-um-anos-do-estatuto-da-crianca-e-do-adolescente-confira-as-novas-acoes-para-fortalecer-o-eca/ECA2021_Digital.pdf/@@download/file/ECA2021_Digital.pdf
• ⁵
  Greco-Soares JP, Dell’Aglio DD. Adesão ao tratamento de adolescentes com diabetes mellitus tipo 1. Psicol Saúde Doenças. 2017;18(2):322-34.
• ⁶
  Zanatta EA, Scaratti MS, Barichello A, Argenta C. Vivências de adolescentes com diabetes mellitus tipo 1. Rev Enfermagem Refer. 2020;5(4):e20044.
• ⁷
  Ladyzynski P, Krzymien J, Foltynski P, Rachuta M, Bonalska B. Accuracy of automatic carbohydrate, protein, fat and calorie counting based on voice descriptions of meals in people with type 1 diabetes. Nutrients. 2018;10(4):518.
• ⁸
  Colodetti R, Prado TM, Bringuente ME, Bicudo SD. Mobile application for the management of diabetic foot ulcers. Acta Paul Enferm. 2021;34:eAPE00702.
• ⁹
  Adu MD, Malabu UH, Malau-Aduli AE, Malau-Aduli BS. Users’ preferences and design recommendations to promote engagements with mobile apps for diabetes self-management: multi-national perspectives. PLoS One. 2018;13(12):e0208942.
• ¹⁰
  Chaves FF, Carvalho TL, Paraíso EC, Pagano AS, Reis IA, Torres HC. Mobile applications for adolescents with type 1 diabetes mellitus: integrative literature review. Acta Paul Enferm. 2017;30(5):565-72. Review.
• ¹¹
  Oliveira JF, Romano MC, Araújo A, Fiedler MW. Efeito de softwares educativos em adolescentes. Rev Enferm UFPE Online. 2018;12(11):3078-88.
• ¹²
  Alves LF, Maia MM, Araújo MF, Damasceno MM, Freitas RW. Development and validation of a MHEALTH technology for the promotion of self-care for adolescents with diabetes. Cien Saude Coletiva. 2021;26(5):1691-700.
• ¹³
  Scaratti M, Argenta C, Almeida WX, Zanatta EA. Mobile application resources to selfcare and Selfmanagement of type i diabetes mellitus: Integrative review. Rev Fund Care Online. 2021;13:1374-80. Review.
• ¹⁴
  Mourão LF, Marques AD, Moreira TM, Oliveira SK. Aplicativos móveis para promoção de cuidados com pé diabético: revisão de escopo. Rev Eletr Enferm. 2022;24:69625.
• ¹⁵
  Pasquali L. Instrumentação psicológica: fundamentos e práticas. Porto Alegre: Artmed; 2010. 568 p.
• ¹⁶
  Rozenfeld H, Forcellini FA, Amaral DC, Toledo JC, Silva SL, Alliprandini DH, et al. Gestão de Desenvolvimento de Produtos: uma referência para a melhoria do processo. São Paulo: Saraiva: 2006. 576 p.
• ¹⁷
  Salgado EG, Salomon VA, Mello CH, Fass FD, Xavier AF. Modelos de referência para desenvolvimento de produtos: classificação, análise e sugestões para pesquisas futuras. Rev Produção Online. 2010;10(4):886-911
• ¹⁸
  Sperandio DJ. A tecnologia computacional móvel na sistematização da assistência de enfermagem: avaliação de um software [tese]. Ribeirão Preto: Universidade de São Paulo, Escola de Enfermagem; 2008.
• ¹⁹
  Polit DF, Beck, CT. Fundamentos da pesquisa em enfermagem: avaliação de evidências para a prática de enfermagem. 7a ed. Porto Alegre: ArtMed; 2011. 428 p.
• ²⁰
  Silva LC, Silva SL, Oliveira AM, Araujo JR, Arruda IK, Maio R, et al. Cintura hipertrigliceridêmica e fatores associados em crianças e adolescentes portadores de Diabetes Mellitus tipo 1. Rev Paul Pediatr. 2020;38:e2019073.
• ²¹
  Garvin DA. Gerenciando a qualidade: a visão estratégica e competitiva. Rio de Janeiro: Qualitymark; 2002. 376 p.
• ²²
  Silva AM, Mascarenhas VH, Araújo SN, Machado RS, Santos AM, Andrade EM. Mobile technologies in the Nursing area. Rev Bras Enferm. 2018;71(5):2570-8.
• ²³
  Costa CC, Gomes LF, Teles LM, Mendes IC, Oriá MO, Damasceno AK. Construction and validation of an educational technology for the prevention of congenital syphilis. Acta Paul Enferm. 2020;33:eAPE20190028.

Edited by

Associate Editor (Peer review process): Rosely Erlach Goldman (https://orcid.org/0000-0002-7091-9691) Escola Paulista de Enfermagem, Universidade Federal de São Paulo, São Paulo, SP, Brazil

Publication Dates

• Publication in this collection
  15 May 2023
• Date of issue
  2023

History

• Received
  30 July 2021
• Accepted
  19 Dec 2022