Development and usability of a photograph-based smartphone app for ocular surface tumor screening

Desenvolvimento e usabilidade de um aplicativo para smartphones baseado em fotografias para rastreio de tumores de superfície ocular

ABSTRACT

Objective:

To develop and examine the usability of a mobile application aimed at improving knowledge about ocular surface tumors among medical students, general practitioners and ophthalmologists.

Methods:

A cross-sectional quantitative study was carried out by a multidisciplinary team and developed in three stages: administration of a specific questionnaire to medical students, assessing the demand for the application, creation and development of the application in collaboration with the Technological Innovation Laboratory of Centro Universitário Christus and usability assessment. General practitioners and ophthalmologists were selected to answer the System Usability Scale questionnaire. Data were exported into the Statistical Package for the Social Sciences, version 20.0 Windows, for quantitative analysis.

Results:

The OncoEye application was developed for the iOS and the Android platforms and included four main menus: Glossary, Patients, Referrals and Queries. Most students (89.9%) considered the development of the application important for wider access to information about the topic. Most students (39.9%) were able to recognize ocular lesions and 26.1% could distinguish between benign and malignant conditions. System Usability Scale questionnaire responses revealed good usability, with an average score of 92.65. Users (100%) described the application as user-friendly and well- integrated.

Conclusion:

An innovative application for ocular surface tumors was created and developed. The application achieved good levels of acceptance and was described as user-friendly by users.

Keywords:
Computers, handheld; Mobile applications; Eye neoplasms; Ophthalmology; Eye

RESUMO

Objetivo:

Desenvolver e avaliar a usabilidade de um aplicativo para dispositivos móveis que aprimore o conhecimento de estudantes de medicina e de médicos sobre tumores da superfície ocular.

Métodos:

Foi realizado estudo transversal e quantitativo por uma equipe multiprofissional, consistindo em três etapas: realização de um questionário específico com estudantes de medicina, avaliando a necessidade de elaboração do aplicativo; criação e desenvolvimento do aplicativo pelo Laboratório de Inovação Tecnológica do Centro Universitário Christus e avaliação de usabilidade. Médicos generalistas e oftalmologistas foram selecionados para responder ao questionário System Usability Scale. Os dados foram exportados para o programa Statistical Package for the Social Sciences, versão 20.0, para Windows para análise quantitativa.

Resultados:

O aplicativo OncoEye foi desenvolvido com quatro menus principais: Glossário, Pacientes, Encaminhamento e Perguntas. Dentre os estudantes, 89,9% consideraram o desenvolvimento do aplicativo importante para orientação sobre o tema, 39,9% souberam reconhecer lesão ocular, e 26,1% puderam diferenciar lesão ocular maligna ou benigna. As respostas dos usuários à ferramenta System Usability Scale demonstraram boa usabilidade, com pontuação média de 92,65 (87,74 a 97,55). Todos os usuários consideraram o aplicativo de fácil manuseio e bem integrado.

Conclusão:

Um aplicativo inovador para tumores da superfície ocular foi criado e desenvolvido, apresentando boa aceitação e fácil manuseio pelos usuários.

Descritores:
Computadores de mão; Aplicativos móveis; Neoplasias oculares; Oftalmologia; Olho

INTRODUCTION

Ocular surface tumors (OST) are rare. However, these lesions may carry a poor prognosis. Thus, an early diagnosis is critical. The diagnosis of OST can be based on medical history and findings of specialized eye exams.(¹) The most common malignant tumors of the cornea and conjunctiva are ocular surface squamous neoplasia, conjunctival melanoma and conjunctival lymphoma.(²) Several factors contribute to OST development, including exposure to ultraviolet radiation, advanced age, male gender and primary or acquired immunodeficiency.(³)

In this context, OST diagnosis can be extremely challenging, since not all physicians are familiar with these lesions, particularly in cases with atypical clinical presentations. Histopathological examination is the gold standard diagnostic method. However, many alternative tools and exams can be used to assist OST diagnosis and management.(³)

Mobile applications developed for use in oncology are on the rise worldwide and offer new alternatives to assist surgeons and clinicians in daily practice. Several applications are available for iOS and Android platforms, especially for breast and skin cancer, which provide information for patients and contribute to appropriate management of these conditions.(⁴) In ophthalmology there is also plenty of room for the development of new smartphone-based tools aimed at facilitating daily routines.(⁵)

This study set out to develop and examine the usability of a mobile application aimed at improving knowledge about OST among medical students, general practitioners and ophthalmologists.

METHODS

Study design

This was a transversal and quantitative study developed in three stages. First, a questionnaire was administered to medical students in order to identify the demand for the application. The mobile application was then created and developed. Finally, the usability of the mobile application was examined using a validated questionnaire administered to general practitioners and ophthalmologists. All participants signed an informed consent form.

This study was approved by the Ethics Committee (CAAE: 02982818.1.0000.5049), in compliance with ethical guidelines of resolution 466/12 of the “Conselho Nacional de Saúde”.

Assessment of the demand for the mobile application

A questionnaire was developed to assess the demand for an OST-related application among third-year medical students at Centro Universitário Christus (Unichristus), in Fortaleza (CE), Brazil. Questionnaire items examined the profile of participants, previous knowledge about OST, difficulties in accessing OST-related information and the demand for an OST-related application.

Application development

The application was created and developed by a multidisciplinary team, in collaboration with Unichristus Technological Innovation Laboratory. An adapted version of the co-design approach was used, in which team members participated actively in application development.(⁶) The OncoEye application was created and developed in the Portuguese and the English languages for use in mobile devices compatible with the iOS and the Android platforms.

To access the application, a home screen with four main menus located at the bottom was developed. The first menu (Glossary) is automatically activated, and three icons (Clinical Appearance, Tumors and Treatment) appear in a single vertical column.

The Clinical Appearance icon comprises two alternatives (Suggestive of Benignity and Suggestive of Malignancy), each containing five major clinical characteristics of each lesion, followed by a brief explanation (Figure 1).

Figure 1
Clinical appearance menu.

By clicking on Tumors, two alternatives (Benign Tumors and Malignant Tumors) appear, each containing descriptions and images of the major benign and malignant ocular surface lesions. Nine benign and five malignant lesions were included. The name of each lesion corresponds to an icon that can be activated to depict colored images of the typical clinical appearance of the selected lesion, followed by a brief description of the respective epidemiology, clinical characteristics and first line treatments (Figure 2).

Figure 2
Tumors menu.

The last icon in the glossary (Treatment) describes current management strategies of the lesions described. This topic was subdivided into four sections: Observation/Follow-up, Surgical Removal, Topical Chemotherapy and Radiotherapy, each containing a brief, clear description to facilitate user understanding.

The second menu at the bottom (Patients) displays a “+” button that can be used to enter patient data. Photographs of lesions can be stored in this section of the application, so that patients can save their data and images in a cloud-based virtual network that enables follow-up throughout the national territory.

The next menu (Referrals) contains the names of the leading referral institutions in Brazil and ophthalmologists specializing in ocular oncology who provide care via the Brazilian public health system. In this manner, application users can refer patients to the closest specialized care service.

The last menu (Queries) provides a fast of communication channel through which users can send e-mails directly to application developers.

Mobile application user feedback

Mobile application usability assessment was carried out with the help of volunteers who agreed to use the application and to answer the System Usability Scale (SUS) questionnaire.

General practitioners and ophthalmologists who carried smartphones equipped with the iOS or the Android system were eligible. Exclusion criteria were as follows: non-medical volunteers and general practitioners or ophthalmologists unfamiliar with smartphones equipped with the iOS or the Android system.

A demo of the application including the respective menus and functions was delivered. After initial instructions were provided, volunteers were allowed to access the application and its functionalities. Volunteers were then instructed to download the application onto their smartphones and test the application.

At this point, participants were invited to answer a questionnaire containing questions about their profile (age, gender, year of graduation and medical specialty). The usability of the application was assessed using the SUS questionnaire, which comprised ten questions with the following response alternatives: strongly disagree, disagree, neutral, agree and strongly agree.

Statistical analysis

Data extracted from the SUS questionnaire were exported into the Statistical Package for the Social Sciences (SPSS) software, version 20.0 for Windows. Mean subcategory and overall scores (zero to one hundred) were provided. The Cronbach’s alpha was then calculated for positive and negative questionnaire items (optimal values>0.7) and if each item was deleted.

Finally, mean SUS questionnaire scores were compared with remaining variables (age, sex, year of graduation and medical specialty) using the Mann-Whitney test.

RESULTS

Assessment of the demand for the mobile application

A total of 138 third-year medical students, 95 women and 43 men (68.8% and 31.1% respectively), answered the questionnaire. Of these, 97 (70.2%) were aged 20 to 23 years. Only four volunteers were aged over 31 years.

The potential availability of an OST-related application appealed to third-year Unichristus medical students, as 89.9% considered the development of an application containing information about ocular surface tumors to be important, even though this is a topic reserved for specialists (Table 1).

Table 1
Assessment of the demand for the application

Question	Yes	No
Are you able to recognize eye involvement?	39.9	60.1
Are you able to distinguish between malignant and benign eye lesions?	26.1	73.9
Did you have previous knowledge of ocular surface tumors?	58.7	41.3
Do you believe there is limited information about ocular surface tumors?	84.1	15.9
Do you find the development of an application related to ocular surface tumors important?	89.9	10.1

• Results expressed as %.

Mobile application user feedback

Seventeen volunteers participated in this analysis (ten women and seven men; 58.8% and 41.1% respectively). Volunteers were aged 26 to 36 years (average age, 31 years). The year of graduation of volunteers ranged from 2008 to 2018. Nine (52.9%) were general practitioners and eight (47.0%) were ophthalmologists. Mean SUS questionnaire scores per group are shown in Table 2.

Table 2
System Usability according to volunteer profile

Evaluated parameters		SUS	p-value
Age
	30 years or less	86.50±13.99	0.237
	>30 years	95.21±6.07
Sex
	Female	92.05±10.54	0.720
	Male	93.75±8.18
Graduated from medical school
	6 years or less	91.07±12.98	0.960
	>6 years	93.75±6.80
Ophthalmology specialist
	No	92.22±11.49	0.961
	Yes	93.13±7.53

• Results expressed as mean ± standard deviation. Mann-Whitney test.
• * p<0.05.
• SUS: System Usability Scale.

Analysis of SUS questionnaire responses revealed good acceptance of the application among users, as shown by an average usability score of 92.65 (Table 3).

Table 3
System Usability Scale questionnaire findings

SUS questions	Descriptive statistics	Cronbach’s alpha
1. I believe I would like to use this system often	4.47±0.62	0.750*
2. I find the system unnecessarily complex	1.24±0.56	0.869*
3. I found the system easy to use	4.82±0.39	0.590*
4. I believe I would need help from a person with technical expertise to use the system	1.35±0.79	0.890*
5. I think the various functions of the system are very well integrated	4.65±0.49	0.701*
6. I think the system is very inconsistent	1.35±0.61	0.843*
7. I believe that people will learn how to use this system quickly	4.88±0.33	0.651*
8. I found the system awkward to use	1.18±0.39	0.912*
9. I felt confident while using the system	4.65±0.61	0.551*
10. I needed to learn several new things before I was able to use the system	1.29±0.47	0.912*
Cronbach’s alpha of positive items	-	0.703†
Cronbach’s alpha of negative items	-	0.908†
SUS score	92.65±9.54	-

• Result expressed as mean ± standard deviation.
• ^*
  Cronbach’s alpha if the item is deleted;
• ^†
  Cronbach’s alpha.
• SUS: System Usability Scale.

Volunteers declared that the application was easy to use and only 5.9% thought they would need technical support. Users (100%) also disagreed that many things had to be learned in order to use the application. As to consistency of the application, 94.1% denied any inconsistencies and only 5.9% were indifferent to its complexity. The vast majority of users (94.1%) declared they would use this application often. As to completeness of application functions, volunteers (100%) thought functions were well integrated and that anyone could quickly learn to use the application (Table 4).

Table 4
Frequency of System Usability Scale questionnaire responses

SUS questions	Strongly disagree	Disagree	Neutral	Agree	Strongly agree
1. I believe I would like to use this application often	-	-	5.9	41.2	52.9
2. I found this application unnecessarily complex	82.4	11.8	5.9	-	-
3. I found the application easy to use	-	-	-	17.6	82.4
4. I believe I would need help from a person with technical expertise to use this application	76.5	17.6	-	5.9	-
5. I found the various functions of the application to be well integrated	-	-	-	35.3	64.7
6. I felt there is a lot of inconsistency in the application	70.6	23.5	5.9	-	-
7. I imagine that most people can learn to use this application very quickly	-	-	-	11.8	88.2
8. I found the application very complicated to use	82.4	17.6	-	-	-
9. I felt very confident while using this application	-	-	5.9	23.5	70,6
10. I needed to learn several things before I could start using this application	70.6	29.4	-	-	-

• Results expressed as %.
• SUS: System Usability Scale.

DISCUSSION

In Brazil, the high incidence of sunlight all year long is an important risk factor for the development of benign and malignant OST. Although most OST lesions are benign, malignant transformation may occur, leading to structural damage or death. Also, the level of awareness about ocular tumors among the general population is low, which makes the diagnosis of these lesions even more difficult.(⁷)

Mobile device applications have been gradually introduced in healthcare settings in the past two decades. In ophthalmology, mobile applications can be divided into several categories: aimed at ophthalmologists, optometrists, general practitioners, medical students or patients.(⁵,⁸) The OncoEye is primarily an educational application for general practitioners, ophthalmologists and medical students.(⁹) Hence, mobile applications can combine clinical examination and educational features in a portable and intuitive platform.(¹⁰)

Findings of OncoEye application usability assessment revealed positive responses in all categories. Ophthalmologists and general practitioners described the application as a user-friendly tool with well-integrated functions and capable of providing a fast-learning experience, regardless of background knowledge. Lack of significant gagging or locking translated into high confidence levels among users.

In this study, groups with different profiles regarding prior experience with OST were compared. However, in spite of high levels of heterogeneity, differences between ophthalmologists and general practitioners were not statistically significant. General practitioners were expected to find the application less user-friendly than ophthalmologists. However, this was not the case.

In order to obtain an appropriate confidence interval (confidence interval of 90%), 12 participants (n=12) would have to answer the SUS questionnaire.(¹¹) This sample comprised 17 participants.

Mobile applications can have complex interfaces and may require technical support for appropriate use. In this study, 94.1% of volunteers reported they did not need technical support to use the application. Users (100%) also felt that the application had well integrated functions.

The use of mobile applications in ophthalmology may assist patient follow-up, contributing to better management of clinical conditions.(¹²) In this context, self-testing applications may be particularly useful for patients who live in remote areas with limited resources, since they provide access to a physician regardless of physical distance.(¹³)

Comparative analysis of the iOS and the Android platforms revealed the first does not support older applications and requires more frequent updates. In contrast, the Android platform offers more support to older applications, contributing to longer use. Also, the purpose of many applications is to serve as an advertisement tool for ophthalmology services. Apple’s HealthKit stands out in this category and can be used to improve communication between health care providers and patients. Such tools serve a marketing purpose and facilitate the integration of ophthalmic applications.(¹⁴)

One of the limitations of OncoEye is the need of an internet connection to use the application, which limits its use in regions with poor internet infrastructure. Sadly, these are the areas where the application would be most beneficial, given the lack of local specialized ophthalmology services.

CONCLUSION

An innovative mobile application for ocular surface tumors could be appropriately developed and submitted to usability assessment. The application has a user-friendly interface and may contribute to better management of patients with these lesions, increasing the knowledge about these diseases.

• Institution: Centro Universitário Christus, Fortaleza, CE, Brazil.
• Financial support: the application was created and developed by a multidisciplinary team, in partnership with the Technological Innovation Laboratory of the Centro Universitário Christus; there was no funding support to develop of the application and write the article.

ACKNOWLEDGEMENTS

We want to express our gratitude to all volunteers and organizations that contributed to writing of this manuscript.

REFERENCES

• ¹
  Mirzayev I. Factors affecting recurrence after surgical treatment in cases with ocular surface squamous neoplasia. Int J Ophthalmol. 2019;12(9):1426-31.
• ²
  Shields C, Shields J. Tumors of the conjunctiva and cornea. Indian J Ophthalmol. 2019;67(12):1930-48.
• ³
  Nanji AA, Mercado C, Galor A, Dubovy S, Karp CL. Updates in Ocular Surface Tumor Diagnostics. Int Ophthalmol Clin. 2017;57(3):47-62.
• ⁴
  Charbonneau DH, Hightower S, Katz A, Zhang K, Abrams J, Senft N, et al. Smartphone apps for cancer: A content analysis of the digital health marketplace. Digit Health. 2020;6:205520762090541.
• ⁵
  Dias FA, Dias VJ, Dutra BA, Marques AC, Marçal E, Lucena AR, et al. Mobile app for pre- and intraoperative assessment of astigmatism correction with toric intraocular lenses in cataract surgery. JOphthal. 2020;2020:1-7.
• ⁶
  Pereira RV, Kubrusly M, Nogueira IC, Gondim VJ, Marçal E. Development of an application to support in‐service training of anesthesiologists on preoperative evaluation in a public hospital in Brazil. J EvalClinPract. 2019;25(5):850-5.
• ⁷
  Virgili G, Parravano M, Gatta G, Capocaccia R, Mazzini C, Mallone S, et al. Incidence and Survival of Patients With Conjunctival Melanoma in Europe. JAMA Ophthalmol. 2020;138(6):601-8.
• ⁸
  Steinhubl SR, Muse ED, Topol EJ. The emerging field of mobile health. SciTransl Med. 2015;7(283):283rv3.
• ⁹
  Theruvedhi N, Karthikeyan S, Thangarajan R, Srinivasan K. Android mobile applications in eye care. Oman J Ophthalmol. 2019;12(2):73-7.
• ¹⁰
  Moradian S, Safi S. Application of mobile phones in ophthalmology. J Ophthalmic Vis Res. 2015;10(2):200-5.
• ¹¹
  Lewis JR, Sauro J. The factor structure of the System Usability Scale. In: Human centered design. Springer Berlin Heidelberg; 2009. p. 94-103.
• ¹²
  Rodin A, Shachak A, Miller A, Akopyan V, Semenova N. Mobile apps for eye care in Canada: an analysis of the iTunes Store. JMIR MhealthUhealth. 2017;5(6):e84.
• ¹³
  Mohammadpour M, Heidari Z, Mirghorbani M, Hashemi H. Smartphones, tele-ophthalmology, and VISION 2020. Int J Ophthalmol. 2017;10(12):1909-18.
• ¹⁴
  Nhavoto JA, Grönlund Å. Mobile Technologies and Geographic Information Systems to Improve Health Care Systems: A Literature Review. JMIR mHealthuHealth. 2014;2(2):e21.

Publication Dates

• Publication in this collection
  16 Mar 2022
• Date of issue
  2022

History

• Received
  16 May 2021
• Accepted
  28 Oct 2021