Open-access Clinical validation of a cross-platform digital visual acuity measurement system

Validação clínica de um sistema digital de medição de acuidade visual multiplataforma

ABSTRACT

Objective:  To validate visual acuity measurements using a cross-platform system in an ophthalmological setting.

Methods:  Visual acuity was assessed at a distance of 5m using two different modalities: printed and digital optotypes. The devices included Android, LG WebOS and Samsung Tizen smart TVs. Optotypes were presented in the logMAR scale. Two modalities, a single row and a block of symbols, presented Sloan letters and Tumbling E symbols.

Results:  Visual acuity measurements of 190 participants aged 12 to 60 years demonstrated good-to-strong test-retest correlation (Intraclass Correlation Coefficient >0.75) and minimal bias (-0.03 to 0.02 logMAR). Limits of agreement were comparable to other studies (0.13 to 0.26 logMAR), with the smallest values for Tumbling E row presentation in all devices and the highest value for Sloan chart in LG device. Analysis of variance revealed no statistical differences in visual acuity across devices. Sloan letters showed superior visual acuity compared to Tumbling E (p<0.001); however, this difference corresponded to only 2 letters of visual acuity.

Conclusion:  The digital cross-platform evaluated serves as a versatile substitute for traditional visual acuity assessments in individuals aged 12 years and older. Further research is necessary for patients with visual acuity worse than 0.5 logMAR and to conduct vision screenings in young children.

Keywords:
Visual acuity; Refraction, ocular; Vision tests; Vision screening; Psychophysics

RESUMO

Objetivo:  Validar as medições de acuidade visual utilizando um sistema multiplataforma em um contexto clínico oftalmológico.

Métodos:  A acuidade visual foi avaliada a uma distância de 5m usando duas modalidades diferentes: optotipos impressos e digitais. Os dispositivos incluíram smart TVs Android, LG WebOS e Samsung Tizen. Os optotipos foram apresentados na escala logMAR. Duas modalidades, sendo uma fila única e um bloco de símbolos, apresentaram letras Sloan e os símbolos Tumbling E.

Resultados:  Medidas de acuidade visual de 190 participantes de 12 a 60 anos demonstraram boa a forte correlação teste-reteste (Coeficiente de Correlação Intraclasse >0,75) e viés mínimo (-0,03 a 0,02 logMAR). Os limites de concordância foram comparáveis aos de outros estudos (0,13 a 0,26 logMAR), com os menores valores para a apresentação em linha única do Tumbling E em todos os dispositivos e o maior valor para o gráfico Sloan no dispositivo LG. A análise de variância revelou não haver diferenças estatísticas na acuidade visual entre os dispositivos. As letras Sloan mostraram acuidade visual superior em comparação com Tumbling E (p<0,001), no entanto, essa diferença correspondeu a apenas duas letras de acuidade visual.

Conclusão:  O teste digital multiplataforma serve como um substituto versátil para avaliações tradicionais de acuidade visual em indivíduos a partir de 12 anos de idade. Pesquisas adicionais são necessárias para pacientes com acuidade visual inferior a 0,5 logMAR e a realização de triagens visuais em crianças pequenas.

Descritores:
Acuidade visual; Refração ocular; Testes visuais; Seleção visual; Psicofísica

INTRODUCTION

Digital or computerized eye charts are increasingly common in vision tests.(1-3) However, the reproducibility of digital visual acuity (VA) applications may be questioned due to the limited number of clinical validation studies,(4-9) which raises concerns about the consistency of digital optotypes tests across different versions, operating systems and display specifications.(10)

Therefore, validating a cross-platform VA assessment software can provide easy access to evidence-based ophthalmic charts for professionals in different locations, simply by using compatible devices to perform VA, and this is the main purpose of our study.

To validate tests across multiple digital platforms, our study aimed to validate VA measurements using a cross-platform system in an ophthalmological setting. So, we compared traditional optotypes (Tumbling E and Sloan letters) with standard printed charts using the cross-platform VA system EyeCharts (JCL Medical, Brazil). This newly trademarked software enables professional offline VA assessments on compatible smart TV devices and across various operating systems.

METHODS

The tests were performed during an ophthalmological consultation at a reference service. Collected data were compared to printed VA tests. The study population consisted of patients from a reference Ophthalmology Service in João Pessoa (PB), Brazil, during the year 2023, as per the details below.

Participants

The project adhered to the core ethical principles of the Declaration of Helsinki and was approved by the following Faculdade de Ciências Médicas’ Ethics Committee in Paraíba. All participants provided informed consent prior to screening for eligibility.

Patients evaluated for an ophthalmological consultation with a refractive objective were selected in December 2023. Patients with a history of any ocular disease, under 12 years old, VA worse than +0.5 logMAR (20/63) in both eyes and record of any cognitive deficit or need for interpretation were excluded. Demographic data including age and gender were collected.

Research design

All individuals underwent non-cycloplegic subjective refraction before the acuity test. Standard clinical practices were employed to assess subjective refraction using a standard ETDRS chart.(2,11,12) If a subject wore contact lenses or if their habitual refraction was outdated, an appropriate spectacle correction was provided using a phoropter. The right eye was examined in each case. Rationally, we chose to use the most widely studied digital verbal and non-verbal tests in medical literature: the Sloan letters and Tumbling E optotypes.(12-14) Patients were initially randomized to different modes of optotype presentation for the vision tests, which included Smart TVs and standard printed charts. This initial randomization aimed to impartially assign participants to a digital format or a traditional paper-based assessment, ensuring that any potential biases related to the mode of presentation were minimized. Following this process, participants underwent a second randomization phase, where they were allocated to one of the two different types of VA tests: Sloan letters or Tumbling E. This two-step randomization procedure was meticulously designed to distribute participants evenly across all test conditions, thereby allowing for a comprehensive comparison of VA outcomes across both optotype presentations and test types.

Tests

The VA was evaluated at a distance of 5m using a digital cross-platform software (EyeCharts V 1.7.0/2023). Two distinct modalities, a single row and a block of symbols (LogMAR chart), were used to show randomly chosen Sloan letters (K, D, H, C, N, O, S, R, Z, and V) and Tumbling E optotypes. In the single-row modality, five random optotypes were presented spaced by the same size of each symbol. The charts (blocks of symbols) in the digital system are made up of rows of five letters, as proposed by Bailey and Lovie2,3, and the maximum number of LogMAR scale rows of optotypes, identical to a logMAR chart, that fit on the screen. The rows from 0.7 to 1.0 logMAR were presented in the first screen and from −0.1 to 0.6 logMAR were shown in a second screen.

Participants were randomly distributed for each modality of optotypes, presentation methods and devices. After 1 week, a repeatability test session was conducted utilizing the same order.

Devices and operating systems were chosen based on native EyeCharts software compatibilities. Digital EyeCharts were randomly displayed on three different devices: 32-inch webOS LG Smart TV 1080p image resolution (LG Electronics, South Korea), 32-inch Samsung Tizen Smart TV 1080p image resolution (Samsung Electronics Co. Ltd, South Korea), and 32-inch Android TV 1080p image resolution (Google Inc., United States). Optotypes were presented in black on a white background with luminance values of 3 and 120 cd m−2, respectively. The contrast was set to a maximum of 500:1. The stimulus display was viewed monocularly.

The LogMAR scale paper chart was printed for a 5m test distance. The optotypes on the LogMAR paper chart had a uniform illuminance of approximately 500lx, as testing was conducted at 5m in a well-lit clinical testing room.

Procedure

To avoid codependence, the VA was only measured in the right eye. Two ophthalmologists were responsible for measuring the visual acuities. The retest was carried out by the same operator as in the first evaluation, and all data were registered in a digital form.

The letter-by-letter method was used to score VA. The participant had to properly interpret at least three out of the five targets for the value associated with that row to be recorded. The number of symbols recognized and orally stated, that is, 3/5, 4/5, or 5/5, were then recorded. All successfully read letters are considered in the VA's final value. VA is quantified as a function of the total number of recognized letters. In this method, LogMAR is computed by adding the value corresponding to the row at which the observer identified at least one character (0.02) for each uncorrected response. For example, if the subject recognizes four letters out of five in row 0.1 LogMAR, the test will score 0.1+0.02=0.12.(1,2,11) Illiterate individuals allocated to the Sloan letter test group were excluded from the study.

For Tumbling E test, participants were asked to indicate the orientation of the branches of the letter E (top, bottom, right, left).

Statistical analysis

The agreement between EyeCharts and standard LogMAR printed chart VA measurements was assessed using Bland-Altman plots, focusing on mean bias and 95% limits of agreement (LoA). Test-retest (TRT) agreement was performed using a two-way mixed-effect model and Intraclass Correlation Coefficient (ICC). One-way analysis of variance (Anova) was performed to compare all VA values.

Data was analyzed using Bland Altman software for Microsoft Excel (Version 6.15.4 for Windows 10) and IBM Statistical Package for the Social Sciences (SPSS) for Windows, version 20 (IBM Corp., Armonk, N.Y., United States).

Development of optotypes

Tumbling E and Sloan letters were developed based on their individual properties i.e., stroke to bounding box ratio of 1:5. The size of the standard optotypes was measured using a formula that uses the following variables: VA, distance from the optotype to the observer,(12) and a calibrator that varies depending on the size of the screen/display:

H = 3,779. D.VA.C.tg(5′)

"H" represents the optotype grid size in pixels; "D" is distance to screen in meters; VA is the visual acuity; 3.779 is used to convert mm to pixels; "C" is the function calibrator, which depends on the screen size. The function calibrator serves to compensate for different screen sizes and keeps the optotype sizes the same, regardless of the device used.

RESULTS

A total of 216 participants were selected for testing. Out of this sample, 190 completed both the test and retest phases. The age of participants (mean ± SD) was 33.7±15.8 years, ranging from 12 to 60 years. Of these, 81 were male and 109 were female. The mean spherical refraction was −0.22 D, ranging from −9.25 D to +4.0 D. The mean cylindrical refraction was −1.20 D, ranging from −4.75 D to 0.00 D.

The repeatability results for all presentation modalities are shown in table 1. The analysis compared the VA scores in different sessions using the ICC. The results showed that Sloan and Tumbling E tests on digital screens, in both single-line and block modalities, showed good (ICC >0,75) and excellent reliability (ICC >0,90). The tests and retests carried on printed charts also demonstrated a strong relationship (ICC >0,75).

Table 1
Mean visual acuity measurements obtained through different testing devices, procedures and sessions

The study compared the effectiveness of various VA measurement methods in digital and paper presentations. Differences between them were plotted against the mean clinical VA measurements from EyeCharts digital screening and standard test screening (printed optotypes) in Bland and Altman plots (Figures 1 to 3). Limits of agreement ranged from 0.1 to 0.26 logMAR for EyeCharts versus printed chart measurements. Bias ranged from −0.03 to 0.02. Anova was also conducted, and significance levels (p) are presented in table 2.

Figure 1
Bland-Altman plots comparing measurements between an Android Smart TV digital system and printed logMAR chart, using various optotypes and presentation modalities. Dashed lines indicate bias and 95% limits of agreement, with shaded areas denoting 95% confidence intervals. Results are presented on a logMAR scale, with the x-axis representing the average of measurements from the first session.
Figure 2
Bland-Altman plots comparing measurements between LG WebOS TV and a printed logMAR chart, with different optotypes and presentation modalities. Dashed lines denote bias and 95% limits of agreement, and shaded areas indicate 95% confidence intervals. Measurements are represented on a logMAR scale.
Figure 3
Bland-Altman plots comparing Samsung Tizen TV and printed logMAR chart measurements, using various optotypes and presentation modes. Dashed lines indicate bias and 95% limits of agreement, with shaded areas denoting 95% confidence intervals. Results are presented on a logMAR scale.
Table 2
Bias and limits of agreement between digital optotypes and their correspondents in a standard test (printed logMAR chart)

Figure 4 shows the interaction between devices and VA for all presentation modalities. One-way Anova was performed to evaluate all ways of presenting the optotypes (Android, Samsung Tizen, LG WebOS and printed chart). We found no significant difference for VA means when evaluating Sloan (F(5,288) = 0,378; p = 0,863) and Tumbling E (F(6,366) = 0,294; p = 0,940) separately, indicating consistent performance across different devices. We also performed a one-way Anova test to evaluate the VA averages between the different types of presentations (single-row and logMAR chart). The results demonstrated equivalence between the presentation methods for Sloan (F(1,684) = 1,048; p = 0,306) and Tumbling E (F(1,684) = 0,372; p = 0,542) separately.

Figure 4
Mean results for the interaction between devices vs. presentation methods for Sloan letters and Tumbling E. The different points represent the mean visual acuity (logMAR).

We further explored the disparities between test types. An Anova revealed a statistically significant difference in the VA measurements (F(2,1959) = 9.768; p<0.001). Although there was a statistical difference, the most substantial discrepancy observed between these tests was 0.042 logMAR, equivalent to roughly two letters.

DISCUSSION

Previous research has found that, in large eye clinics, in order to be confident that a real change in VA has occurred between measurements, a difference of at least 0.15 logMAR (eight letters on a standard logMAR VA chart) is required.(15) In our study, the bias was less than 0.15 logMAR when comparing digital optotypes with their printed counterparts (-0,03 to 0,02 logMAR). However, the comparison between different devices (digital and printed charts) must consider LoA data from similar validation studies for a better understanding of the agreement between results. Compared to previous studies, which reported LoA of approximately 0.12 to 0.208 logMAR,(6,16,17) the LoA found In this research were similar (0.13 to 0.26) with the smallest values for Tumbling E row presentation in all devices and the highest value for Sloan chart in LG device.

The small LoA and bias reported in this research indicate a stable correlation between digital and paper charts and suggest good agreement and consistency compared to other digital VA systems.(7,16-18) Furthermore, the high correlation across TRT measurements indicates high consistency and reliability of the digital optotypes.

In the present study the effect of crowding effect(19,20) was not present in any of the predicted directions. VA was similar for all single-row and block presentations as results from Anova demonstrated no statistical difference. The authors suggest that the crowding effect would be better evaluated with the presentation modality in separate letters.

The cross-platform validation also demonstrated the reproducibility of tests between different devices, when each test was evaluated separately. However, we noticed better VA results with letters than with Tumbling E. Despite the statistical discrepancy, the largest difference between these tests was only 0.042 logMAR (or approximately two letters).

The literature presents conflicting findings on differences in VA measurements among various optotypes. Superior VA demonstrated with the Sloan letter test, observed in a study involving digital and printed optotypes,(21) was replicated in the current study. Notably, VA values with letters surpassed those with symbols (Landolt Rings, Tumbling E, and LEA symbols). This discrepancy may stem from participants’ literacy, suggesting familiarity with letters. Conversely, research focusing on a cohort of young adults and patients with diabetic retinopathy identified no significant difference in VA measurements between the Sloan letters and the Tumbling E optotypes.(22) Such variability in the accuracy or consistency of VA assessments among different optotypes may be attributed to various factors, including the testing environment (clinical versus research settings) and the demographic and clinical characteristics of the participants (such as their age and level of visual impairment).(23)

The evaluation of only verbal patients was a limitation of our study, since some optotypes like Tumbling E are designed to overcome the difficulty of evaluating nonverbal patients. These points could be addressed in future studies including young children and nonverbal participants to provide specific clinical information for these populations.

Another issue we raise is the difference in resolution between the VA screens and the printed charts. Although all devices used in this research have a 68.88 ppi resolution and a printed paper has an average resolution of 300 ppi,(24-26) no difference was observed in VA results between the digital screens and the printed chart. But considering the large difference in resolution, and especially in case an observer is placed at small distances from the screen, we also recommend new study approaches based on different characteristics of screen resolution and viewing distances.

The study authors believe that utilizing high-definition devices for digital optotypes offers clear advantages, including detailed fonts that reduce the effort needed for letter deciphering and image differentiation. Higher resolutions enhance clarity and detail, particularly for close viewing. Their observations align with studies on display resolution's role in visual information processing,(27,28) indicating that reading from digital screens is slower than from printed paper, which has seven times higher resolution. Additionally, higher resolution devices led to less accommodation lag and reduced visual fatigue compared to lower resolution ones.

While the Sloan letters and Tumbling E optotypes have been extensively used and validated across various studies using digital optotypes,(13,21,29,30) the necessity to explore a broader array of optotypes is clear. Alternative digital optotypes, such as Landolt C, Snellen E, and LEA symbols, could provide valuable insights into the nuances of VA measurement across diverse populations, including those with specific communication or developmental limitations. Acknowledging this gap, we aim to extend our research to incorporate these and other optotypes using a multi-platform methodological approach in future studies. This methodological expansion will not only enhance our understanding of potential differences in accuracy and reliability of VA measurements among various optotypes but will also facilitate the selection of the most effective optotype for specific clinical applications, thereby maximizing patient benefits across different clinical and research settings.

CONCLUSION

Based on the results of this clinical validation study, the EyeCharts digital visual acuity test has proven to be an effective cross-platform alternative to traditional clinical visual acuity assessments for individuals aged 12 and older. This tool offers professionals easy access to evidence-based ophthalmic charts across different locations. However, further research is necessary for patients with visual acuity worse than 0.5 logMAR and for conducting vision screenings in young children.

  • Institution:
    Department of Vision Sciences, Centro Oftálmico Tarcízio Dias, João Pessoa, PB, Brazil.
  • Financial support:
    no financial support for this work.

REFERENCES

  • 1 Aleci C, Rosa C. Psychophysics in the ophthalmological practice. Ann Eye Sci. 2022;7(1):37.
  • 2 Bailey IL. Visual acuity. In: Benjamin WJ, ed. Borish's clinical refraction. Elsevier Health Sciences; 2006.
  • 3 Bailey IL, Lovie-Kitchin JE. Visual acuity testing. From the laboratory to the clinic. Vision Res. 2013;90:2-9.
  • 4 Harris PA, Roberts LE, Grant R. Comparison of backlit and novel automated ETDRS visual acuity charts. Optom Vis Perform. 2018;6(2):87-96.
  • 5 Claessens JL, Geuvers JR, Imhof SM, Wisse RP. Digital tools for the self-assessment of visual acuity: a systematic review. ophthalmol Ther. 2021;10(4):715-30. Erratum in: Ophthalmol Ther. 2021;10(4):731-2.
  • 6 Thirunavukarasu AJ, Mullinger D, Rufus-Toye RM, Farrell S, Allen LE. Clinical validation of a novel web-application for remote assessment of distance visual acuity. Eye (Lond). 2022;36(10):2057-61.
  • 7 Labiris G, Panagiotopoulou EK, Delibasis K, Duzha E, Bakirtzis M, Panagis C, et al. Validation of a web-based distance visual acuity test. J Cataract Refract Surg. 2023;49(7):666-71.
  • 8 Wisse RP, Muijzer MB, Cassano F, Godefrooij DA, Prevoo YF, Soeters N. Validation of an independent web-based tool for measuring visual acuity and refractive error (the manifest versus online refractive evaluation trial): prospective open-label noninferiority clinical trial. J Med Internet Res. 2019;21(11):e14808.
  • 9 Brady CJ, Eghrari AO, Labrique AB. Smartphone-based visual acuity measurement for screening and clinical assessment. JAMA. 2015;314(24):2682-3.
  • 10 Carkeet A, Johnson A, Hopkins S. Different pictogram vision charts give different slopes for psychometric functions, in preschool children. Invest Ophthalmol Vis Sci. 2022;63(7):2555-F0509.
  • 11 Facchin A, Maffioletti S, Martelli M, Daini R. Different trajectories in the development of visual acuity with different levels of crowding: the Milan eye chart (MEC). Vision Res. 2019;156:10-6.
  • 12 de Jong PT. A history of visual acuity testing and optotypes. Eye (Lond). 2024;38(1):13-24. Erratum in: Eye (Lond). 2024;38(1):226.
  • 13 Yulianti NF, Munawir A, Adji NK. Validity of electronic device-based application for visual acuity examination: A systematic review. Indones J Electron Electromed Eng Med Inform. 2022;4(1):41-7.
  • 14 Good WV. Vision assessment of nonverbal patients. Am Orthopt J. 2007;57(1):13-8.
  • 15 Siderov J, Tiu AL. Variability of measurements of visual acuity in a large eye clinic. Acta Ophthalmol Scand. 1999;77(6):673-6.
  • 16 Pang Y, Sparschu L, Nylin E. Validation of an automated-ETDRS near and intermediate visual acuity measurement. Clin Exp Optom. 2020;103(5):663-7.
  • 17 Dawkins A, Bjerre A. Do the near computerised and non-computerised crowded Kay picture tests produce the same measure of visual acuity? Br Ir Orthopt J. 2016;13:22-8.
  • 18 Rosser DA, Cousens SN, Murdoch IE, Fitzke FW, Laidlaw DAH. How sensitive to clinical change are ETDRS logMAR visual acuity measurements? Invest Ophthalmol Vis Sci. 2003;44:3278-81.
  • 19 Anstice NS, Jacobs RJ, Simkin SK, Thomson M, Thompson B, Collins AV. Do picture-based charts overestimate visual acuity? Comparison of Kay pictures, Lea symbols, HOTV and Keeler log MAR charts with Sloan letters in adults and children. PLoS One. 2017;12:e0170839.
  • 20 Lalor SJ, Formankiewicz MA, Waugh SJ. Crowding and visual acuity measured in adults using pediatric test letters, pictures and symbols. Vision Res. 2016;121:31-8.
  • 21 Campo Dall’Orto G, Facchin A, Bellatorre A, Maffioletti S, Serio M. Measurement of visual acuity with a digital eye chart: optotypes, presentation modalities and repeatability. J Optom. 2021;14(2):133-41.
  • 22 Plainis S, Kontadakis G, Feloni E, Giannakopoulou T, Tsilimbaris MK, Pallikaris IG, et al. Comparison of visual acuity charts in young adults and patients with diabetic retinopathy. Optom Vis Sci. 2013;90(2):174-8.
  • 23 Dobson V, Clifford-Donaldson CE, Miller JM, Garvey KA, Harvey EM. A comparison of Lea Symbol vs ETDRS letter distance visual acuity in a population of young children with a high prevalence of astigmatism. J AAPOS. 2009;13(3):253-7.
  • 24 Marran L, Liu L, Lau G. Desktop publishing and validation of custom near visual acuity charts. Optom Vis Sci. 2008;85(11):1082-90.
  • 25 Larimer JO, Gille J, Powers MK, Liu HC. Hyperacuity on high-resolution and very high resolution displays. In: Human Vision and Electronic Imaging IX. Vol. 5292. SPIE; 2004. p. 211-7.
  • 26 Boggess B, Wiechel J, Morr D, Anderson J, Pipo J. Anatomical limitations of the visual field of view: An example of driving perspective. SAE Technical Paper No. 2008-01-1876; 2008. [cited 2024 July 20]. Available from: https://www.sae.org/publications/technical-papers/content/2008-01-1876/
    » https://www.sae.org/publications/technical-papers/content/2008-01-1876/
  • 27 Ziefle M. Effects of display resolution on visual performance. Hum Factors. 1998;40(4):554-68.
  • 28 Hynes NJ, Cufflin MP, Hampson KM, Mallen EA. The effect of image resolution of display types on accommodative microfluctuations. Ophthalmic Physiol Opt. 2022;42(3):514-25.
  • 29 Georgeson MA, Barhoom H, Joshi MR, Artes PH, Schmidtmann G. Revealing the influence of bias in a letter acuity identification task: A noisy template model. Vision Res. 2023;208:108233.
  • 30 Vafaie MH, Ahmadi Beni E. Design and Implementation of a Smart Wireless Controlled Visual Acuity Measurement System. J Med Signals Sens. 2023;13(4):307-18.

Publication Dates

  • Publication in this collection
    07 Oct 2024
  • Date of issue
    2024

History

  • Received
    08 Apr 2024
  • Accepted
    20 July 2024
location_on
Sociedade Brasileira de Oftalmologia Rua São Salvador, 107 , 22231-170 Rio de Janeiro - RJ - Brasil, Tel.: (55 21) 3235-9220, Fax: (55 21) 2205-2240 - Rio de Janeiro - RJ - Brazil
E-mail: rbo@sboportal.org.br
rss_feed Stay informed of issues for this journal through your RSS reader
Accessibility / Report Error