Teleophthalmology: Evaluation of Phone-based Visual Acuity in a Pediatric Population
This prospective study was approved by the Virginia Commonwealth University Institutional Review Board. Informed consent was obtained from parents of the subjects, and assent to study them was obtained from subjects >7 years of age. From April 2019 to February 2020, established patients in the practice of a single-pediatric ophthalmologist (E.S.) between the ages of 3 and 18 were eligible for the study. This population was selected because the goal of this study was to evaluate the visual acuity of children with known and treated refractive errors (if required). Visual acuity was measured in three ways: 1) by the GoCheck Kids mobile application (Scottsdale, Arizona); 2) by HOTV-Amblyopia Treatment Study protocol (HOTV-ATS) on M&S Technologies (Niles, Illinois); and 3) by regular clinic protocol. To mimic visual acuity tested at home, visual acuity was checked by the parent, guardian, or sibling (designated the “tester”) of the patient with GoCheck Kids. A medical student or the pediatric ophthalmologist checked the vision for HOTV-ATS, and an ophthalmic technician checked vision for the regular clinic protocol. The iPhone 7 plus (Apple, Cupertino, California) used for the study was provided to the study by GoCheck Kids. Visual acuity from the GoCheck Kids application was measured with the following protocol. A trained medical student or an ophthalmologist taught the parent or sibling of the patient how to use the application. GoCheck Kids is used to check threshold visual acuity by displaying a test letter in crowding bars (HOTV for children 6 years and younger and ETDRS letters for children 7 years and older) below 4 randomized letters in crowding bars ( ). The child subject indicates which direction the tester should tilt the device until the test letter matches the letter in the randomized row. The child then notifies the tester of a correct placement of the letter, and the tester tilts the phone downward to lock in the answer ( [Supplemental Material available at www.ajo.com ]). The protocol is based on software used in the video game EyeSpy. The test is performed at 5 feet, using a previously measured piece of rope held between the child and the tester to ensure proper distance. The right eye was tested before the left eye, and the nontested eye was occluded. Testers were instructed not to look at the screen to blind them to the test. No feedback was given to the parents during the test to mimic testing environment at home. The GoCheck Kids application can report vision from 20/20 (0.0 logMAR) to 20/63 (0.5 logMAR). HOTV-ATS visual acuity was measured at 16 feet (the length of the examination room from chair to visual acuity system) using the M&S system running the Amblyopia Treatment Study protocol. HOTV with crowding bars were used for all participants. The right eye was tested before the left eye, while the nontested eye was occluded. Visual acuity was also measured at 16 feet with the M&S system, using the clinic's protocol, testing with the most challenging acuity that the child could reliably perform: Sloan > HOTV line > HOTV crowding bar (“chart acuity”). Also recorded was the relationship of the tester to the child, the child's sex and race, the glasses prescription, and the child's ophthalmic diagnoses. The presence of vision disease was defined as a diagnosis that affected the child's visual acuity (eg, a child with juvenile idiopathic arthritis with normal examination results was classified as not having a vision disease, and a child with amblyopia was classified as having a vision disease). Data for this study were analyzed using SAS/STAT version 9.4 software (SAS, Cary, North Carolina) for Windows (Microsoft, Redmond, Washington). Descriptive statistics were computed for demographic variables and for GoCheck Kids acuity and HOTV-ATS acuity and the paired difference between them in LogMAR units. To assess the significance between acuities, generalized estimating equations were used to test whether the mean differences were equal to zero. To assess agreement between measures of acuity, intraclass correlations with 95% confidence intervals (CI) were computed. In addition, the percentage of children with visual acuity as measured by GoCheck Kids that were within 1 and 2 lines of HOTV-ATS and of regular clinic protocol were calculated. Study data were managed using Research Electronic Data Capture (REDCap) hosted by Virginia Commonwealth University. , REDCap is a secure, web-based software platform designed to support data capture for research studies, providing 1) an intuitive interface for validated data capture; 2) audit trails for tracking data manipulation and export procedures; 3) automated export procedures for seamless data downloads to common statistical packages; and 4) procedures for data integration and interoperability with external sources. 

Fifty-three children were enrolled in the study. The application was prematurely stopped when the vision measurements in 2 children were recorded as much worse than their true vision (96% testability). Observation from the study personnel suggested that the tester was tilting the phone downward while tilting the phone to the side, thus prematurely locking in an answer. These children were not used in the statistical evaluation. Demographics presented in . There were 15 children <7 years of age. Four eyes could not be measured by the GoCheck Kids application because they could not identify the largest (20/63; −0.5 logMAR) optotype on the device. At least 2 testers physically dropped the phone during testing. The mean logMAR (imperial) acuity assessed by GoCheck Kids was 0.106 (20/25.5); 0.012 (20/20.5) by HOTV-ATS; and 0.096 (20/22) by chart. The mean differences and CI intervals between GoCheck Kids and HOTV-ATS acuities (acuity differences: 0.094; 95% CI: 0.074-0.114) were significantly different ( P < .001). The mean differences and 95% CI between GoCheck Kids and chart acuities (acuity difference: 0.010; 95% CI: −0.010 to 0.030) were not significantly different ( P = .319). The mean difference and CIs between HOTV-ATS and chart acuities (0.084: 95% CI: 0.014-0.063) were significantly different ( P < .001). The intraclass correlation between GoCheck Kids and HOTV-ATS acuities was 0.55 (95% CI: 0.40-0.68) and 0.59 (95% CI: 0.45-0.71) between HOTV-ATS and chart acuities, indicating fair agreement between each of the 2 sets of measurements of acuity. The intraclass correlation between GoCheck Kids and chart acuities was 0.66 (95% CI: 0.53-0.76), indicating a modest agreement between the measures. Bland-Altman plots are presented in . The percentage of eyes with visual acuity as measured by GoCheck Kids that was within 1 line of the HOTV-ATS, and chart acuity was 65.3% and 86.7%, respectively. The percentage of eyes that was within visual acuity as measured by GoCheck Kids was within 2 lines of the HOTV-ATS, and both chart acuities were 96.9%. There were 14 children with a history of unilateral amblyopia. The differences between the acuity between the amblyopic eye and the nonamblyopic eye as measured by each method are in . 

Telemedicine has arrived. COVID-19 has pushed us to adapt so we can continue to safely serve our patients. Before COVID-19, telemedicine was rapidly developing in the field of ophthalmology, mostly for diseases like diabetic retinopathy and glaucoma, by obtaining images and sending them to an ophthalmologist for review , , , , or for interpretation by artificial intelligence and in retinopathy of prematurity using deep neural networks. In the COVID-19 era of teleophthalmology, we do not have the benefit of these sophisticated methods of image acquisition and interpretation. We also need a way of accurately measuring visual acuity, especially in children, in order help diagnose and treat amblyopia. The ideal method of checking visual acuity at home would be easy to use, accurate, precise, inexpensive (or free), ensure monocular testing and be available to test every single patient and family. ABCD-Vision (Anchorage, Alaska) developed a paper-based HOTV crowding bar visual acuity test to be used for remote use in Alaska. This 8.5- × 11-inch paper acuity chart can be emailed for at-home printing or mailed to patients who do not have access to a printer. Another method of checking vision at home is using a smartphone or computer. Some of these programs are based on peer-reviewed studies (Peek Acuity; Peek Vision, London, United Kingdom); EyeHandbook (Cloud Nine Development, Overland Park, Kansas); and Jaeb Visual Acuity Screener (Jaeb Center for Health Research, Tampa, Florida) , , , , , , , ; and some are not (Farsight; Farsight.care; London, United Kingdom); Eye Chart Pro (Dok LLC; Apple, Cupertino, California); Visual Acuity Charts (Fonlo); and Snellen Acuity (João Meneses [available as an application]) ( ). There are several issues with using electronic applications to check visual acuity. The first issue is access. Some applications are only available on Android (Google, Mountain View, California), and others are exclusive to iOS (Apple). Other applications are available only on Windows-based computers (Microsoft, Redmond, Washington). Although smartphones are considered ubiquitous, consideration should also be taken into account for those without access to this technology due to their financial situations. This limits the ability to provide uniform directions to patients and have consistent measurements across all patients. However, if a patient uses the same method for checking visual acuity over 2 visits, the acuities are theoretically comparable. Second, we have to trust that the developer has modified the size of the optotypes for different screen sizes. Some applications provide configuration screens to ensure the correct size of the optotypes. Third, the choice of optotypes can be important. Tumbling “E” optotypes (used by Peek Acuity) can be difficult for very young children who have not developed the ability to express the orientation of the optotypes, although more recent studies report excellent testability. Fourth, some orientation may be required for a family to check vision at home, for example, via in app directions, a video guide, or testing while on a teleconference with an ophthalmic technician. This unique study explores the use of GoCheck Kids to check visual acuity by the patient's relative with minimal training. There have been many studies evaluating the assessment of visual acuity by lay screeners in vision screening programs. , , , , , , , The goal of those studies is to refer children with decreased visual acuity below an age-based critical line for further examination by an eye care provider. Now, the narrative is shifting to explore testing visual acuity as measured by those methods as a visual acuity that an ophthalmologist will use for their evaluation and decision making process. This study shows that visual acuity using GoCheck Kids at 5 feet has fair agreement with HOTV-ATS. On average, the visual acuity as measured by GoCheck Kids is 1 line worse than HOTV-ATS and very close to the chart acuity. The visual acuity measured by HOTV-ATS averages one line better than the chart acuity. Therefore, it is unclear if both GoCheck Kids and chart acuity underestimate vision, or the HOTV-ATS overestimates vision. The average age of children in our study was 10 years old; these children may have understood that there were only 4 choices in the HOTV-ATS and guessed on the letters based on similar recognition rather than true identification, thus measuring vision as better than the other methods. In addition, prior studies have shown that HOTV-ATS overestimates vision compared to E-ETDRS by 0.68 lines for amblyopic eyes and 0.25 lines for fellow eyes. This is the largest weakness of this study. A similar study was performed for the Peek Acuity application. This study tested 111 children, 3-17 years of age and compared visual acuity from Peek Acuity with that from clinic protocol visual acuity and showed a good ICC for the first (0.88) and second (0.84) eyes tested; although the ICC dropped for second eyes (0.45) in children 3-5 years old. The authors did not test for that variable. When assessing and treating amblyopia, one might argue that the exact visual acuity does not matter as much as the difference of visual acuity between the 2 eyes. In children with amblyopia, there was no consistency in the difference between the two eyes among all the methods of checking visual acuity. A total of 1 in 14 children had the same difference of acuity among all methods. A strength of this study is the method that visual acuity was checked using the GoCheck Kids application. We purposefully had the family members of the child perform the measurement to simulate checking vision at home. The study personnel also limited communication and instruction to the tester once the test had started to further mimic home testing environment. Children were able to learn the visual acuity game quickly. Parents had mild difficulty learning the motions needed to manipulate the device, as indicated by the dropped phones and the premature stopping of the test. Further studies could be performed by having family members check vision at home with the application prior to presenting to the clinic for their examination. We recommend further studies use HOTV-ATS for children <7 years of age and E-ETDRS for children >7 years of age. A weakness of this study was that the children were not patched, using occlusion with patches over glasses, occlusion glasses (that have occluders that flip down over the nontested eye), or a handheld occluder. Children are more consistent with visual acuity when an occlusive patch was applied directly to the skin. In addition, as most children who would be treated for amblyopia may be <7 years of age, this study was limited the our small sample size in this age group. In its current form, GoCheck Kids is limited by the size of an iPhone Plus screen size. The lowest acuity that can be measured at 5 feet is 20/63 (0.5 logMAR). Some testing methods allow children to move closer to the chart and changing the numerator of the visual acuity, but this is not possible with the already short testing distance of 5 feet for GoCheck Kids. This significantly limits the use of this application when testing children with amblyopia, as many of them will have an acuity of <0.5 logMAR. When testing vision at home, it may be important to have a backup test if clinical suspicion arises that the acuity is not consistent with prior tests of vision or if the visual acuity is too low to be tested by the application. It is unknown how GoCheck Kids will perform for measuring visual acuity in children with uncorrected refractive errors. It may over-estimate distance acuity in children with myopia due to the 5-foot testing distance. A program that still allows for matching when needed and the ability to test at 10-feet would be beneficial for a pediatric testing device; a second device that is paired may be helpful for the child to hold when matching the letter at distance (though this would require the family to have two devices). In June 2020, GoCheck released an application with an updated user interface dedicated to checking visual acuity. It is available outside of the GoCheck Kids vision screening ecosystem. In conclusion, GoCheck Kids has a high rate of testability and provides a modest correlation of visual acuity compared to the chart screen and a fair correlation of visual acuity compared to HOTV-Amblyopia Treatment Study protocol, although most are within 1 line. More studies are needed to evaluate this method of checking visual acuity at home for teleophthalmology. 

Evan Silverstein: Conceptualization, Data curation, Formal analysis, Methodology, Project administration, Writing - original draft. Jonathan S. Williams: Methodology, Data curation, Writing - review & editing. Jeffrey R. Brown: Investigation, Writing - review & editing. Enjana Bylykbashi: Investigation, Writing - review & editing. Sandra S. Stinnett: Formal analysis, Writing - original draft. 

