PeriText: Utilizing Peripheral Vision for Reading Text on Augmented Reality Smart Glasses



Intro clip:


This work is the first to explore how peripheral vision, instead of central vision, can be used to read text on AR and smart glasses. We propose PeriText, a multiword reading interface using rapid serial visual presentation (RSVP). It allows users to observe the real world using central vision, while using peripheral vision to read virtual information.

The AR overview of PeriText. PeriText is a multi-word reading interface for peripheral vision on augmented reality smart glasses.  While (left) users focus on tasks in the real world such as walking, (top-right) PeriText provides real-time text information using rapid serial visual presentation (rsvp), with words sequentially displayed below users' center gaze, represented by the red crosshair.

Background

Human vision can be categorized into central, para-central, and peripheral vision. The fovea provides central vision, which is the very center of gaze with an eccentricity of 2.5 (5 of the field of view), and has the highest visual acuity. Para-central vision has an eccentricity of 4, and the rest is peripheral vision, which can be up to more than 90. Peripheral vision is weaker at distinguishing detail, color, and shape, because the corresponding density of receptor and ganglion cells is lower and the representation in the visual cortex of the brain is much smaller. Therefore, unlike central vision, peripheral vision might respond differently to various text design.

Field of View of the Human Eye. The central vision is the very center of our gaze with an eccentricity of 2.5°, and para-central vision has an eccentricity of 4°. The rest is peripheral vision, which is weaker at distinguishing details and colors.

Study and Results

To better understand the effect of different text transformation in our peripheral vision area, we conduct our first study to measure the reading efficiency under 34 different conditions, combinations of 6 text transformations, 2 retinal eccentricities, and 3 numbers of words displayed at a time for RSVP.

Text transformations and positions in Study 1. We investigated the performance of reading accuracy using peripheral vision with a total of 6 different text transformations: full capitalization (FC),title case (TC), serif font (SF), character-wise typewriter effect (CT), word-wise typewriter effect (WT), and control text without transformation (C). The 2 text positions are 5 and 8 retinal eccentricities. In addition, the number of words displayed at a time for RSVP paradigmis 1, 2, and 3.

We investigated the performance of reading accuracy using peripheral vision with a total of 6 different text transformations: full capitalization (FC), title case (TC), serif font (SF), character-wise typewriter effect (CT), word-wise typewriter effect (WT), and control text without transformation (C). The 2 text positions are 5 and 8 retinal eccentricities. In addition, the number of words displayed at a time for RSVP paradigm is 1, 2, and 3.

The experiment setup for user study 1 and 2. The participants sat at a distance of 60 cm from the 22-inch LCD monitor, and the text are displayed in the lower visual field. We told the participants to look towards the red crosshair in the middle and used Tobii EyeX eye tracker to ensure the fixation.

In Study 1, the participants were asked to sit at a distance of 60 cm from a 22-inch LCD monitor. To ensure that text was presented at the desired eccentricities, we monitored the observers fixation using Tobii EyeX, a monitor-mounted eyetracking device calibrated beforehand. The goodput evaluated time which the period from showing the sentence to the participant reading it out was recorded, and the answers were typed into a text file immediately for further comparison. The eye-tracker started to record the eye positions for each trial at the beginning of a sentence and stopped recording right after the sentence was completed. The experiment order of the 6 text transformations was counter-balanced through a Latin Square.

The average goodput values (in WPM) under 6 different text transformations, 2 eccentricities, and 3 numbers of words for RSVP paradigm. The goodput value is calculated as the number of words perceived correctly per minute. Green background indicates better performance than baseline, i.e., , larger average goodput value.

The results can be reported respectively in the following design aspects:

According to the goodput results and analysis of study 1, we designed PeriText, our multiword reading interface for peripheral vision, to be 1) title case, 2) sans-serif, and with 3) animation of word-wise typewriter effect.

We then conducted our second study to compare our design and basline text presentation with the similar experimental settings in study 1.

Goodput results of Study 2, comparing the performance of PeriText against control text. The goodput values are calculated as the average number of words perceived correctly per minute. PeriText resulted in 6.7% and 12.4% higher goodput values than control text, for both 5° and 8° eccentricities, and significance found through ANOVA.

The results showed that PeriText achieved significantly higher goodput values than control text, for both 5° (+6.7%) and 8° (+12.4%) eccentricities.

To further understand how PeriText performs in real-world settings, we designed a field study (study 3) where users wore AR glasses and walked around a university campus with 2 different levels of loadings. One is light-loading task, in which users walked on pedestrian-only walkways on the university campus and in the meantime read the text presented by PeriText. The other is heavy-loading walking task, in which users have to cross a street on campus where there would be bicycles and other vehicles. The participants were always accompanied by two experimenters all the time during the study for their safety.

The participants were asked to put on Microsoft Hololens, a head-mounted display with eye-tracker by Pupil Labs to detect the user’s conscious and unconscious glances towards the peripheral text. After user finished the 20 trials for each condition, we collected NASA-TLX index ratings, including the following 6 aspects: mental, physical, and temporal loading, as well as performance, effort, and frustration.

Reading performance when using PeriText in a real-world walking scenario under 4 conditions:  5° and 8° eccentricities x light-loading and heavy-loading tasks. The goodput values, averaged from all participants,  are calculated as the number of words perceived correctly per minute, i.e., in WPM. Significant difference is found between 5°and 8°of eccentricity, for both tasks.

As for overall goodput, the reading performance of reading at 5° eccentricity is better than reading at 8° under both light-loading and heavy-loading scenario.

The average NASA-TLX scores from the field study. For both light-loading and heavy-loading tasks, the overall task load required for 5° eccentricity is lower than that for 8°.  In addition, significantly lower (p<.05) scores found in three individual subscales: 1) mental, 2) physical, and 3) frustration.

For both light-loading and heavy-loading tasks, the overall task load required for 5° eccentricity is lower than that for 8°.

Limitation and Future work

The main limitation of our interface is that during our study, each sentence was displayed once for each trial, and the goodput value was calculated without testing if the participants comprehended the whole sentences. However, several reading studies have shown that readers do not read sequentially, but re-visit text that was previously read, which is called back regressions. In order to evaluate the actual reading speed, it is necessary to introduce such backward mechanisms into PeriText.

Also, an investigation into the relationship between PeriText and various primary tasks is interesting and worthwhile, since what users are doing with their central vision might have influence on the reading performance of using PeriText.

Please refer to our paper, video and talk for detailed information.


Publication / Media