PersonalTouch: Improving Touchscreen Usability by Personalizing Accessibility Settings based on Individual User’s Touchscreen Interaction

A user perform six different kinds of gestures and PersonalTouch analyzes user's input to recommend personalized, optimal accessibility settings including 2.00 seconds for 'Hold Duration' and 2.00 seconds for 'Tap Assistance with Initial Touch Location'.

Modern touchscreen devices have recently introduced customizable touchscreen settings to improve accessibility for users with motor impairments. For example, iOS 10 introduced the following three Touch Accommodation settings: 1) Hold Duration, 2) Ignore Repeat, 3) Tap Assistance Gesture Delay. These three independent settings lead to numerous possible configurations, making it impractical to manually determine the optimal setting. We present PersonalTouch, which collects and analyzes touchscreen gestures performed by individual users, and recommends personalized, optimal touchscreen accessibility settings.

We describe our suggestion pipeline as follow:

Three steps for PersonalTouch to provide recommendations. First, collect user's touchscreen gestures data. Secondly, simulate different combinations of settings on the input touchtracks. And finally, choose the optimal settings via cross-validations and settings responsiveness.

PersonalTouch goes through three steps to provide personalized settings suggestion, which includes a) Touch data collection. b) Gesture recognizer simulation and c) Settings recommendation.

a) Touch data collections:
We cover 6 basic gestures (tap, scroll, swipe, long press, rotate and pinch) from iOS and Android default recognizers with total 108 trials.
b) Gesture recognizers simulation:
We reversed-engineered iOS gesture recognizers using Xcode and Hopper decoder to simulate 979,837 possible accessibility settings by quantizing 3 parameters into 79 levels. For each combination of settings, we compute the success rate recognized by the simulated system recognizer for each trial, resulting in a set of 108 input success rates.
c) Settings recommendation:
We conducted a 5-fold cross-validation ten times to find the optimal setting. For each run, we select the optimal setting by the highest overall success rate (weighted by gesture group) and the highest time responsiveness (lowest total duration).

Results

Bar chart of the overall sucess rates for Default (55.7%) and PersonalTouch (75.9%) recognition models.

Our 12-motor-impaired-participant study shows that PersonalTouch significantly improved overall success rate by 20.2% over the default gesture recognizer.

For further information of our system and study, please refer to the paper.

Publication / Media

Peng, Yi-Hao, Lin, Muh-Tarng, Chen, Yi, Chen, TzuChuan, Ku, Pin Sung, Taele, Paul, Lim, Chin Guan, and Chen, Mike Y.: PersonalTouch: Improving Touchscreen Usability by Personalizing Accessibility Settings based on Individual User’s Touchscreen Interaction, In Proceedings of CHI ‘19, May, 2019.
This work was featured in the Apple Worldwide Developers Conference (WWDC) 2019 Session - ResearchKit and CareKit Reimagined (02:57).

Competition achievement

This work won the overall 1st place for the 2019 undergraduate research competition held by CS department at National Taiwan University.