I develop data-driven methods and interactive agents to decode the semantics of multimedia content and user interfaces, enabling accessible consumption, creation, and interaction for users.
I am currently a Ph.D. student at CMU HCI Institute, advised by professors Jeff Bigham and Amy Pavel
For more details, check out my: CV | Google Scholar | Linkedin
Publications
-
DreamStruct: Understanding Slides and User Interfaces via Synthetic Data Generation Yi-Hao Peng, Faria Huq, Yue Jiang, Jason Wu, Amanda Xin Yue Li, Jeffrey Bigham, Amy Pavel European Conference on Computer Vision (ECCV 2024) project website | paper
-
UIClip: A Data-driven Model for Assessing User Interface Design Jason Wu, Yi-Hao Peng, Amanda Xin Yue Li, Amanda Swearngin, Jeffrey Bigham, Jeffrey Nichols ACM Symposium on User Interface Software and Technology (UIST 2024) project website | paper | video preview
-
Long-form Answers to Visual Questions Asked by Blind and Low Vision People Mina Huh, Fangyuan Xu, Yi-Hao Peng, Chongyan Chen, Hansika Murugu, Danna Gurari, Eunsol Choi, Amy Pavel Conference on Language Modeling (COLM 2024) project website | paper | video preview
-
AdvCAPTCHA: Creating Usable and Secure Audio CAPTCHA with Adversarial Machine Learning Hao-Ping (Hank) Lee, Wei-Lun Kao, Hung-Jui Wang, Ruei-Che Chang, Yi-Hao Peng, Fu-Yin Cherng, Shang-Tse Chen NDSS Symposium on Usable Security and Privacy (USEC 2024) paper | video preview
-
Designing a conversational telepresence robot for homebound older adults Yaxin Hu, Laura Stegner, Yasmine Kotturi, Caroline Zhang, Yi-Hao Peng, Faria Huq, Yuhang Zhao, Jeffrey Bigham, Bilge Mutlu ACM Conference on Designing Interactive Systems (DIS 2024) paper | video preview
-
GenAssist: Making Image Generation Accessible Mina Huh, Yi-Hao Peng, Amy Pavel ACM Symposium on User Interface Software and Technology (UIST 2023) project website | paper | video preview
-
Slide Gestalt: Automatic Structure Extraction in Slide Decks for Non-Visual Access Yi-Hao Peng, Peggy Chi, Anjuli Kannan, Meredith Ringel Morris, Irfan Essa ACM Conference on Human Factors in Computing Systems (CHI 2023) paper | video preview
-
WebUI: A Dataset for Enhancing Visual UI Understanding with Web Semantics Jason Wu, Siyan Wang, Siman Shen, Yi-Hao Peng, Jeffrey Nichols, Jeffrey Bigham ACM Conference on Human Factors in Computing Systems (CHI 2023) paper | video preview
-
Reality Rifts: Wonder-ful Interfaces by Disrupting Perceptual Causality Lung-Pan Cheng, Yi Chen, Yi-Hao Peng, Christian Holtz ACM Conference on Human Factors in Computing Systems (CHI 2023) paper | video preview
-
AVScript: Accessible Video Editing with Audio-Visual Scripts Mina Huh, Saelyne Yang, Yi-Hao Peng, Xiang Anthony Chen, Young-Ho Kim, Amy Pavel ACM Conference on Human Factors in Computing Systems (CHI 2023) paper | video preview
-
Diffscriber: Describing Visual Design Changes to Support Mixed-Ability Presentation Authoring Yi-Hao Peng, Jason Wu, Jeffrey P. Bigham, Amy Pavel ACM Symposium on User Interface Software and Technology (UIST 2022) project website | paper | video preview
-
ImageExplorer: Multi-Layered Touch Exploration to Encourage Skepticism for Imperfect AI-Generated Image Captions Jaewook Lee, Jaylin Herskovitz, Yi-Hao Peng, Anhong Guo ACM Conference on Human Factors in Computing Systems (CHI 2022) paper | video preview | talk
-
Slidecho: Flexible Non-Visual Exploration of Presentation Videos Yi-Hao Peng, Jeffrey P. Bigham, Amy Pavel ACM Conference on Computers and Accessibility (ASSETS 2021) project website | paper | talk
-
Say It All: Feedback for Improving Non-Visual Presentation Accessibility Yi-Hao Peng, JiWoong Jang, Jeffrey P. Bigham, Amy Pavel ACM Conference on Human Factors in Computing Systems (CHI 2021) project website | paper | video preview | talk | slides
-
StrengthGaming: Enabling Dynamic Repetition Tempo in Strength Training-based Exergame Design Sih-Pin Lai, Cheng-An Hsieh, Yu-Hsin Lin, Teepob Harutaipree, Shih-Chin Lin, Yi-Hao Peng, Lung-Pan Cheng, Mike Y. Chen ACM Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI 2020) paper | video preview | talk | demo recording
-
WalkingVibe: Reducing Virtual Reality Sickness and Improving Realism while Walking in VR using Unobtrusive Head-mounted Vibrotactile Feedback Yi-Hao Peng, Carolyn Yu, Shi-Hong Liu, Chung-Wei Wang, Paul Taele, Neng-Hao Yu, Mike Y. Chen ACM Conference on Human Factors in Computing Systems (CHI 2020) paper | video preview | talk
-
PersonalTouch: Improving Touchscreen Usability by Personalizing Accessibility Settings based on Individual User’s Touchscreen Interaction Yi-Hao Peng, Muh-Tarng Lin, Yi Chen, Tzu-Chuan Chen, Pin-Sung Ku, Paul Taele, Mike Y. Chen ACM Conference on Human Factors in Computing Systems (CHI 2019) paper | official apple research kit documentation
-
PhantomLegs: Reducing Virtual Reality Sickness using Head-Worn Haptic Devices Shi-Hong Liu, Neng-Hao Yu, Liwei Chan, Yi-Hao Peng, Wei-Zen Sun, Mike Y. Chen IEEE Conference on Virtual Reality and 3D User Interfaces (VR 2019) paper | video preview | talk
-
PeriText: Utilizing Peripheral Vision for Reading Text on Augmented Reality Smart Glasses Pin-Sung Ku, Yu-Chih Lin, Yi-Hao Peng, Mike Y. Chen IEEE Conference on Virtual Reality and 3D User Interfaces (VR 2019) paper | video preview | talk
-
SpeechBubbles: Enhancing Captioning Experiences for Deaf and Hard-of-Hearing People in Group Conversations Yi-Hao Peng, Ming-Wei Hsu, Paul Taele, Ting-Yu Lin, Po-En Lai, Leon Hsu, Tzu-Chuan Chen, Te-Yen Wu, Yu-An Chen, Hsien-Hui Tang, Mike Y. Chen ACM Conference on Human Factors in Computing Systems (CHI 2018) paper | video preview | talk
Press Coverage
-
Researchers Say Head-mounted Haptics Can Combat Smooth Locomotion Discomfort in VR (covered Project WalkingVibe), Road to VR, May, 2020
-
ResearchKit and CareKit Reimagined (covered Project PersonalTouch), Apple Worldwide Developers Conference — session 217 (02:57), June, 2019
-
The Radical Frontier of Inclusive Design (covered Project SpeechBubbles), Fast Company, May, 2018