[CHI 2025] VRCaptions: Design Captions for DHH Users in Multiplayer Communication in VR

Tianze Xie, Xuesong Zhang, Feiyu Huang, Di Liu, Pengcheng An, Seungwoo Je

In Proceedings of the 2025 ACM CHI Conference on Human Factors in Computing Systems (CHI 2025)

Abstract

Accessing auditory information remains challenging for DHH individuals in real-world situations and multiplayer VR interactions. To improve this, we investigated caption designs that specialize in the needs of DHH users in multiplayer VR settings. First, we conducted three co-design workshops with DHH participants, social workers, and designers to gather insights into the specific needs of design directions for DHH users in the context of a room escape game in VR. We further refined our designs with 13 DHH users to determine the most preferred features. Based on this, we developed VRCaptions, a caption prototype for DHH users to better experience multiplayer conversations in VR. We lastly invited two mixed-hearing groups to participate in the VR room escape game with our VRCaptions to validate. The results demonstrate that VRCaptions can enhance the ability of DHH participants to access information and reduce the barrier to communication in VR. [doi]