Publication Type

Conference Proceeding Article

Version

publishedVersion

Publication Date

6-2026

Abstract

A common collocated group setting in mixed-reality (MR) collaboration is a person wearing a MR headset (HMD user) and presenting MR contents to audiences who are not provided with such specialized devices (Non-HMD users). In this setting, while Non-HMD users can view the MR environment shown on a large physical display, it still remains challenging for the HMD user to interpret their pointing gesture when they spatially refer to objects in the MR environment. To address this, we designed and evaluated two pointing techniques—SCREEN and SCREEN+SPACE—that support Non-HMD users in referring to MR content. Screen pointing allows users to refer to MR objects by pointing at their representation on the display, while Screen+Space pointing additionally enables direct spatial references toward the HMD user’s environment. In a study involving 36 participants (18 pairs) performing an expert-novice collaboration task, both techniques improved communication fluency and reduced cognitive workload. Notably, Screen+Space pointing was preferred over Screen pointing, where Non-HMD users often pointed relative to the HMD user’s physical space rather than the screen. This suggests that people naturally treat MR content as embedded in shared space, and that space pointing enables more natural and effective communication in asymmetric MR collaborations.

Keywords

mixed reality, collocated collaboration, asymmetric, pointing gestures

Discipline

Artificial Intelligence and Robotics | Graphics and Human Computer Interfaces

Research Areas

Software and Cyber-Physical Systems

Areas of Excellence

Digital transformation

Publication

AVI '26: Proceedings of the 2026 International Conference on Advanced Visual Interfaces, Venice, Italy, June 8-12

First Page

1

Last Page

9

Identifier

10.1145/3811427.3811428

Publisher

ACM

City or Country

New York

Additional URL

https://doi.org/10.1145/3811427.3811428

Download

Share

COinS