Li et al. 2024. Investigating Creation Perspectives and Icon Placement Preferences for On-Body Menus in Virtual Reality. PACM HCI: ISS 2024.

On-body menus in VR embed interfaces onto the user’s body, offering a unique interaction method. This research examines the impact of the creation process on menu effectiveness, comparing first-person, third-person, and mirror perspectives. Findings suggest the mirror perspective enhances creation speed and recall. User preferences for icon placement based on category were identified, with insights to guide intuitive on-body interactions in virtual environments.

Video | DOI | PDF

Li et al. 2024. Swarm Manipulation: An Efficient and Accurate Technique for Multi-Object Manipulation in Virtual Reality. Computers & Graphics.

In this paper, we extend our previous ACM SUI 2023 paper with a second user study that explores more realistic and open-ended VR scenarios, exploring the transformation of user ownership, agency, and perception through the use of the Swarm Manipulation technique. We also introduce a Double Diamond design process to guide future research on designing swarm interaction techniques in mixed reality.

DOI | PDF

He et al. 2024. MetaDragonBoat: Exploring Paddling Techniques of Virtual Dragon Boating in a Metaverse Campus. ACM MM 2024.

We propose MetaDragonBoat, a VR solution to preserve the Dragon Boat Festival by enabling virtual participation in dragon boat racing. Using digital twin technology and haptic feedback controllers, it replicates physical exertion, allowing broader demographics to experience and appreciate this cultural heritage.

Video | DOI | PDF

Patibanda et al. 2023. Fused Spectatorship: Designing Bodily Experiences Where Spectators Become Players. PACM HCI: CHI PLAY 2023.

We propose Fused Spectatorship, where spectators control their hands using an EMS system to actively participate in digital games. A study showed participants couldn't distinguish between watching their hands play and playing the games themselves. We offer design considerations and discuss ethical implications for future fused spectatorship experiences.

Video | Presentation | DOI | PDF

Patibanda et al. 2023. Auto-Paízo Games: Towards Understanding the Design of Games that Aim to Unify a Player’s Physical Body and the Virtual World. PACM HCI: CHI PLAY 2023.

We propose Auto-Paízo Games, a novel approach that unifies the physical body and the virtual world in digital bodily games. By using an Electrical Muscle Stimulation (EMS) system, players control one hand (input) to compete against the other hand (output). Thematic analysis of a study with 12 participants revealed four player experience themes, highlighting engagement with bodily movements and the ambiguity of using the body as play material. Our work aims to bridge the divide between the physical and virtual realms.

Video | Presentation | DOI | PDF

Li et al. 2023. Swarm Manipulation in Virtual Reality. ACM SUI '23.

This paper introduces Swarm Manipulation, a VR-based technique for swarm control, and compares its performance to Virtual Hand and Controller methods through a 12-participant study. The evaluation spans three tasks across five conditions, revealing Swarm Manipulation's faster speeds in most tasks and reduced resizing errors. However, a speed-accuracy trade-off exists in rotation tasks. These findings suggest the technique's potential to improve VR usability and set the stage for future research on swarm interaction and control.

DOI | PDF

Mueller et al. 2023. Towards Understanding the Design of Intertwined Human-computer Integrations. ACM TOCHI 2023.

Our work explores human-computer integration trends, specifically focusing on shared agency over the user's body between the user and computational machines. Termed "intertwined integration," this approach blurs the distinction between user and machine control. Through three case studies, we propose two dimensions ("awareness of machine's agency" and "alignment of machine's agency") to define a design space. We identify four roles (angel, butler, influencer, and adversary) that machines can assume within this space and discuss strategies for designers based on our experience. Our aim is to advance the understanding of shared agency in the HCI field.

Presentation | DOI | PDF

Mueller et al. 2021. Limited Control Over the Body as Intriguing Play Design Resource. ACM CHI 2021.

The integration of interactive play and the human body in "bodily play" systems is gaining interest. While these systems prioritize player control over bodily actions, we explore the potential of "limited control over the body" as a design resource. Using three bodily play systems, we demonstrate how designers can engage players through varying degrees of indirect control. We propose four strategies: Exploration, Reflection, Learning, and Embracement, to employ limited control over the body. By applying these strategies, designers can enhance body engagement and promote the benefits of play.

Presentation | DOI | PDF

Lu et al. 2020. Exploration of Hands-free Text Entry Techniques For Virtual Reality. IEEE ISMAR 2020.

This paper explores two hands-free VR text entry methods: BlinkType and NeckType, using eye blinks and neck movements, respectively, compared to the conventional DwellType. A user study shows BlinkType achieves the highest text entry rate and is the preferred technique, enhancing hands-free text entry in VR.

DOI

Xu et al. 2020. Exploring Visual Techniques for Boundary Awareness During Interaction in Augmented Reality Head-Mounted Displays. IEEE VR 2020.

We explore visual techniques for boundary awareness in Augmented Reality (AR) Head-Mounted Displays (HMDs) during mid-air hand interactions. Users often move their hands outside the tracked area, impacting dynamic tasks. Boundary awareness issues in AR devices are underexplored. Through a formative study, we identify challenges and propose four methods (static surfaces, dynamic surface(s), static coordinated lines, dynamic coordinate line(s)). We evaluate these methods against a baseline condition. Our findings show that visual boundary awareness methods can assist dynamic mid-air hand interactions, but their effectiveness depends on the user.

Presentation | DOI

Xu et al. 2020. Results and Guidelines From a Repeated-Measures Design Experiment Comparing Standing and Seated Full-Body Gesture-Based Immersive Virtual Reality Exergames: Within-Subjects Evaluation. JMIR Serious Games (Q1, SCI). 

This study compared full-body gesture-based iVR standing exergames with seated exergames. Among 52 participants, seated exergames showed higher heart rate, exertion, and perceived exertion. However, they also led to higher peripheral and sopite-related sickness. Seated exergames were rated higher in value/usefulness for intrinsic motivation. They offer benefits such as increased exertion and suitability for small spaces, but careful design and consideration of motion sickness are important.

DOI

Lin et al. 2024. The Framework of NAVIS: Navigating Virtual Spaces with Immersive Scooters. ACM MUM 2024.

We present the conceptual framework of NAVIS--Navigating Virtual Spaces with Immersive Scooters, a novel system that utilizes a scooter-based interface to enhance both navigation and interaction within virtual environments. NAVIS combines real-time physical mobility, haptic feedback, and CAVE-like (Cave Automatic Virtual Environment) technology to create a realistic sense of travel and movement, improving both spatial awareness and the overall immersive experience. 

DOI

Li et al. 2021. vrCAPTCHA: Exploring CAPTCHA Designs in Virtual Reality. ACM CHI 2021. Interactivity.

To address the need for exclusive and interactive CAPTCHA designs in virtual reality (VR) devices, this paper presents four traditional 2D CAPTCHAs (text-based, image-rotated, image-puzzled, and image-selected) adapted for VR. Additionally, two VR-specific CAPTCHA design prototypes (task-driven and bodily motion-based) are proposed based on VR's 3D interaction characteristics. A user study with six participants evaluated the feasibility of these designs. The findings suggest that the proposed VR CAPTCHAs can serve as inspiration for future CAPTCHA designs in VR.

Video | DOI

Li et al. 2023. GesMessages: Using Mid-air Gestures to Manage Notifications. ACM SUI 2023. Poster.

This paper presents GesMessages, a mid-air interaction application for managing real-time message notifications on computing devices. Utilizing onboard cameras, the system supports three key gestures for notification management: expansion, hiding, and deletion. We detail the system architecture and discuss its applicability in context-aware systems, thereby contributing to the field of gestural interaction. This research advances our understanding of the role of mid-air gestures in message management and interactive systems.

DOI

Li et al. 2022. GesPlayer: Using Augmented Gestures to Empower Video Players. ACM ISS 2022. Poster.

We introduce GesPlayer, a gesture-based video player that allows users to control playback, volume, and screen brightness through semantic gestures. By using hand gestures as an interface, users can navigate videos without the need for a mouse or keyboard. We aim to explore the inclusiveness of hand-based interaction and advance the use of semantic gestures in interactive environments through computational interaction methods.

Video | DOI

Patibanda et al. 2021. Actuating Myself: Designing Hand-Games Incorporating Electrical Muscle Stimulation. ACM CHI PLAY 2021. WIP. 

We explore the use of Electrical Muscle Stimulation (EMS) technology to enhance traditional hand-games like Rock-paper-scissors and Thumb-war. We designed and evaluated two EMS games, "Slap-me-if-you-can" and "3-4-5," which can be played individually. Through thematic analysis, we identified three key themes: gameplay experiences and the influence of EMS hardware, interaction with EMS and the calibration process, and shared control and its impact on playing EMS games. Our findings contribute to the understanding of EMS in supporting hand-games, advancing movement-based games as a whole.

Video | DOI

Li et al. 2021. Myopic Bike and Say Hi: Games for Empathizing with The Myopic. ACM CHI PLAY 2021. Student Game Design Competition. 

Myopia, a prevalent eye condition, impacts people's ability to focus on distant objects and reduces their quality of life. To bridge the empathy gap, we developed two virtual reality (VR) games, "Myopic Bike" and "Say Hi," simulating daily challenges faced by myopic individuals. Through questionnaires and interviews with four participants, we found that the games successfully engaged non-myopic players and fostered empathy towards myopia sufferers. Our VR games offer an immersive and judgment-free experience, promoting understanding and empathy for myopia.

Video | DOI

Xu et al. 2020. VirusBoxing: A HIIT-based VR Boxing Game. ACM CHI PLAY 2020. Student Game Design Competition. 

We developed VirusBoxing, a boxing-based VR exergame, to address the time constraints of regular exercise. By incorporating high-intensity interval training (HIIT) and empowering players with enhanced abilities, such as precise jabs at distant objects, VirusBoxing offers an efficient, effective, and enjoyable exercise experience. Our paper discusses the adaptation of HIIT protocols and gameplay features to promote physical activity and improve players' health and quality of life.

Video | DOI