projects

Enhancing agency in Redirected Walking with Haptic Nudges

May 2022

The first desire when entering virtual reality is to reach out to the world around you, the second? To move around.

The draw of virtual environments is their ability to overrule the limitations of the “real” world. You have no worries about cost, practicality, or transportation.

The challenge - VR traps you within a box - the walls often feel like they are closing in when a simple misstep can lead to you punching a wall or stubbing a toe. These feelings break the experience of VR; we can do better.


We propose Haptic Nudges, an extension to redirected walking 1 techniques to provide an intuitive, unobtrusive understanding of the real-world space. Haptic Nudges maximise virtual traversal distance2 while increasing agency 3. In addition, we update existing Open Source redirected walking toolkits (RDWTK 4, OpenRDW 5) with new redirection algorithms and expose artificial potential fields 6 for all techniques. The combination of these techniques can be applied as a drop-in addition to any existing Unity project.

Introduction

”How far can you walk in a straight line… blindfolded?”

This well-studied problem shows the dominance and reliance of the visual system when determining straight lines; proprioceptive senses are insufficient and result in drifting off course. However, the dual of this problem allows for the visual system to manipulate otherwise straight motion imperceptibly.

How we can break the limitation of movement in physical space when interacting with virtual reality?

Manipulation Techniques

Subtle Gain-based Redirection

Techniques for Gain Redirection
Techniques for Gain Redirection

The most well-studied technique for redirection that maintains coherency in virtual space is “Gain-based” Redirection.

A gain describes a relative difference between real and virtual motion, traditional locomotion techniques will enforce a one-to-one mapping and a gain of 1. A translation gain manipulates the virtual velocity of the user causing the user to walk faster or slower with respect to the real world. Rotational gains cause the viewpoint in the virtual space to rotate faster or slower relative to the tracked head rotation. Curvature-based techniques apply rotation to the world during forward motion. Applying rotation, causes straight lines to curve alongside this vibration.

Example maze redirection

Our extension of OpenRDW / RDWTK performs Gain-based redirected walking, and supplies “Resetters” such as the 2-1 turn. As part of these changes, the setup was updated for modern Unity releases and XRTK HMD allowing for additional Headset types.

Due to the lack of real visual input in a virtual reality setting, people have a threshold for noticing both positive and negative gains (with a preference for positive gains). There have been extensive studies specifying maximal thresholds for awareness which we used to define parameters in our model/framework.

Even with this restriction, however, the virtual-space compression is exponential. The larger the existing tracking space, the larger the virtual space you can recreate allowing for any arbitrary movement.

Virtual Space compression example with a 1.5 exponential factor
Virtual Space compression example with a 1.5 exponential factor

Restricting movement, e.g. a straight corridor allows for potentially infinite walking within moderately sized areas. The width of a tennis court, 36ft, was sufficient for our testing.

Gain-based Resetters

Subtle Gain-based techniques only apply when there is existing motion, and have limits to their redirected force. In essence, they are under-actuated dynamic control systems.

As such we often need an overt technique to add additional motion to produce additional input. The primary technique we use is the 2-1 rotation that converts a 360-degree virtual rotation into a 180-degree physical rotation. This technique is imperceptible (beyond the overt command to spin) and allows for rapid direction change at the cost of observability.

2-1 Spin used to continue a 21m straight-line path despite a 7x7 tracking space
2-1 Spin used to continue a 21m straight-line path despite a 7x7 tracking space

Modern Redirection via APFs

These redirection techniques are best applied with some awareness of the tracking space and the “ideal” points for which to be located to maximise both the current trajectory and possible future trajectories. The redirection will direct the user to these optimal points. For instance, an unobstructed square will have the center point as the minimum potential point that allows for the maximum distance given no current motion.

This notion can be formalised as Artificial Potential Fields 6. This provides a gradient vector for optimal movement and we use this for all redirection algorithms for our haptic nudges.

Haptic Nudges

All the techniques discussed so far use the visual system. Why not use the other senses?

Imagine we had a nasty smell to our right, we’d be repulsed from that area in preference to our, nice smelling, left.

If you’ve ever been driving with the windows down, you can hear the other cars around you and adjust your plans accordingly.

Equally, if you’ve ever been blindfolded and guided by a friend, you can feel their hand on your shoulder and adjust your path accordingly.

In each example, a sense can control your movement without you being hyper-aware of it. The most subtle and intuitive sense is touch.

We apply a gentle haptic nudge to suggest a direction that will maximise the distance you can travel. Taking you back to the center of a square room, or the area of the minimal potential field.

We call this “Haptic Nudging”, because, unlike redirection, we do not redirect where the user goes; we merely aid them in decision-making. Much like you can ignore the touch on the arm of your friend, you can ignore the haptic nudge.

Consider moving around a house in virtual reality, there are often multiple decision points where you could move in multiple ways; you can turn into the living room or the kitchen.

Typically this choice occurs by fancy; you’re feeling a little pekish and head to the kitchen. However sometimes this virtual movement will lead you to a tracking space boundary. You’re going to run into a wall, or a chair, or a table. This will force an overt redirection where you have to spin in pace (for instance).

You aren’t aware of this trade-off, and would be perfectly happy moving into the living room. This choice of “circulation” - which pathway to take - can create dramatic impacts on the experience. When applied alongside gain redirection techniques, you can end up visiting the kitchen after having moved around the living room without ever having to spin. Now when they return to the same virtual point, they are located in a completely different physical location.

Guiding you to the “right” circulation path is the goal of Haptic Nudging. This redirection is subtle and ignorable if desired for a particular game-play element.

After using the user will often subconsciously prefer directions that lack vibration, allowing for nudging without the user paying conscious attention to the feeling.

However, the key disadvantage of this technique is that it requires a choice to be influential. It applies well in high complexity environments where multiple pathways exist, but not in simple environments such as the straight line corridor. That isn’t to say it is useless in such an environment as it can still encourage wandering paths & head rotation that can be utilized by subtle gain-based techniques.

Visualisation of a leftward haptic nudge

Nudge ‘Left’

Visualisation of a rightward haptic nudge

Nudge ‘Right’

We apply Haptic Nudges by considering the global space top-down vectors XdirX_{dir} (the normalised direction from the current user pose) and XmingradX_{mingrad} the vector to the nearest local minimum in the APF field.

Together these encode the haptic amplitude, A(t)A(t), and signal frequency F(t)F(t) at time tt. Note that RR here denotes the “bounding” circle of the tracking space.

A(t)=AmaxXdir(t)X˙mingrad(t)180A(t) = A_{max} \frac{|X_{dir}(t) \dot X_{mingrad}(t)|}{180} F(t)=1+FmaxXmingrad(t)RF(t) = 1 + F_{max} \frac{|X_{mingrad}(t)|}{R}

These signal commands are serialised with a simple convention of <device_name>:O<dc_offset>A<amplitude>F<frequenncy> and broadcasted via Bluetooth serial to the connected device.

A bump wave graph

Haptic Rendering of “Bump” wave given the input O1A2F1 (offset 1V, amplitude 2V, and frequency 1Hz)

The ESP32s render the given waveform by adjusting the duty cycle of a PWM-controlled ERM motor on a maximum period of 1ms. This allows up to 1000Hz frequencies to be rendered. We use nn to denote the loop counter.

D(t)=O+Asin(2(nmod1000)πf1000)D(t) = O + |A \sin(\frac{2 * (n \mod 1000) * \pi * f} {1000})|

Next Steps

Virtual Environment Awareness

All the algorithms for redirection currently implemented are virtual-space aware but tracking-space blind. This means they do not take into account the variations in virtual space to reduce potential trajectories and hence optimise the redirection because of it. This is exemplified in “Virtual Environment Awareness”, where the blind algorithm will attempt to optimise position to the center point assuming that all possible trajectories are equally likely. Meanwhile, a virtual-space-aware algorithm will adjust the possible trajectories to realign the desired tracking-space, virtual-space mapping to support the right-wards movement at the detriment of left-wards movement which is blocked by a virtual wall.

Virtual Environment Aware Algorithm
Aware:

Virtual Environment Aware Algorithm, the virtual space is offset to allow for the wall in the tracking space.

Blind:

The virtual space is not offset resulting in multiple pathways colliding with the edge of the tracking space.

Saccadic Redirection

Gain redirection produces a consistent, smooth transformation of trajectories in real and virtual space. However, the continuous nature prevents small spaces (<100m2<100m^2) from being used for redirection; instead, a rapid and discrete method would allow for greater compression in small spaces.

Detection of Saccadic suppression and redirecting users during the resulting temporary blindness allows for this 7. Applying both Gain Redirection, Saccadic Redirection, and Haptic Nudges will vastly improve the potential compression.

  1. Redirected Walking in Place
    Razzaque, Sharif, David Swapp, Mel Slater, Mary C Whitton, and Anthony Steed. 2002. In EGVE, 2:123–30.

  2. walk a long way in VR

  3. You feel like you are in control. You are causing the movement, the movement isn’t being done to you.

  4. The Redirected Walking Toolkit: A Unified Development Platform for Exploring Large Virtual Environments
    Azmandian, Mahdi, Timofey Grechkin, Mark Bolas, and Evan Suma. 2016. In 2016 IEEE 2nd Workshop on Everyday Virtual Reality (WEVR), 9–14. IEEE.

  5. OpenRDW: A Redirected Walking Library and Benchmark with Multi-User, Learning-Based Functionalities and State-of-the-Art Algorithms.
    Li, Yi-Jun, Miao Wang, Frank Steinicke, and Qinping Zhao. 2021. In 2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 21–30. IEEE.

  6. Dynamic Artificial Potential Fields for Multi-User Redirected Walking
    Dong, Tianyang, Xianwei Chen, Yifan Song, Wenyuan Ying, and Jing Fan. 2020. In 2020 IEEE Conference on Virtual Reality and 3d User Interfaces (VR), 146–54. IEEE. 2

  7. Towards Virtual Reality Infinite Walking: Dynamic Saccadic Redirection
    Sun, Qi, Anjul Patney, Li-Yi Wei, Omer Shapira, Jingwan Lu, Paul Asente, Suwen Zhu, Morgan McGuire, David Luebke, and Arie Kaufman. 2018. ACM Transactions on Graphics (TOG) 37 (4): 1–13.