Enhancing agency in Redirected Walking with Haptic Nudges

The first desire when entering virtual reality is to reach out to the world around you, the second? To move around.

The draw of virtual environments is their ability to overrule the limitations of the “real” world. You have no worries about cost, practicality, or transportation.

The challenge - VR traps you within a box - the walls often feel like they are closing in when a simple misstep can lead to you punching a wall or stubbing a toe. These feelings break the experience of VR; we can do better.

We propose Haptic Nudges, an extension to redirected walking ¹ techniques to provide an intuitive, unobtrusive understanding of the real-world space. Haptic Nudges maximise virtual traversal distance² while increasing agency ³. In addition, we update existing Open Source redirected walking toolkits (RDWTK ⁴, OpenRDW ⁵) with new redirection algorithms and expose artificial potential fields ⁶ for all techniques. The combination of these techniques can be applied as a drop-in addition to any existing Unity project.

Introduction

”How far can you walk in a straight line… blindfolded?”

This well-studied problem shows the dominance and reliance of the visual system when determining straight lines; proprioceptive senses are insufficient and result in drifting off course. However, the dual of this problem allows for the visual system to manipulate otherwise straight motion imperceptibly.

How we can break the limitation of movement in physical space when interacting with virtual reality?

Manipulation Techniques

Subtle Gain-based Redirection

The most well-studied technique for redirection that maintains coherency in virtual space is “Gain-based” Redirection.

A gain describes a relative difference between real and virtual motion, traditional locomotion techniques will enforce a one-to-one mapping and a gain of 1. A translation gain manipulates the virtual velocity of the user causing the user to walk faster or slower with respect to the real world. Rotational gains cause the viewpoint in the virtual space to rotate faster or slower relative to the tracked head rotation. Curvature-based techniques apply rotation to the world during forward motion. Applying rotation, causes straight lines to curve alongside this vibration.

Due to the lack of real visual input in a virtual reality setting, people have a threshold for noticing both positive and negative gains (with a preference for positive gains). There have been extensive studies specifying maximal thresholds for awareness which we used to define parameters in our model/framework.

Even with this restriction, however, the virtual-space compression is exponential. The larger the existing tracking space, the larger the virtual space you can recreate allowing for any arbitrary movement.

Restricting movement, e.g. a straight corridor allows for potentially infinite walking within moderately sized areas. The width of a tennis court, 36ft, was sufficient for our testing.

Gain-based Resetters

Subtle Gain-based techniques only apply when there is existing motion, and have limits to their redirected force. In essence, they are under-actuated dynamic control systems.

As such we often need an overt technique to add additional motion to produce additional input. The primary technique we use is the 2-1 rotation that converts a 360-degree virtual rotation into a 180-degree physical rotation. This technique is imperceptible (beyond the overt command to spin) and allows for rapid direction change at the cost of observability.

Modern Redirection via APFs

These redirection techniques are best applied with some awareness of the tracking space and the “ideal” points for which to be located to maximise both the current trajectory and possible future trajectories. The redirection will direct the user to these optimal points. For instance, an unobstructed square will have the center point as the minimum potential point that allows for the maximum distance given no current motion.

This notion can be formalised as Artificial Potential Fields ⁶. This provides a gradient vector for optimal movement and we use this for all redirection algorithms for our haptic nudges.

Haptic Nudges

All the techniques discussed so far use the visual system. Why not use the other senses?

Imagine we had a nasty smell to our right, we’d be repulsed from that area in preference to our, nice smelling, left.

If you’ve ever been driving with the windows down, you can hear the other cars around you and adjust your plans accordingly.

Equally, if you’ve ever been blindfolded and guided by a friend, you can feel their hand on your shoulder and adjust your path accordingly.

In each example, a sense can control your movement without you being hyper-aware of it. The most subtle and intuitive sense is touch.

We apply a gentle haptic nudge to suggest a direction that will maximise the distance you can travel. Taking you back to the center of a square room, or the area of the minimal potential field.

We call this “Haptic Nudging”, because, unlike redirection, we do not redirect where the user goes; we merely aid them in decision-making. Much like you can ignore the touch on the arm of your friend, you can ignore the haptic nudge.

Consider moving around a house in virtual reality, there are often multiple decision points where you could move in multiple ways; you can turn into the living room or the kitchen.

Typically this choice occurs by fancy; you’re feeling a little pekish and head to the kitchen. However sometimes this virtual movement will lead you to a tracking space boundary. You’re going to run into a wall, or a chair, or a table. This will force an overt redirection where you have to spin in pace (for instance).

You aren’t aware of this trade-off, and would be perfectly happy moving into the living room. This choice of “circulation” - which pathway to take - can create dramatic impacts on the experience. When applied alongside gain redirection techniques, you can end up visiting the kitchen after having moved around the living room without ever having to spin. Now when they return to the same virtual point, they are located in a completely different physical location.

Guiding you to the “right” circulation path is the goal of Haptic Nudging. This redirection is subtle and ignorable if desired for a particular game-play element.

After using the user will often subconsciously prefer directions that lack vibration, allowing for nudging without the user paying conscious attention to the feeling.

However, the key disadvantage of this technique is that it requires a choice to be influential. It applies well in high complexity environments where multiple pathways exist, but not in simple environments such as the straight line corridor. That isn’t to say it is useless in such an environment as it can still encourage wandering paths & head rotation that can be utilized by subtle gain-based techniques.

We apply Haptic Nudges by considering the global space top-down vectors $X_{dir}$ (the normalised direction from the current user pose) and $X_{mingrad}$ the vector to the nearest local minimum in the APF field.

Together these encode the haptic amplitude, $A(t)$, and signal frequency $F(t)$ at time $t$. Note that $R$ here denotes the “bounding” circle of the tracking space.

These signal commands are serialised with a simple convention of <device_name>:O<dc_offset>A<amplitude>F<frequenncy> and broadcasted via Bluetooth serial to the connected device.

The ESP32s render the given waveform by adjusting the duty cycle of a PWM-controlled ERM motor on a maximum period of 1ms. This allows up to 1000Hz frequencies to be rendered. We use $n$ to denote the loop counter.

Next Steps

Virtual Environment Awareness

All the algorithms for redirection currently implemented are virtual-space aware but tracking-space blind. This means they do not take into account the variations in virtual space to reduce potential trajectories and hence optimise the redirection because of it. This is exemplified in “Virtual Environment Awareness”, where the blind algorithm will attempt to optimise position to the center point assuming that all possible trajectories are equally likely. Meanwhile, a virtual-space-aware algorithm will adjust the possible trajectories to realign the desired tracking-space, virtual-space mapping to support the right-wards movement at the detriment of left-wards movement which is blocked by a virtual wall.

Saccadic Redirection

Gain redirection produces a consistent, smooth transformation of trajectories in real and virtual space. However, the continuous nature prevents small spaces ($<100m^2$) from being used for redirection; instead, a rapid and discrete method would allow for greater compression in small spaces.

Detection of Saccadic suppression and redirecting users during the resulting temporary blindness allows for this ⁷. Applying both Gain Redirection, Saccadic Redirection, and Haptic Nudges will vastly improve the potential compression.