Game Engine Haptic Editor

The ask

How might we enable creation of advanced haptic content for multiple platforms like Nintendo Switch and virtual reality?

Context

These are authoring tools built into Unreal Engine and, later, Unity that let designers create and sync tactile feedback without writing code. I built them around a timeline, the mental model people already know from audio and video editors, so the learning curve is near-zero and effects can be auditioned live and pinned to exact moments in an animation. The goal was to take something that used to require an engineer and put it directly in a creative's hands.

My role

As the UX lead I owned the interaction flow and interface requirements, working closely with research to iterate toward something people found immediately intuitive. I drove the interaction and feature set design end to end, in partnership with engineering to build and validate it.

Research impact

Our interviews with developers and game designers surfaced a consistent pattern: rumble design was an afterthought, deferred to the very end of production and often handed to an intern to knock out simple buzzing effects. We knew the opposite was required. Convincing effects depend on tight sync and dynamic ties to animation, a demand only growing as faster, higher-fidelity vibrotactile actuators arrived. The implication was clear: effect design had to live as part of the animation and audio process, not bolted on after it.

A second insight sharpened the approach. Developers told us, time and again, that they would only try a new tool if they had spare time, always with a knowing laugh that they never did. That ruled out an external tool. We had initially explored a standalone application running outside the engine, but this insight pointed elsewhere, and I advocated for embedding the haptic editing flow directly in the engine, wired into the animation frameworks of Unity and Unreal. By meeting developers exactly where they already worked, we lowered the cognitive barrier to trying something new rather than asking for the time they did not have.

Unreal Engine 4

The layout is familiar to those with any timeline experience and can quickly be used to create and audition haptics on the fly. This tool supports multiple configurable tracks to simplify haptic design for multiple devices. Technical knowledge of actuators are not necessary due to the abstracted nature of the interface. Complex haptics can be generated and synced to discrete points within an animation to enable the most complex haptic sequences possible on a platform in a fraction of the time. This build was used to create complex effects for Immersion's VR Gauntlet demo. Developers who experienced the effects estimated a two-week time span to recreate what they felt. In practice, effects were created in under 30 minutes.

This has been demonstrated publicly at both GDC 2017 and the SVVR Expo 2017 to accolade. Not only does this tool click with any developer that has seen it, but it has also been called "the missing link" for those trying to create complex and immersive VR experiences. Coverage can be found on UploadVR, Tom's Hardware, and Road to VR.

Unity

A followup to the UE4 version, the Unity version was a notable upgrade and included a refined timeline editor, clip-specific curve editing for strength and pitch (if on an applicable endpoint), animation and audio import, and automatic effect triggering when designed with animations. For this release, emphasis was placed on a higher rate of design and iteration. It was successful: the design of all haptics in the space demo took a combined ten to twenty minutes total to complete. In older workflows, these type of effects would have taken hours to construct and sync.