By Chris Ullrich, CTO at Immersion
There is tremendous hype surrounding the upcoming launch of the PlayStation 5. Many developers and reviewers have expressed enthusiasm for the haptic capabilities of the DualSense™ controllers. Godfall Game Director Keith Lee was quoted recently in TheGamer:
“What’s exciting about the DualSense controller is the fact that it has stereo vibration in terms of the rumblers, as well as resistance on the triggers, so one of the things that you can do is to create, for the first time, a sensation of your weapon hitting another weapon and how it resonates,” Lee said. “The fact that if you’re sliding across the ground, depending on the surface of the material–you might slide on gravel or sand or water–the way the vibrations work really feel like the way that those surfaces would feel. So, it’s a lot more powerful in terms of creating that experience.” (TheGamer, Sean Murray)
We’re obviously very excited to see a meaningful upgrade to the haptic functionality of consumer game controllers. We’re looking forward to awesome, immersive experiences when creatives use this capability. As CTO of Immersion, I’m elated to see this technology go mainstream – and I’m betting that this is just the beginning of a touch revolution in gaming, virtual reality, and other consumer devices. Haptics in the Sony PlayStation 5 means more than a new console with games enabled with the sense of touch. It means the emergence of a new field of creativity and exploration, ripe for innovation.
The first thing to understand is that the human sense of touch is not like vision and audition. It’s both primal and nuanced. A real-world tactile experience consists of at least 15 distinct dimensions of sensation broadly categorized into texture, adhesion, thermal, compliance, and friction (check out SynTouch, for more details). Tactile perception is fundamentally exploratory, and most rich touch experiences depend on body motion to explore the tactile dimensions. Think of feeling a piece of fabric or the distinct sensations of touching wood vs. glass surfaces.
The second thing to understand is that the sense of touch is integrated into a multi-sensory perception of the world. This is the sound of sliding across a rough surface vs. a smooth one used by our brain to create a richer model of the world.
In DualSense™ controller-based experiences, content creators will start to explore the expressive dimensions of high definition vibration and resistive triggers as an element of immersive gameplay. I expect these creators will quickly realize that they can leverage the perceptual assumptions of users to enable the four actuation channels to cover a broader set of the 15 dimensions of touch than what is strictly available physically within the new system. For consumer devices, this is critical learning, and we’ll quickly see experiences that stretch the hardware to the limits.
However, vibration and resistance are insufficient, even with multimodal perception, to cover the entire range of human touch experiences. The good news is that there are many haptic devices at the research and early-stage commercial level that can fill out these other dimensions, including:
Friction displays: Both Tanvas and Hap2u have demonstrated commercially viable surface haptic display technology. The experience of these devices is one of stickiness (Tanvas) or slipperiness (Hap2u) and can be extremely compelling. It’s very hard/impossible to reproduce this sensation using only vibration feedback.
Ultrasound displays: Ultraleap has developed a futuristic non-contact haptic feedback system that can create a ‘light touch’ experience without any surface contact. Although the sensations are not really distinct from what can be done with vibration, there is nothing to wear or touch, so the experience is uniquely valuable.
Shape-changing wearables: The Hey bracelet is a squeeze-based wearable for couples that provides a novel, non-vibration wristband experience. Pressure and stiffness changing devices bring a new dimension to consumer electronics that are not widely used in the market.
Robotic exoskeletons: There are a few companies that are making whole-hand kinesthetic exoskeletons. HaptX and Contact CI are currently active in this area. These devices are somewhat like the resistive triggers on DualSense™, but for each finger or finger joint. The HaptX glove can also render complex surface textures and compliance.
Thermal displays: Embr Labs have made some progress in commercializing small, low power Peltier elements to provide programmatic heating and cooling to devices.
Multi-actuator/multi-modality: Companies like Neosensory and b-haptics make vibration-based wearables that contain multiple actuators. These devices are capable of much greater levels of immersion and information bandwidth due to the parallel nature of tactile stimulation.
There are other technologies and companies, of course, working in this rapidly growing ecosystem, and this list is not meant to be exhaustive.
At Immersion Corporation, we’ve had decades to explore the commercial possibilities of touch and have learned some important (and expensive) lessons on what is likely to work and what is not likely to work at scale. In my experience, it is these factors that ultimately drive the introduction and adoption of new haptic technologies at the mass-market level.
- Standard hardware: Those of you that have ready my Haptic Stack Series know that the physical hardware is, of course, one of the most impactful parts of a haptic system. In order to produce consistent haptic experiences, at a mass-market scale, this hardware needs to be quite robust with mature manufacturing supply chains. One outcome of the PS5 controller is that high definition vibration will become commonplace in consumer devices because the suppliers will have the ability to make similar motors at scale.
- Content bridging: Like the introduction of 4K TVs, new haptic hardware suffers from a dearth of available content. Just like 4K TVs, hardware manufacturers will bridge this gap by upsampling an appropriate signal. We’ve seen this in haptics as mobile OEMs scramble to introduce audio-to-vibe technology so that on-device content can exploit more capable (and more expensive) vibration hardware. Ultimately, these bridging technologies fade away as more and more new content gets created that is authored for the new capability, and the content creation ecosystem matures.
- Consumer value: Ultimately, new touch capabilities live or die based on the value that they bring to end-users. Do the new capabilities enable new, richer, or more valuable human-machine experiences? Do they enable new levels of personal connection? Do they reduce user frustration? If 90% of the user value can be achieved with a fidelity refinement of vibration, then that’s the most likely path for the market.
Of course, there are many other enabling technologies and ecosystem relationships that make or break a new haptic capability in the market. Still, these principles have shown themselves to be reliable beacons for ultimate success.
Given all that, what should we look for in post-PS5 devices?
- The commoditization of high definition vibration in consumer devices will become widespread. We should expect OEMs to find clever ways to use multiple vibration motors to achieve high-value features in their devices.
- The resistive trigger concept in the PS5 will also percolate out into other gaming peripherals, location-based devices, virtual reality controllers, automotive interfaces, and other markets. I expect this concept will be extended in various ways, including in novel peripherals such as exoskeletons and other devices.
Beyond these two technologies, it’s clear that there is a vibrant ecosystem that is rapidly growing up as haptics becomes a better understood value of end-user devices. I look forward to seeing a range of the technologies in this post (and others) come to market in high-end devices and eventually in devices that are commonplace.