Posted in: Corporate, Immersive Entertainment

By:

How do you describe a feeling? Not the mood you are in, but the physical sensation of what you touch? This is the challenging question asked when designing tactile effects for video. For creatives not used to working in a tactile medium, it can be somewhat intimidating.

An audio designer understands the intricacies of a sound. The film editor understands the complexities of the visual frame. However, the process for creating touch sensations in a digital format is actually not what you would expect. Most people would assume that it is the direct translation of your sense of feeling, but that is incorrect. Tactile design is closer to accompanying the motion of an actor or the sound of thunder.

Try this…try to think about the act of physically sensing something with your fingertips. For example, when you press the shutter button on your camera to snap a picture. Your mind has given that feeling a verb – like click or snap. You think that you hear a sound, but in reality, there is no sound. Your mind has interpreted that “feeling” to a verbal understanding of a sound, or even, something you might recognize visually. If you try to work that backward to create that “feeling” sensation, it does not work. The wave files don’t equal out.

The technology we use to create the haptics in mobile devices today – small actuators that jolt or spin – are not capable of absolute realism. They can’t deform to the shape of the cloth a character runs their hands through, nor can they simulate the sudden change in heat caused by an explosion. These technological constraints require that we rely on the use of illusion and metaphor, rather than realism, to determine the specifics of the tactile pattern. For example, when a ball bounces on the screen, you’re mind has already associated that visual with the real-life experience. Haptics’ role in this instance is to fill in the gaps to complete the picture further.

This may be daunting concept at first, but understanding few key guidelines can help you learn how to “create a feeling” for the specific scene in a movie, or an action point in a game.

1. As a traditional audio or video editor, you will need to suspend some of your respective best practices when working on haptic designs. While it is easy for those with an audio or video background to try to apply effects based on their respective disciplines, this new medium has new design parameters.

a. Keep open mind for the placement of effects, it may not make sense for the audio or video world, but might be a whole world of difference in haptics.

b. You have a basic knowledge of human perception, use it. It is key to imagine all of your senses working together to create a fuller picture.

2. Understand the key constraints of the actuators for which you are designing – and keep consumer experience in mind.

a. For example, short effects are great for building complex patterns while, long and strong buzzy effects will annoy the audience.

b. Haptic density can be pleasing when effects are frequent enough to fall into a regular cadence, like a musical beat, but there is no haptic equivalent to “room tone.”

3. Accept that while you cannot completely simulate reality, you may use the audio and video to fill in the gaps to generate a perceptual haptic realism—effects that the brain accepts as logical when coinciding with the other senses. Think of it as video + tactile + sound = whole picture.

Tricking the brain is one of the keys to a successful haptic track. Generally, a haptic designer will have the most success by applying tactile effects to the element most obvious in the scene, whether a video or audio element. This might be the largest object on screen, the actions of a speaking actor, or a loud Foley sound to shock the audience. Alternatively, a subtle hint towards the advancement of the story opens up the possibility of haptics for foreshadowing, much like smart editing or subtle audio cues.

Once you have moved past the obvious applications of haptics, you will discover “mood” haptics. These are tactile sensations that are not explicitly associated with any auditory or visual cues, but instead stand alone as their own expressive layer. Imagine a tense situation where the director wants to build suspense: haptics could employ the pattern of a heartbeat – that goes faster and faster as it gets closer to the scary moment climax.

While tactile design is a subtle and complex art, we have been able to successfully initiate both audio and film editors into the world of content haptics. Much like the traditional editing mediums, the more you experiment, the more you’ll internalize the design space and be able to achieve unique and pleasant effects. With regular feedback from a trained haptic designer, audio/video editors will be able to find success within the first few pieces of content they create – and claim touch as another medium that they can use to tell their story, creatively.