Institute For The Future

Feeling is Believing

Interview with David Birnbaum

[Download article as a PDF]

David Birnbaum manages Haptic User Experience Design at Immersion Corporation. During his eight years there, Birnbaum has been working to transition consumer-facing haptics—technology that engages your sense of touch—from curiosity to necessity. He’s been exploring the emerging and expanding role of haptics, from forces and vibrations that help gamers become immersed in virtual worlds, to social touch behaviors enabled by mobile device apps that help people feel more connected and present with each other. We talked to Birnbaum about the potential of haptics, and how the landscape is opening up for this new design discipline.

Some of our readers might not be too familiar with haptics. Can you quickly bring them up to speed?

The term “haptics” derives from the Greek word haptesthai, which means “to contact,” or “to grasp.” While optics is for the eye, haptics is for the hand. It’s technology that engages your sense of touch. The interesting thing about the sense of touch is that in some ways it’s more complex and nuanced than our other sensory systems.

Touch flies under the radar, and it’s easy to assume that it’s not important. We have tools like Instagram and YouTube for conveying visual and auditory ideas, but we don’t have anything like that for touch. It’s too bad, because touch is extremely important to our relationships and social interaction.

Why are you interested in haptics?

The first time I ever felt haptics was at a workshop with interaction design pioneer Bill Verplank, and it was immediately obvious to me that this was going to start a revolution in human computer interaction. We think about human computer interaction as being a visual display, maybe speakers, and then a mouse which is an extremely low fidelity gesture input device. But when we create a powerful touch experience, it’s mind-blowing, because when you can touch it, it’s real. You can find this implicit assumption in media reports about haptics all the time. Journalists usually write that haptics let you “touch” virtual objects. They use quotes, because, of course, you can’t touch things that aren’t really there! Touch is reality. Haptics messes with that. That’s what I love about this field.

Are there design rules you’re working with?

There are some key best practices. Haptics is a feedback response to something. Very rarely do you get a tactile sensation in a vacuum without other sensations—even if you rub this table, you’re getting visual and audio feedback at the same time as the tactile feedback, and you use all three to perceive the texture. There’s a lot of research proving this, but here’s one example. We might think that we touch fabrics to feel their textures and use that feeling to identify them. But research shows that if you plug your ears and wear a blindfold, and then try to identify textiles just by feel, your ability to identify them plummets.

We see this play out all the time in design of apps. For example, if you play a soft, randomized tactile pattern on a phone with the vibration motor that’s in there, if you’re like most people you’re not going to think it feels like anything in particular. But, if you synchronize that haptic pattern to a video of rain falling, the tactile sensations suddenly feel like droplets. And all of a sudden you’re like, “Oh my God, I can feel the rain.” It’s a very intense illusion.

That multi-modal synchronicity is extremely important, and there are guidelines for how to do it well. There are relative magnitudes that you need to keep in mind, and there’s only a certain amount of latency that we can tolerate before we start to feel that the haptics is disconnected from the visual.

Tactile designers play with these concepts when they design new experiences. We’ve been doing haptic design for animated chat stickers lately— short animations with a synchronized haptic track. We were designing a crying sad sticker, and in the first draft, the artist had created these waterfalls that were continuously flowing from the character’s eyes. I thought, “Well, we can’t really make an interesting tactile effect with the motor in a smartphone that feels like evenly flowing water, but I know that we can make a great droplet illusion, so why don’t we make the character cry individual tears?” So the animation was changed so that the tears fall slowly one by one, and splash onto a table, and the final design came together.

What problems will haptics solve in the foreseeable future?

Haptics is going to make gesture control in the Internet of Things feasible. Without haptics, you’d be surrounded by interactive objects and you won’t feel in control of them. Think about when your operating system starts to feel laggy or unresponsive, and you feel helpless—now imagine your house feels like that. Haptics will help you stay literally and figuratively connected to your smart environment, so you understand what’s going on and feel in control.

In another arena, robots are going to be here very soon. Physical interaction with robots is going to be a key thing to get right in order to make them fully functional members of society. You will be in the same room as robots and if they can’t pick up on subtle touch cues, you’re not going to feel like you can really communicate with them like you could with a human. It’s a really exciting problem and we’re going to have to solve it.

What are the obstacles to widespread adoption of haptics?

Haptics tends to be hardware intensive. You need physical things in the room with you, you probably need to be touching or holding things, or you may need to be wearing something you feel is cumbersome. People aren’t used to buying and investing time and effort into that kind of thing. But, the amount of “stuff” required and the inelegance of having to put things on only matters as far as the experience is valuable. If it’s of value for your work or your social life, you’ll do it. You’ll buy the stuff and you’ll put it on. That’s the threshold we need to cross. I would buy a suit and put it on if it meant that I didn’t have to travel halfway across the planet for short meetings.

But this technology is getting better rapidly. We’re seeing flexible electronics, we’re seeing embedded sensors and actuators in clothing, and it’s possible that, in 10 years, haptified sensor-infused clothing will be no more inconvenient than the clothing that you have today.

Touch is inherently intimate. Are there privacy concerns?

Haptics is something happening to your body, so in a very real way you could be assaulted on the Internet if you have a sufficiently high-fidelity haptic interaction. We’re doing some innovative work around preventing unwanted interaction with other people. How would the permissions for that work? How would they be granted and taken away while preserving the illusion of presence?

Haptics has taken a back seat to other interaction modalities. Is this changing?

Haptics has been a dark horse. I firmly believe that haptics is so central to the human experience that when this stuff is really nailed, the world will be transformed. Touch is a missing element from almost all our digital advances. Haptics has been overlooked, but that’s changing. Microprocessors, displays, and graphics got huge investment and mindshare because they got so good so fast. There was so much money to be made in games and everything else that haptics was neglected.

For ages, end users weren’t asking for better haptics. Now they’re starting to, and some big players are entering the field. Haptics hasn’t typically been a marketing headline for device manufacturers, so it’s been hard to get them to put in the time to really think about the integration of tactile design with industrial and visual design. That’s changing now.

What do you say when someone asks what you do?

I have two 5-year-old boys, and ever since they were babies, when they’d Skype with their grandparents they’d instinctively touch the screen to try to touch their grandparents’ faces. So I explained what I do to my boys by saying, “I’m trying to invent technology that will let you touch people through the screen.” I overheard one of them explaining what I do to his friend last week and it was cool. He said, “My dad’s trying to invent a hole in the screen so you can reach through it.” For kids like them, the problem and the solution are obvious–there’s something in the way and we need to figure out how to get through it. It’s a simple way of stating our mission. We’re inventing a hole in the screen. And there are tons of consequences.

[Download article as a PDF]