Can You Feel Me Now? The Sensational Rise of Haptic Interfaces

Your first experience with haptics was probably your phone vibrating in your pocket. Or maybe it was the rumble pack on your N64 controller. But whatever the case, you probably didn’t know it as a haptic interface.

Haptics is to touch the way optics is to sight. It's a user interface that circumvents the cluttered inputs of sight and sound, and it's appearing in an increasing number of objects we interact with daily. Vibration is just the beginning.

Any sort of information received through touch is haptic; braille could be considered haptic communication. But as it appears in technology, it's generally either tactile (expressing texture) or kinesthetic (expressing force or position). Haptics is used to better robotic control, to increase realism in gaming, and even to sit up straighter.

The roots of haptic technology are mechanical, says Will Provancher, a professor of mechanical engineering at the University of Utah and co-chair of the World Haptics Technical Committee.

"Right around the time of WWII, people were trying to handle radioactive materials, and if you have direct contact with these materials, you will eventually die," he says. "So to be able to handle these materials safely, people started making kinematic linkages."

That is, scientists and engineers used a mechanical apparatus to manipulate the samples — pull, and it pulls, turn, and it turns. But more recently, computers have become an interface between controller (master) and controlled (slave). Motor control is much finer, but that's not always enough.

Haptics in Robotics

Take robot surgery, for example. It's difficult to respond accurately to solely visual cues, and when you're wielding a cutting tool, it's important to be able to tell how much pressure you're applying to whatever you're cutting, lest you slice right through it (and anything else in the way).

"People believe that by adding some tactile, or kinesthetic, just generally haptic feedback, there might be some potential advantages for improved surgical outcomes," says Provancher.

That's one path Kinea Design is trying to forge, with haptic tactors that provide feedback based on signals from artificial fingers. The company's research is targeted more at prosthetics — it's involved in DARPA's Revolutionizing Prosthetics program — but much of the feedback technology accomplishes similar functions. Sensors on the prosthetic's fingertips relay pressure, texture, and friction information back to a mount that artificially recreates it in the nerve endings at the juncture to the body.

Photo: Courtesy of HDT Global, which acquired Kinea in 2011

Phone Feedback

It was relatively simple to introduce haptics into phones. Actuators are cheap, especially relative to the cost of a smartphone, and they're regularly in contact with our hands or bodies, so there's a direct link. But the ways haptics can be integrated are expanding, first from vibrating ringers to buzzing on a key-press.

As haptic outputs get more precise, says Provancher, they'll be able to impart still more information in the form of vibrotactile signatures. And you'll be able to feel it, just as the vibrations from a tuba feel different from those from a car engine, or the bass in a movie theater.

"So you could make different waveforms that felt much more different from each other," he says. "They’re calling this area HD haptics."

The oscillations of a vibrotactile device can be described by a sine wave. They've got an amplitude and a frequency, and varying the speed of a vibrating motor changes them. But incorporating more nuanced vibrators could allow devices to vary amplitude and frequency independently, giving them a much wider range of possible signatures — enough even for every contact in your phone book.

But that's not the only way to expand haptic technology in phones. Tactus Technology is working on a dynamic smartphone screen that features physical buttons that rise up out of the screen when needed.

The idea is to blend the qualities of large-screen phones with the tactile interface of keyboard phones, like BlackBerries, and Tactus says it could replace haptic key-press buzzers with buttons.

Tactus uses a microfluidic technology that pressurizes a fluid in the buttons based on a signal from the device. The fluid expands through tiny holes beneath the screen, and pushes a small bulb out over a predetermined location.

Currently, it exists as an additional layer, set over a touchscreen, that raises the buttons about a millimeter, but it could also be integrated into new phones and tablets, or even non-screen locations such as car doors or keypads.

Photo: Courtesy of Tactus Technology

Videogames

Haptic connections aren't just limited to reproducing reality; they can give important information about virtual situations as well. That's what Disney Research is developing with Surround Haptics. By introducing vibration to gaming environments, Disney is trying to increase the realism, calling it an "immersive tactile experience."

This project goes well beyond rumble packs. By integrating it into seats, gloves, vests, and shoes, Surround Haptics has been able to reproduce road conditions in driving games, but also represent skidding, acceleration, and collisions.

It works by drawing patterns of vibrations on the user's back; instead of feeling two distinct vibrating points, the brain perceives the input as one vibrating point, equidistant from the real ones. A network of points used this way can impart many different sensations, associated with different gameplay actions.

The fruits of the research are available in an Avengers branded gaming pad that sells for $100, but as the seat relies only on audio input — it simply matches vibrations to the game's sound — it's more of a glorified rumble pack than an immersive tactile experience.

Photo: Courtesy of Marvel

Safety and Notifications

Cadillac has spent much of its history trying to eliminate vibrations from the road. But with several of the 2013 line, Caddy is also rumbling seats.

The Safety Alert Seat is linked to sensors on the car's outside. If there's an impending collision, the car will notify you via your backside. Cadillac says this is to cut through all the other sensory inputs — i.e., sight and sound — vying for your attention.

Cadillac ATS Photo: Courtesy of General Motors

Interpersonal Communication, or How to Be Suave

Somewhere between the Surround Haptics research, with it's network of vibrating nodes, and the Cadillac, with its left/right directional notifications, is the RISR body language training shirt.

Basically, the RISR uses an iPhone app to read the wearer's posture and give feedback in the form of vibrations. Furthermore, it uses an infrared camera on the front to monitor social cues from a date, and helps the wearer position him or herself in response. Or at least that's what it would do, if it were real.

Unfortunately for the awkward, it's not available; it's a fictional product based on a design-school project.

Photo: Courtesy of RISR

Passive Haptics and Balance

Unlike the feedback-based, interactive vibrations from Surround Haptics, Cadillac, and RISR, the vibrating insoles Jim Collins is working on don't impart information to the user — at least not consciously.

Collins — a bioengineer at Boston University and the Wyss Institute at Harvard — found that random vibrations introduced in the feet of participants helped them sway less, or keep their balance better.

"It's not a feedback-based system," says Collins, who has been studying the effect for almost 20 years. "What we're doing is introducing a bias signal, to their sensory neurons, which is basically serving as a pedestal or booster for the signals they normally would detect."

That is, small vibrations in the feet cause heightened nerve sensitivity, and thus users — including stroke victims, diabetics, the elderly, but also the young and fit — detect signals they would normally miss.

The insoles aren't on the market yet, but Collins envisions additional future applications in sports equipment, like ski boots and golf shoes.

Photo: Courtesy of the Wyss Institute

Texture

Another arm of Disney Research — called REVEL — has been exploring reverse electrovibration as a way to project texture onto smooth objects.

Electrovibration is the practice of running an alternating current through an object to give it texture: a finger drawn across it will alternately be attracted and released, almost imperceptibly, and the result will feel like a rubbery friction.

Disney's project, headed by Olivier Bau and Ivan Poupyrev, runs the current through the user, instead of the object. That way, in combination with a sensor, a computer system knows what they're interacting with and can alter the item's texture based on a predetermined condition.

As it currently exists, the technology depends on a lot of conditions: the object must include conductive and insulating layers; an apparatus has to be connected to the body to transmit the signal; an external camera or sensor must provide locational context; and both objects must connect to a common ground.

But the core concept, that you can run a signal through your finger based on what's observed in your proximity, and project tactile information on it, could be useful in entertainment, education, and even security. Variations in the current (which remains lower than the threshold for human sensation) can change the perception of the environment.

More Notifications: Turn Signals

Makers and hackers are exploring haptics too: this student project installed vibrating actuators in bike helmets (though handlebars could work also) to notify riders when a turn is approaching. The system uses an Arduino to run a signaling algorithm, which tells the actuators when to vibrate based on a Bluetooth connection to an Android device running a GPS and a custom app.

Involuntary Contractions

Electrodes placed on the forearm can cause (painless) involuntary muscle contractions. A user holding a phone — say, for a video game — would tilt that phone to one side or the other.

Pedro Lopes and Patrick Baudisch of the Hasso-Plattner-Institut, an engineering and design school in Potsdam, Germany, built a prototype "force feedback" device that uses electrodes (and the body's own muscles) instead of a mechanical actuator to give haptic feedback. If a windmill blows your digital plane one way, your (arm will cause the) device to tilt, and you'll have to respond, forcing the other arm back up.

Photo: Courtesy of Hasso-Plattner-Institut

Education, Motility, and Robot-Assisted Driving

Infants and toddlers develop socialization skills in conjunction with motility skills, says University of Delaware Professor of Physical Therapy Cole Galloway. But for special needs children, those skills are often learned disparately, out of context.

Galloway and his colleagues Sunil Agrawal (pictured with Galloway), Xi Chen, and Christina Ragonesi discovered that the hard way, while experimenting with smaller mobility carts — around 25 pounds, compared to a typical 250-pound power chair — for infants and toddlers.

The team used haptic joysticks to teach special needs kids from six months to four years old to navigate an obstacle course while mounted on the carts. Sonar and laser guides mounted on the carts calculate a dynamically-changing open path. Then the joystick offers slight physical resistance if the kid pushes it toward an obstacle instead of the clear path.

"We can teach you to be mobile, and the robotics can teach you to be social," says Galloway. "You're driving. But there's a soft, probabilistic push."

Though the kids learned to zig-zag around mats in a gymnasium, they refused to drive the carts when placed in a social setting. So the researchers used the same technology to encourage the kids to interact with each other, gently nudging the drivers toward a group of kids playing ball.

"What we're seeing is, that toddlers who wouldn't necessarily go over and play ball with kids, we can turn on the robotics and it sort of softly gets them over," says Galloway. "And then when we turn it off, they go over by themselves."