Cool Kinect Hack Brings Sight to those with Visual Impairments

Mar 29, 2011

There have been many cool Kinect hacks, but two graduate students at Germany’s Universität Konstanz are working on a hack that may bring sight to people with visual impairments.

The two students, Michael Zöllner and Stephan Huber created a project called NAVI (Navigation Aids for the Visually Impaired). NAVI collects visual data from a Microsoft Kinect infrared camera which is mounted on a helmet and then relays audio instructions through a wireless Bluetooth headset. The 3D imaging capabilities help detect objects that would normally be outside the radius of a white cane and then alerts the wearer to the location of obstacles through audio and vibro-tactile feedback.

In a Human-Computer Interaction blog post, it explains the students wanted to “augment the visually impaired person’s impression of a room or building by providing vibro-tactile feedback that reproduces the room’s layout.”

The post also lists details of the Kinect hack, including the “vibrotactile waistbelt” (pictured above) which uses vibration motors in a belt that are connected via USB “to a laptop that was mounted onto a special backpack-construction, which has holes for cables and fan” (pictured below). The belt vibrates on the right or left when the Kinect camera detects an obstacle, alerting the user to an upcoming object.

The helmet is a hard hat with a Kinect camera mounted on the top. An example of the audio feedback via the helmet and Bluetooth would be if you walked toward a door, you would hear “Door ahead in 3, 2, 1,” then “pull the door” but each part of the relayed information depends on the distance to the door.

The students wrote the software in C# and .NET. They used ManagedOpenNI wrapper for the Kinect and the managed wrapper of the ARToolKitPlus for marker tracking. The voice instructions are synthesized using Microsoft’s Speech API and all input streams are glued together using Reactive Extensions for .NET.

It’s a bit bulky at this point, but it definitely holds promise. Here’s a video to better show off this cool Kinect hack.

Image Credits: Human-Computer Interaction | Usability Engineering | Konstanz

Author: Tessa