AAU logo
Gesturing at robots

Gesturing at robots

New technology developed by researchers at Aalborg University makes it possible to control robots and other types of machinery through gestures and arm motions. The invention may completely change the way humans and machines interact.

In a few years from now, exo-skeletons will be far a more common sight than today. The wearable strap-on skeletons give the wearer support and strength to lift and carry items and to move around by supporting the body’s natural movement trough electric motors. Now, a new invention makes it even easier to use exo-skeletons and to control them with great precision.

The invention is called Human Intention Detection Sensor. It was invented by ass. Professor Shaoping Bai and research assistant Muhammad Raza Ul Islam from Aalborg University.

Through an armband with force sensing resistors that is strapped around the lower arm, the system analyses muscle tensions to precisely determine how the arm, hand and digits are being moved and use the information to respond accordingly.

- You only need to flex a muscle a tiny bit to make the system react and perform an action, explains Shaoping Bai. And that is not all. If you need to lift something while wearing an exo-skeleton equipped with a Human Intention Detection Sensor, the system is able to detect the weight of the object and administer a suitable amount of power to carry it.

Ass. Professor Shaoping Bai (left) and research assistant Muhammad Raza Ul Islam holding up their patented Human Intention Detection Sensor, making it possible to control robots through gestures. Photo: Jakob Brodersen

A plethora of uses

In the AAU robotics lab, Shaoping Bai helps Muhammad Raza adjust the sensor band around his arm for a demonstration. The sensors need to be placed on the muscle just below the elbow where muscle impulses can be picked up immediately if you move your hand or your fingers.

To demonstrate the functionality, the two researchers have connected their invention to a remote controlled LEGO robot on wheels. When Muhammad Raza is done calibrating the sensors, the robot reacts by moving forward when he points his hand down flatly. As he turns his palm to either side, the robot turns around to left and right on the small table. When he makes a fist, it immediately stops in its tracks.

- This technology can be applied to many other things than exo-skeletons, says Shaoping Bai. It is a completely new way of interacting with robots. Imagine controlling a huge, hydraulic crane with this type of technology. Instead of having a crane operator pulling levers and switches in a small control room high up in the air, he could control it very precisely by gesturing his hands in the empty air. Also, consider the impact this type of control system would have on computer gaming, he says.

Gesturing at the future

The two inventors have joined up with Professor Ole Madsen and former AAU alumni Charles Xing to form the company BIOX ApS. Here, their aim is to make a commercial success out of the invention.

- There is already a lot of interest from private investors that see the obvious potential in controlling robots through gestures, says Shaoping Bai. If we can attract the investments to keep up the pace we are going at right now, I expect that the first prototypes will be ready within a year or so.

BIOX has signed an exclusive licensing agreement with Aalborg University for commercialization.

With a sensory armband around the lower arm it is possible to steer and control robots simply by using hand gestures. The technology opens a new path for interaction possibilities between humans and technology. Photo collage: Jakob Brodersen


Watch video clip of Shaoping Bai and Muhammad Raza Ul Islam controlling a LEGO robot with gestures using the Human Intention Detection Sensor in the AAU robotics lab:


The new technology was invented partially with support from Innovation Fund Denmark and Aalborg University.


For further information, please contact Shaoping Bai: shb@m-tech.aau.dk