Machine Learning is Sneaking Up On You

Posted on March 13, 2011 by


ML lurks in consumer products

Folks are sometimes skeptical of my methods when I tell them our fund management tools use Machine Learning.  The term sounds futuristic and is perhaps a little bit scary.  But ML is not hard to understand, and  most people are not aware of the growing use of ML in everyday products.

read related articles

I was reminded of this today when Irfan Essa, a colleague of mine at Georgia Tech, shared a link to a technical paper published by researchers at Microsoft.  The paper describes how the Xbox uses Machine Learning to estimate the pose of people standing in front of a kinect sensor.

The kinect feeds a 3D “depth image” to the Xbox, where the pose estimation algorithms take over. When the Xbox estimates “pose” that means that it knows the location of a person in front of it and also the location and orientation of all their major joints and appendages.  With that information an Xbox user can control the actions of a character in a video game (their “avatar”).

Before this development, when researchers wanted to estimate the location of people (and their arms and legs) from a depth image, they would use software to try and match the orientation of a model (a kind of stick figure) to the orientation of the person in the image. That worked OK, but it was slow, and it could fail if the person in the image was built differently from the model — for instance if their arms and legs were longer or shorter than the model’s.

To get around these problems the Microsoft team used Machine Learning to train their system.  They used millions of example depth images paired with known poses of people as to “train” their system. After the training, the software can accurately figure out the poses of fat people, skinny people, and short and tall people too; all at real time rates on an Xbox.

What is Machine Learning?

The many types of Machine Learning can be distinguished by kinds of problem they solve and the flavor of algorithm they use to solve it.  I’m primarily interested in supervised machine learning — this is also the flavor the Xbox uses, as well as what works well for may equity price prediction strategies.

The Xbox uses an ML algorithm called “decision trees.”

The idea behind supervised learning is simple: You provide the computer with example input/output pairs and it “memorizes” these examples. In the case of the Xbox for instance, the input is a depth image and the output is a pose.  In the case of equity price prediction, the input might be measurable features about the company, and the output is a predicted price.

In general, the more examples you feed the computer, the more accurately the predictor works.

There are many algorithms that can use supervised learning, including linear regression, KNN, decision trees, and groups of decision trees, called decision forests. Microsoft used decision forests for their Xbox algorithm, and it is also my preferred approach as well.

Other examples

Many consumer cameras use ML-based face detection software.

I’ll finish up with two other examples of ML in consumer products. The first is face detection: Over 70% of the photos people take have a person’s face as the main subject. So engineers at Nikon, Sony, and others have put software in their cameras that is able to automatically find people’s faces in the image and then use that information to adjust exposure focus. Here’s a webpage that describes the underlying algorithms well.

Another example, again from consumer photography, is the “matrix metering” in Nikon cameras, and “evaluative metering” in Canon cameras. When a camera is in automatic mode it uses one of these algorithms to set shutter speed and aperture appropriately. The manufacturers claim that the exposure setting decisions draw from thousands of example images taken by professional photographers.

What is happening under the hood here, is that the camera treats these thousands of images by the professionals like a library, and it tries to find the closest match.  It then uses the exposure setting from that example to set the exposure for your picture.

There are many other examples of ML in consumer products (one other comes to mind: Tivo’s and Netflix’s algorithms that recommend TV programs and movies). Expect to see these algorithms sneaking into more and more products going forward.