New Deep Learning System Allows AI To Solve 'Catastrophic Forgetting' Problem

This article is more than 2 years old.

Reading the news you'd imagine Artificial Intelligence technologies as almighty and unstoppable: after all, they beat human players in ancient Chinese board games, make self-driving cars smarter, under one form or another could soon replace bankers, lawyers and who knows what next.

Yet, as the CEO of Boston-based startup Neurala Massimiliano "Max" Versace would put it, in terms of developing and deploying AIs we're still "technology cavepeople". So far AI works great when it is set to focus on a single task, like forecasting bitcoin fluctuations, but it's less reliable when it has to deal with a number of simultaneous, interwoven factors.

One of the current constraints of artificial intelligence is called "catastrophic forgetting", and researchers have been struggling with it for a while.

In short this means that an AI system needs to forget the skills and knowledge it has learnt in the past, in order to learn new ones.

In other words, to add a single object or a single task, while keeping the same overall amount of information, a neural network would have to be retrained on all of the objects, which is usually done using powerful servers located in the cloud.

Google's Deep Mind researchers have demonstrated recently that this is not a insurmountable challenge. Today Neurala, which recently closed a whooping $14 funding round, will be demoing its own solution at the GPU Technology Conference in San Jose, California.

According to sources from the company, Neurala’s breakthrough solves the “catastrophic forgetting” problem for deep learning neural networks instantly at the computing device, without the need of being connected to a server.

“The ability to learn on the fly and at the edge means that the Neurala approach enables learning directly on the device, without all the drawbacks of cloud learning. In addition, it eliminates network latency, increases real-time performance, and ensures privacy where needed. Most importantly, it will unlock the development of a sea of cloud-less applications," Versace says.

Industrial machines, for instance, could be updated in the field for specific tasks; drones could learn how to identify problems at the tops of cell towers or solar panel arrays, anything from "smart" toys to self-driving cars could be personalized by the owners according to their whims, without having too much to worry about privacy issues (as the data would not be shared externally).

Neurala’s neural network software uses a bio-inspired approach to mimic the way the human brain learns and analyzes its environment. Dubbed the "Neurala Brain" approach, it was recently published as U.S. Patent No. 9,626,566, under the name “Methods and Apparatus for Autonomous Robotic Control.”

Instead of the current stove-piped processing that parses discrete inputs, as Versace describes it in a guest post on Venture Beat, in the future AIs will be able to leverage integrated deep learning frameworks that work simultaneously toward the same goal. Just like the human brain does, when it takes into account multiple and simultaneous factors before making a decision.

The "Brain" was originally developed for the NASA, to help its rovers navigate and investigate other planets autonomously, overcoming constraints of limited processing power, scarce battery life and limited communications. Current customers of the startup include Parrot and Teal Drones.
