Why 90% of machine learning models never hit the market

Share on facebook
Share on google
Share on twitter
Share on linkedin

Corporations are going through rough times. And I’m not talking about the pandemic and the stock market volatility.

The times are uncertain, and having to make customer experiences more and more seamless and immersive isn’t taking off any of the pressure on companies. In that light, it’s understandable that they’re pouring billions of dollars into the development of machine learning models to improve their products.

But there’s a problem. Companies can’t just throw money at data scientists and machine learning engineers, and hope that magic happens.

The data speaks for itself. As VentureBeat reported last year, around 90 percent of machine learning models never make it into production. In other words, only one in ten of a data scientist’s workdays actually end up producing something useful for the company.

Even though 9 out of 10 tech executives believe that AI will be at the center of the next technological revolution, its adoption and deployment leave room for growth. And the data scientists aren’t the ones to blame.

Corporations aren’t set up for machine learning

Leadership support means more than money

The job market for data scientists is pretty great. Companies are hiring, and they’re ready to pay a good salary, too.

Of course, managers and corporate leaders expect from these data scientists that they add a lot of value in return. For the moment, however, they’re not making it easy to do so.

“Sometimes people think, all I need to do is throw money at a problem or put a technology in, and success comes out the other end,” says Chris Chapo, SVP of data and analytics at GAP.

To help data scientists excel in their roles, leaders don’t only need to direct resources in the right direction, but also understand what machine learning models are all about. One possible solution is that leaders get some introductory training to data science themselves, so they can put this knowledge into practice at their companies.

Lacking access to data

Companies aren’t bad at collecting data. However, many companies are highly siloed, which means that each department has its own ways of collecting data, preferred formats, places where they store it, and security and privacy preferences.

Data scientists, on the other hand, often need data from several departments. Siloing makes it harder to clean and process that data. Moreover, many data scientists complain that they can’t even obtain the data they need. But how should you even start training a model if you don’t have the necessary data?

Siloed company structures — and inaccessible data — might have been manageable in the past. But in an era where technological transformation is happening at breakneck speed, companies will need to step up and set up uniform data structures throughout.