Import AI: Issue 2: Microsoft’s AI chips, George Hotz’s bandwidth bill, and Spy! VS Spy!

by Jack Clark


Deep learning’s dirty data secret: modern deep learning approaches require a ridiculous quantity of data. How much data? A hacker house in SF used 3.8 terabytes of bandwidth in July. No coincidence that one of its residents is George Hotz, founder of self-driving car startup The house has now upgraded to an ‘unlimited’ bandwidth plan, making Hotz foot the bill for the bit-flurry.

Free data! released seven and a quarter hours of highway driving data so world+dog can make their own AVs, and François Chollet, creator of the Keras deep learning framework, has made three image classification models available for free. That means you can now classify an image in as little as three lines of code.
( model = VGG16(weights=’imagenet’)
preds = model.predict(imgs)
print(decode_predictions(preds)) ).
How many lines to drive a car badly?

Show me the research: Spotting the difference between an AI startup with genuine technology and one that has re-implemented commodity systems is tricky. It’s easier to judge if the company participates in the wider AI research community. Therefore points should be awarded to robotics company Brain Corporation, text analysis startup Maluuba, and image recognition outfit Curious AI, which have all published papers recently. Keep them coming! (Extra kudos to Curious AI for publishing code.)

Spy VS Spy! Seven autonomous computer systems battled each other at the Darpa Cybersecurity Challenge in Las Vegas last week. ‘Mayhem’, the winner, was built by CMU startup For All Secure. The system performed automated program analysis (via symbolic execution) to find and exploit weaknesses in running programs. It appears to use a smart scheduler to let it run multiple checks on a piece of software in parallel, then selectively pause certain jobs to throw resources at promising vulnerabilities. This lets it efficiently explore the vast underbelly of the program and selectively focus on weak points, like a sophisticated, thieving octopus. (Unfortunately, some of the papers are paywalled.) Podcast with more information here.

Care for a little AI with your global government, sir? Google DeepMind would eventually like to donate its technologies to the United Nations, according to CEO Demis Hassabis.

Deep learning chips: Typical x86 processors are a bad fit for (most) modern AI tasks. GPUs are a bit better, but still not optimal. So expect new hardware substrates for AI. Microsoft will launch a new cloud service within a few months that lets people accelerate their neural network workloads with FPGA co-processors, said Qi Lu at the Scaled Machine Learning Conference. Google has already indicated it will offer its TPUs to people to accelerate TensorFlow workloads. Startup Nervana plans to produce its own chip to accelerate its ‘Neon’ software further. There are also numerous stealth chip and hardware startups that are re-thinking systems around deep learning (think: tens of thousands to millions of processors for low-precision matrix multiplication, etc).

GoodAI, a European AI research group founded by game developer Marek Rosa, has fleshed out its research and development roadmap and begun work on analyzing the overall AI landscape. It’s also developed some software for prototyping AI systems “with highly dynamic neural network topologies” named Arnold. It’s very pretty! The company will release it as open source software in a few months months, Rosa tells me.

Open VS Closed AI development? “Developing a joint set of openness guidelines on the short and long term would be a worthwhile endeavor for the leading AI companies today,” says Victoria Krakovna, co-founder of the Future of Life Institute.

Imagine that, computer. Generative models are gathering interest (as mentioned last week) and have a huge number of potential applications. For more information check out these two slide decks from Russ Salakhutdinov at CMU and Shakir Mohamed from DeepMind, from their sessions at the Deep Learning Summer School this week.

You have questions, they have answers: Google’s Brain team will be taking questions on Reddit and OpenAI’s Ian Goodfellow on Quora. Both on August 11th.
