AI Flips Kanye West’s Frown Upside Down, Turns Taylor Swift Terrifying

by Jamie Beckett

Tom White doesn’t care if you have anything to smile about this holiday season. The New Zealand researcher’s deep learning smile bot brings joy to the world by putting a happy face on a picture of anyone — naughty or nice.

Thanks to White’s software, a scowling Liam Neeson breaks into a jolly smile more fitting to the Santa suit he’s wearing. A painting of famously frowning artist Frida Kahlo gets into the spirit with a red dress and a big grin. Even rap superstar Kanye West, known for the scarcity of his smiles, gets happy with Smilevector.

A deep learning smiling bot helps Liam Neeson look more like Santa.
No Bad Santa here. With AI, actor Liam Neeson gets jolly for the holiday.

Specializing in celebrity makeovers, White’s Smilevector bot uses deep learning to turn even the stubbornly stone-faced into a gleeful grinner.

The bot works by scraping photos from Twitter accounts for photos or other images of people, then detecting whether the person is smiling or not. If there isn’t a smile, it adds one by adjusting portions of the face, such as crinkling the eyes or filling out the cheeks.

The deep learning bot can easily wipe away that grin, too. It’s designed to switch existing expressions to their opposite, said White, a senior lecturer at Victoria University of Wellington’s School of Design. You can see his creation in action on Twitter @smilevector.

If It’s Automated, Is it Still Creative?

It all started when White, a computer scientist and artist, began to wonder how deep learning would transform creative industries like design. He left his engineering job working on deep learning and returned to academia to research its possibilities.

“Deep learning has implications for people in creative industries,” he said. “If you can automate processes, how does it change how what we think of as creative?”

White created Smilevector as an experiment. He wanted to see how deep learning could be used for a creative task. So he trained his deep learning algorithm on a dataset of about 200,000 celebrity photos using a NVIDIA GeForce GTX 1080 GPU, the CUDA parallel computing platform and the cuDNN-accelerated Theano deep learning framework. (To learn more about how White trained his deep learning smile bot, read his paper on generative networks. He has exhibited his work at numerous museums and has an upcoming exhibit in Savannah, Ga.)

Kristin Stewart lights up in a smile, thanks to the deep learning smile bot.
Kristin Stewart makes most lists of Celebrities Who Never Smile. The AI smile bot changed all that.

Once trained, his algorithm could add or remove smiles, but it needed work. Because men are less likely to smile in photos than women, removing women’s smiles made them look more masculine. At first, Smilevector blurred images in a way that made teeth look bloody.

Smilevector’s Other Powers

After fixes to the software, Smilevector’s transformed photos looked as real as the originals. That’s except for when White experiments with new photo-manipulation models. With them, Smilevector can superimpose . Or create a menacing Taylor Swift by blackening her eyes with too much makeup.

White thinks deep learning will revolutionize design the same way desktop publishing and digital photography did more than two decades ago. That revolution won’t be just for designers. These technologies open up new creative opportunities to anyone, he said.

So smile for your seasonal snapshots. Or Smilevector might do it for you.

To find out more about deep learning, listen to our AI Podcast on iTunes or Google Play Music or read our blog explaining this fast-growing branch of machine learning.

A serious-looking Benjamin Franklin cracks a smile in this portrait, thanks to the work of the AI smile bot.
Like most people posing for portraits, Benjamin Franklin takes on a serious look. But AI shows his lighter side.