TOM TOM FESTIVAL '18

As a co-sponsor of the Festival’s Applied Machine Learning Summit, our team at Capital One wanted to do more than just talk about machine learning—we sought to create an interactive experience showcasing a real-live example of the technology in an engaging, participatory way, while also highlighting some of the area’s amazing local artists.

A central theme of Tom Tom is “learn to see your world in a new way.” We took inspiration from that ethos to create a pop-up art gallery that would further explore the idea of viewing the world from a new perspective, powered by a machine learning technique called style transfer.

 

Enter our Style Transfer Gallery, where festival participants were invited to have their picture taken, which was then superimposed into the style and technique of a featured artist, and projected among an array of digital screens. This created a real-time, visual tapestry of participants while also preserving a history of previous festival-goers and providing a unique lens into the community.

Under the hood, we used a machine learning technique called “neural style transfer.” This model takes inspiration from what we know about how the human brain processes visual information in order to teach the algorithm how to create images in the style of a particular work of art. Training for a single painting’s style takes hours, but once complete, the model can transform a new image into that style in a fraction of a second. This allowed us to process video collected from a camera in real-time, so participants could see themselves moving and interacting as if they were in the painting, before snapping a self-portrait to be added to the Gallery.

 


RESPONSIBLITY

Ideation with team.

Repurpose a DDR mat into a photo booth trigger. Built the interaction in Python.