AI Case Study
Magenta creates an app which predicts next steps and finishes sketches started by human users
Magenta has created a sketching app called sketch-rnn which uses recurrent neural networks to predict and finish a drawing started by a user according to a subject category.
Industry
Consumer Goods And Services
Entertainment And Sports
Project Overview
According to the blog, the research team at Magenta "made an interactive web experiment that lets you draw together with a recurrent neural network model called sketch-rnn. We taught this neural net to draw by training it on millions of doodles collected from the Quick, Draw! game. Once you start drawing an object, sketch-rnn will come up with many possible ways to continue drawing this object based on where you left off. You can also choose different categories to get the model to draw different objects based on the same incomplete starting sketch, to get the model to draw things like square cats, or circular trucks. You can always interrupt the model and continue working on your drawing inside the area on the left, and have the model continually predict where you left off afterwards. In addition to predicting the rest of an incomplete drawing, sketch-rnn is also able to morph from one drawing to another drawing."
Function
R And D
Product Development
Background
From its blog, "Magenta is a research project exploring the role of machine learning in the process of creating art and music. Primarily this involves developing new deep learning and reinforcement learning algorithms for generating songs, images, drawings, and other materials."
Reported Results
Admittedly, according to the Magenta blog "predicted endings sometimes feel expected, sometimes unexpected and weird, and also can sometimes be hideous and totally wrong." However, in the research paper, a predicted use case includes "pattern designers can apply sketch-rnn to generate a large number of similar, but unique designs for textile or wallpaper prints".
Benefits
Technology
According to the research paper, "Our model is a Sequence-to-Sequence Variational Autoencoder (VAE). Our encoder is a bidirectional RNN that takes in a sketch as an input, and
outputs a latent vector of size Nz. Our decoder is an autoregressive RNN that samples output sketches conditional on a given latent vector z. Our training procedure follows the approach of the Variational Autoencoder, where the loss function is the sum of two terms: the Reconstruction Loss, LR, and the Kullback-Leibler Divergence Loss, LKL. We train our model to optimize this two-part loss function."
Data
According to the research paper: "We constructed QuickDraw, a dataset of vector drawings obtained from Quick, Draw!, an online game where the players are asked to draw objects belonging to a particular object class in less than 20 seconds. QuickDraw consists of hundreds of classes of common objects. Each class of QuickDraw is a dataset of 70K training samples, in addition to 2.5K validation and 2.5K test samples. We use a data format that represents a sketch as a set of pen stroke actions."