Skip to main content

Julia Evans

Day 5: drawing lots of faces with sketch-rnn

Hello! This week it’s generative art week at RC, and I thought it would be fun to generate drawings of faces.

Someone suggested the Google Quickdraw dataset, which has a lot of pictures of faces. And even though I think most of the faces in there are not really that interesting, I really quite like some of them, like these two:

So that seems like somewhere to start!

step 1: get the sketch-rnn model

sketch-rnn is an RNN trained on the quickdraw dataset that generates line drawings. You can see it in action in Monica Dinculescu’s very fun magic sketchpad demo here – you start making a drawing, and it’ll complete you drawing as a cat / bridge / whatever you want.

I figured that would be a fun place to start, and ml5js has a tutorial showing how to write Javascript code to draw things with sketch-rnn, which I followed.

step 2: make the tutorial code use async/await

The ml5js example had a bunch of global variables and callbacks and I found it difficult to work with, so I spent a while refactoring it to use async/await so that I could play around with it more easily. This took a while but I got it to work.

step 3: make sketch-rnn draw lots of faces and put them into a gallery

I started out with making a more interactive website, but decided to instead do something really simple to start: just get the model to draw a lot of faces and see how I felt about them.

Here’s the resulting (very janky, not responsive) website I made, which draws faces and then puts them into a little “gallery” on the right: lots of sketch-rnn faces. You can see it “live” drawing the faces which is fun.

The set of images that comes out looks something like this:

I don’t really like any of these faces, but it’s a start! It’s also very slow on my 2015 laptop, but faster on the iPad. I didn’t spend a lot of time profiling it, but it seems to spend a lot of time in some function with lstm in its name – I don’t know what an LSTM is exactly but I know it’s a component of an RNN, so I guess (as you’d expect) it just has a lot of math to do in Javascript to calculate the next line to draw and that’s slow.

next step: maybe find out if sketch-rnn can tell the difference between “interesting” and “boring” faces

I think that this face:

is a lot more interesting than this face:

Can I convince the neural network to distinguish between faces that I think are ‘interesting’ and ‘boring’ and maybe only generate more “interesting” faces? We’ll see! Right now I am stuck on trying to get a pre-trained model loaded into Python, so there’s a long way to go.

I did find someone who’d done something kind of similar, on bad flamingos vs good flamingos in the quickdraw dataset.

I still don’t really know anything about RNNs, but maybe if I can answer this question I will learn something about them.

also there’s a refrigerator poetry forum

Here’s that refrigerator poetry forum I was talking about last week: https://refrigerator-poetry-forum.herokuapp.com/. You can write magnet poetry-style poems on a refrigerator. People wrote some charming poems and I’m happy with it. I think I might use Rails for another project in the future.

Hopefully people won’t abuse it, if there’s abuse I’ll just take it off the internet probably.

How do you write simple explanations without sounding condescending? Day 8: Start with something that works