Part of what makes SeatGeek amazing are the performer images which help turn it from something I would make as a developer (a database frontend) into a beautiful site. These previously required a lot of man power and a lot of time to collect and resize so we’ve recently created a new process using OpenCV face detection to automatically crop our images.

We use these images in our iPhone App, Explore pages as well as our performer pages throughout the site:

Old and Busted Way

Collection of images would take up to a month any time we wanted to add a new size. Outsourcers would be commissioned to collect 5000 images of a certain size. A simple step but expensive, inflexible, and time consuming.

New Hotness OpenCV!

OpenCV does the hard work of finding shapes that resemble a human face and returning the coordinates. It includes a python version but there are other libraries that wrap it up and make it a bit easier to work with.

Since we don’t have to do the hard work of finding faces why don’t we just get really big images and automate the process of making smaller ones!

1
2
3
4
5
6
7
8
9

def faces_from_pil_image(pil_image):
    "Return a list of (x,y,h,w) tuples for faces detected in the PIL image"
    storage = cv.CreateMemStorage(0)
    facial_features = cv.Load('haarcascade_frontalface_alt.xml', storage=storage)
    cv_im = cv.CreateImageHeader(pil_image.size, cv.IPL_DEPTH_8U, 3)
    cv.SetData(cv_im, pil_image.tostring())
    faces = cv.HaarDetectObjects(cv_im, facial_features, storage)
    # faces includes a `neighbors` field that we aren't going to use here
    return [f[0] for f in faces]

The haarcascade_frontalface_alt.xml file contains the results of training the classifier, which you can find as part of the OpenCV library or with a quick search online.

Starting with this picture of Eisley:

we can use PIL to draw rectangles around the faces that OpenCV found:

1
2
3
4
5
6
7
8
9
10

def draw_faces(image_, faces):
    "Draw a rectangle around each face discovered"
    image = image_.copy()
    drawable = ImageDraw.Draw(image)

    for x, y, w, h in faces:
        absolute_coords = (x, y, x + w, y + h)
        drawable.rectangle(absolute_coords)

    return image

How to resize once we can find faces

Once we have a face we need to resize the image. In our case the images we collected are landscape format images and we use landscape images for our larger sizes. Staying in the format makes resizing a bit easier, we mostly make thinner images, a higher aspect ratio, so we can just resize to the correct width and crop it into a rectangle with the correct height that we want.

1
2
3

face_buffer = 0.5 * (target_height - all_faces_height)
top_of_crop = top_of_faces - face_buffer
coords = (0, top_of_crop, target_width, top_of_crop + target_height)

The face_buffer is the amount of space we want to leave above the top-most face after finding the height from the top of the top face to the bottom of the bottom face to make sure we aren’t cropping anyone out of the photo.

Generally we want to include as much of the image as possible without cropping out anyones face so this works reasonably well for images where the final size is a higher aspect ratio then when you started. Now that you have the faces though you can use any sort of cropping that you need.

Installing into a Virtualenv

If you are installing this on your Ubuntu machine then the system packages should include everything you need. If you use virtualenvs you are going to run in to some issues. With virtualenvs I’ve found the steps for installing Simple CV to be incredibly helpful as a starting point.

Learnings

This was originally setup for some small images used in a few places around the site and would resize on-the-fly if the CDN didn’t have the image cached. Live resizing works for smaller images reasonably well, sometimes it would just take a couple of seconds to load an image, not ideal, but not horrible. As the images grew in size the face detection and resizing would take up to 20 seconds, safely a deal-breaker.

Resizing the width first and only cropping the height is an easy first step if the final aspect ratio will be greater. That will likely become an issue when other people find out they can get all of the images in whatever size they want. If you have to make images small enough that you can’t fit all of the faces into the image then you will really need to make something more intelligent.

We actually use ImageMagick instead of PIL since the service this is part of was already using it. ImageMagick is rather quirky and can sometimes ignore your commands without any mention of why.

3rd party services exist that can do this for you as well. With a little development work to integrate the service it is still cheaper than hiring someone to resize all of the images and still significantly faster. If you don’t want to pay for external hosting you can easily store them on your servers or S3.

A full example can bee seen as a gist. If you want to use these images and more to code some great interfaces we’re hiring frontend developers and more!