A Lightweight iOS Image Cache

seatgeek open sourced seatgeek/SGImageCache

A flexible image caching library for image rich iOS applications

Our iOS app is image rich. To create appealing views we rely heavily on performer images, all of which must first be fetched from a remote server. If each image needed to be fetched from the server again every time you opened the app, the experience wouldn’t be great, so local caching of remote images is a must.

Version 1 - Ask for an image, get it from disk

Our first image cache was simple but effective. For each image view we’d ask for an image from cache, using its remote URL as the cache key. If it was available in the local disk cache a UIImage would be created from the file on disk, and returned immediately. If it wasn’t found on disk it would be fetched async from the remote URL, cached to disk, then a new UIImage returned.

For our purposes at the time this was perfectly adequate. But it had one point of unnecessary weakness: each cache request required the image to be loaded again from disk, which comes with the performance cost of disk access and image data decoding.

Version 2 - Memory caching

Thankfully Apple’s UIImage has a built in memory cache. So by changing a single line of code our image cache could go from being a disk only cache to a disk and memory cache.

When you ask UIImage for an image via imageNamed: it first checks its own memory cache to see if the image has been loaded recently. If so, you get a new UIImage at zero cost. So instead of something like this:

return [UIImage imageWithContentsOfFile:[self absolutePathForURL:url]];

We could get memory caching for free, simply by doing this:

return [UIImage imageNamed:[self relativePathForURL:url]];

UIImage will search its memory cache and, if found, return the image at no cost. If it isn’t in the memory cache it will be loaded from disk, with the usual performance penalty.

Version 3 - Fetch queues, prefetching, and variable urgency

As the design of our app evolved we became increasingly image greedy, wanting to show richer, larger images, and more of them.

Getting these larger images on screen as quickly as possible is critical to the experience, and simply asking the cache for each image at display time wasn’t going to cut it. Larger images take longer to load over the network, and asking for too many at once will result in none of them loading until it’s too late. Careful consideration of when the image cache is checked and when images are fetched from remote was needed. We wanted precaching and fetch queues.

fastQueue and slowQueue

We settled on two queues, one serial and one parallel. Images that are required on screen urgently go into the parallel queue (fastQueue), and images that we’ll probably need later go into the serial queue (slowQueue).

In terms of a UITableView implementation, this means that a table cell appearing on screen asks for its image from fastQueue, and every off screen row’s image is prefetched by adding it to slowQueue.

We’ll need it later

Assuming we request a page of 30 new events from the server, once those results arrive we can queue up prefetching for each of their images.

- (void)pageLoaded:(NSArray *)newEvents {
    for (SGEvent *event in newEvents) {
        [SGImageCache slowGetImageForURL:event.imageURL thenDo:nil];
    }
}

The slowGetImageForURL: method adds the image fetch to slowQueue, allowing them to be fetched one by one, without bogging down the network.

The thenDo: completion block is empty in this case because we don’t need to do anything with the image yet. All we want is to make sure it’s in the local disk cache, ready for immediate use once its table cell scrolls onto screen.

We need it now

Cells that are appearing on screen want their images immediately. So in the table cell subclass:

- (void)setEvent:(SGEvent *)event {
    __weak SGEventCell *me = self;
    [SGImageCache getImageForURL:event.imageURL thenDo:^(UIImage *image) {
        me.imageView.image = image;
    }];
}

The getImageForURL: method adds the image fetch to fastQueue, which means it will be done in parallel, as soon as iOS allows. If the image was already in slowQueue it will be moved to fastQueue, to avoid wasteful duplicate requests.

Always async

But wait, isn’t getImageForURL: an async method? If you know the image is already in cache, don’t you want to use it immediately, on the main thread? Turns out the intuitive answer to that is wrong.

Loading images from disk is expensive, and so is image decompression. Table cells are configured and added while the user is scrolling the table, and the last thing you want to do while scrolling is risk blocking the main thread. Stutters will happen.

Using getImageForURL: takes the disk loading off the main thread, so that when the thenDo: block fires it has a UIImage instance all ready to go, without risk of scroll stutters. If the image was already in the local cache then the completion block will fire on the next run cycle, and the user won’t notice the difference. What they will notice is that scrolling didn’t stutter.

Thought we needed it but now we don’t

If the user scrolls quickly down a table, tens or hundreds of cells will appear on screen, ask for an image from fastQueue, then disappear off screen. Suddenly the parallel queue is flooding the network with requests for images that are no longer needed. When the user finally stops scrolling, the cells that settle into view will have their image requests backed up behind tens of other non urgent requests and the network will be choked. The user will be staring at a screen full of placeholders while the cache diligently fetches a backlog of images that no one is looking at.

This is where moveTaskToSlowQueueForURL: comes in.

// a table cell is going off screen
- (void)tableView:(UITableView *)table
        didEndDisplayingCell:(UITableViewCell *)cell
        forRowAtIndexPath:(NSIndexPath*)indexPath {

    // we don't need it right now, so move it to the slow queue         
    [SGImageCache moveTaskToSlowQueueForURL:[[(id)cell event] imageURL]];
}

This ensures that the only fetch tasks on fastQueue are ones that genuinely need to be fast. Anything that was urgent but now isn’t gets moved to slowQueue.

Priorities and Options

There are already quite a few iOS image cache libraries out there. Some of them are highly technical and many of them offer a range of flexible features. Ours is neither highly technical nor does it have many features. For our uses we had three basic priorities:

Priority 1: The best possible frame rate

Many libraries focus heavily on this, with some employing highly custom and complex approaches, though benchmarks don’t show conclusively that the efforts have paid off. We’ve found that getting the best frame rates is all about:

Moving disk access (and almost everything else) off the main thread.
Using UIImage’s memory cache to avoid unnecessary disk access and decompression.

Priority 2: Getting the most vital images on screen first

Most libraries consider queue management to be someone else’s concern. For our app it’s almost the most important detail.

Getting the right images on screen at the right time boils down to a simple question: “Do I need it now or later?” Images that are needed right now get loaded in parallel, and everything else is added to the serial queue. Anything that was urgent but now isn’t gets shunted from fastQueue to slowQueue. And while fastQueue is active, slowQueue is suspended.

This gives urgently required images exclusive access to the network, while also ensuring that when a non urgent image later becomes urgently needed, it’s already in the cache, ready to go.

Priority 3: An API that’s as simple as possible

Most libraries get this right. Many provide UIImageView categories for hiding away the gritty details, and most make the process of fetching an image as painless as possible. For our library we settled on three main methods, for the three things we’re regularly doing:

Get an image urgently

__weak SGEventCell *me = self;
[SGImageCache getImageForURL:event.imageURL thenDo:^(UIImage *image) {
    me.imageView.image = image;
}];

Queue a fetch for an image that we’ll need later

[SGImageCache slowGetImageForURL:event.imageURL thenDo:nil];

Inform the cache that an urgent image fetch is no longer urgent

[SGImageCache moveTaskToSlowQueueForURL:event.imageURL];

Conclusion

By focusing on prefetching, queue management, moving expensive tasks off the main thread, and relying on UIImage’s built in memory cache, we’ve managed to get great results in a simple package.

Code, Design, and Growth at SeatGeek

Jobs at SeatGeek