New image loaders

[update: Felix Buenemann pointed out that libMagick can be asked to load just a single page of a multi-page document, so I've added this to the git master magickload and updated this post]

git master libvips has just gained three new image loaders. There's a nice performance bump, so I thought I'd write them up as a blog post.

tl;dr

Loading GIF, SVG and PDF images with git master libvips (what will be libvips 8.3) is now faster, sometimes much faster.

Background

libvips has roughly three types of image loader. First, for a few formats, libvips has real code to decode that format into an image. Formats like .vips, .hdr (Radiance), and .csv are handled like this.

Next, libvips has code that links it to a number of image decoding libraries. Formats like .tif, .jpg, and .png are handled in this way. libvips doesn't do the actual decoding, but it does carefully control how the decoding takes place so that it matches the way libvips works internally.

Finally, if all else fails, libvips hands the file over to libMagick, the library version of the ImageMagick package. This can decode perhaps 100 different image formats, so if that doesn't work, libvips just gives up.

The libMagick decoders are very flexible, well-tested and accurate, but they are also very generic and don't exactly fit the way that libvips works. It's therefore possible to get a nice speedup if you write some extra code to move a format from the libMagick-supported group to one of the first two.

GIF

The GIF format was supported via libMagick. The libMagick gif loader loads the gif file as a 16-bit RGBA frame. libvips then converts that to 8-bit RGBA for processing.

libvips now has it's own GIF loader based on giflib. This loader will only read as much of the gif file as it needs to to display the frame you ask for, and does not decode frames to memory, it renders the 8-bit output image directly. This makes it a bit faster and use less memory.

$ time vips magickload spring_map.gif x.png
real    0m0.659s
user    0m0.676s
sys     0m0.064s
peak memory: 70MB

$ time vips copy spring_map.gif x.png
real    0m0.597s
user    0m0.572s
sys     0m0.032s
peak memory: 40MB

PDF

Again, PDF was supported via libMagick. Depending on configuration libMagick has several ways of handling PDF, but the one used by the Debian packages involves copying the source document to a temporary file, invoking a separate ghostscript process to render the PDF to a bitmap file, loading the bitmap and converting to 16-bit RGBA.

With the nip2 manual, for example, you see:

$ time vips magickload nipguide.pdf x.tif
real    0m0.211s
user    0m0.196s
sys    0m0.028s

libvips now has it's own PDF loader built using the libpoppler library. This all runs in-process, and renders the page in chunks to keep memory use down.

$ time vips copy nipguide.pdf x.tif
real    0m0.053s
user    0m0.048s
sys    0m0.008s

The new libpoppler loader has a few disadvantages. First, it can only make sRGB images, so if your PDF makes CMYK, you'll get sRGB anyway. Secondly, libpoppler is licensed under the GPL, so if you use this loader, your whole program is also GPL.

There's another case where the new loader won't be a good choice. To keep memory use low, it renders PDF images in chunks of (usually) 128 scan lines. If your PDF image has embedded objects which span several chunks, they will be decoded once for each chunk they touch. The worst case is a JPEG file that covers the whole page: it'll be decoded many, many times.

SVG

Like the other formats here, libvips used to load SVG via libMagick. Again, for this format, in the Debian configuration, libMagick uses a command-line program and a set of temporary files, it's not very quick:

$ time vips magickload vips-profile.svg x.png
real    0m0.706s
user    0m0.648s
sys    0m0.052s

libvips now has an SVG loader based on librsvg. This runs in-process and avoids all of the copying.

$ time vips copy vips-profile.svg x.png
real    0m0.125s
user    0m0.116s
sys    0m0.008s

vipsthumbnail

vipsthumbnail now knows about the PDF and SVG loader. When it makes a thumbnail, rather than rendering a high-resolution bitmap and then shrinking it, it just asks the loader to render an image of the correct size immediately. Combined with the new PDF and SVG loaders, this produces a spectacular speedup.

With libvips 8.2, asking for a thumbnail of the nip2 manual took a while:

$ time vipsthumbnail nipguide.pdf
real    0m9.212s
user    0m9.068s
sys    0m0.216s

Now it's pretty much instant.

$ time vipsthumbnail nipguide.pdf
real    0m0.047s
user    0m0.040s
sys    0m0.008s

A 200x speedup!