trim (auto crop) with ruby-vips

[update: here's the program in vips8 Python as well]

Someone asked how to do an equivalent of ImageMagick's -trim function in ruby-vips. It seems like a common request, so I thought I'd write it up as a blog post.

-trim is an auto-cropper. It searches in from the edge of an image and crops off the boring bits: the pixels which are equal to the edge. There's a section on the ImageMagick website introducing the command.

Here it is in operation: on the left is a (small) version of the NASA marble earth image, on the right the same image after ImageMagick's -trim command:

convert tiny_marble.jpg -trim +repage cropped.jpg 

As you can see, it's cut away a lot of black from the image. It hasn't cropped very closely though, you can still see a black border at the edges. If you zoom in you'll discover why: -trim will stop at the smallest difference from the edge, and jpeg compression introduces enough noise to throw it off.

To trim a noisy or compressed image you need to stop at a significant difference from background. One way to do this is to blur the image and trim when the blur goes above a threshold. The page on the ImageMagick site above suggests this command:

convert tiny_marble.jpg -crop \
    `convert tiny_marble.jpg -virtual-pixel edge -blur 0x15 -fuzz 15% -trim \
        -format '%wx%h%O' info:` \
    +repage cropped.jpg

Which is much better, though it's still stopping a little early on the left, I'm not sure why.

The simplest way to do this in ruby-vips is with the .project operator. This scans an image, building an array of column sums and an array of row sums. We can then look along the list of column sums and the index of the first non-zero entry will be how many pixels we can cut from the left edge of the image.

That will obviously only work for images with a black background, so we get the pixel at (0, 0), put the whole image through a 3x3 median filter to remove 1-pixel noise, subtract the background, take the absolute value, and threshold.

Here's a complete ruby-vips script to do auto-trim. I've added a lot of comments, hopefully it's easy to read. Here it is running on the marble image:

It's cropped very exactly (just as well, after all that work).

Performance is good too. If I try cropping the 8000x8000 pixel NASA image in ImageMagick on my laptop I see:

$ time convert marble.jpg -crop `convert marble.jpg -virtual-pixel edge -blur 0x15 -fuzz 15% -trim -format '%wx%h%O' info:` +repage cropped.jpg
real    0m40.150s
user    2m30.959s
sys     0m0.775s 
peak mem: 979m 

And for ruby-vips I see:

$ time ~/try/trim3.rb marble.jpg cropped.jpg
real    0m6.233s
user    0m20.006s
sys     0m0.475s
peak mem: 58m

Though the peak-mem figure is a little unfair, ruby-vips is creating a 200mb temporary file during processing.

Plain -trim is much faster again:

$ time convert marble.jpg -fuzz 1% -trim +repage cropped.jpg
real    0m2.036s
user    0m2.595s
sys     0m0.300s
peak mem: 942m

But that won't work so well with noisy images.