Another six months has rolled past so there's another new version of vips. Here's what's new in headlines, see the
libvips ChangeLog and
nip2 ChangeLog for more detail. Sources, plus binaries for Windows and OS X are
here, homebrew should update soon.
Better file format support
jpegsave has four new options. The first lets you save progressive JPEG images. These are slower to decode, but arguably better for web browsers, since users can see the image arriving sooner. There's another flag to turn on coding optimization. This produces slightly smaller files at the cost of slightly slower writing. Another option disables chrominance subsampling, useful at very high Q values, and a final option strips metadata from images, useful to reduce file size again.
The
Radiance read and write operations have been rewritten and are about three times faster and use a lot less memory.
Better vipsthumbnail
vipsthumbnail has a couple of new options. It now supports linear light image shrinking, which can give better results in some cases, and also supports non-square bounding boxes. There has been
a blog post about these improvements.
XYZ PCS
li
bvips now supports the use of XYZ as a profile connection space.
Built-in profiling
libvips now has a built-in profiling system. Run with --vips-profile to dump timing information to a file, use the new vipsprofile command to calculate some statistics and draw timing and memory use graphs. There has been
a blog post about this new feature.
Optimization
The new profiler has been used to tune the memory and threading systems. Memory use is halved in some cases, with less allocate / free cycling. We've also marked up most of the inner loops for compiler auto-vectorizers. This can give a useful speed up: compile with gcc -O3, for example.
libvips rewrite continues
We are slowly continuing the complete rewrite for vips8. This time it was convolution, frequency-domain filtering and morphology. There's only the mosaicing functions left to do now!
Thanks for the hard work! Can vips thumbnail generate rounded corners?
ReplyDeleteNot directly, no, you'd need to write an extra thing using ruby-vips or whatever. Or do it in CSS I guess.
DeleteHello. What do you do, if you need separate one image to much many smaller, and stick together they then?
ReplyDeleteI faced with 2 problems - stack and memory.
Problem with the stack I decided to limit the number of recursive calls, such, for example:
[code]
foo start finish
= 0, start > finish
= 1 + foo (a - 1)
bar start finish
= 0, start > finish
= (foo start (min_pair finish (start + 300)) + bar (start + 300) finish
[/code]
its example, i dont test it now. Its good decision? - where i can see tail recursion optimisation? ^)
But i cant solve memory problem.
1) I separate image to 300000 small parts by extract area
2) I create new image by:
backgr = image_new Img.width Img.height Img.bands Img.format Img.coding Img.type 255 255 255;
// Img - is source image
3) And I use insert_noexpand to create new image.
I test similar variants...for example use join_tb and make it without lists (i think, that lists can save image, so garbage collector dont working about them). But i cant solve this problem.
Give me small example please.
I hope that the community will be supportive to me thanks to an article.
Link to small article: http://pro-prof.com/archives/1364
Hi, I enjoyed your article, thank you for writing it.
DeleteI think nip2 will not scale to tasks that large, unfortunately. I use it for workspaces with up to about 10,000 large images and it works OK, but 300,000 is another order of magnitude and I don't think it will do it gracefully.
It has two menu items that do what you want: Image / Tile / Chop Into Tiles will cut an image into non-overlapping tiles with a size you choose. They are returned as an array. This is very quick, thanks to lazy evaluation.
Use Image / Join / Array to join the array back into a single image again. This is quick for 100 tiles, slow but OK for 1,000 tiles, not useful for 10,000. The array join in nip2 is quite fancy: you can give alignment and margins for each join, and it doesn't use the fastest path. You could make an array join that was perhaps 10 times faster, but it probably still wouldn't do your 300,000 problem.
vips comes with a command-line tool which can do the chop-into-tiles half of your problem. Try:
vips dzsave hugeimage.tif outputdir --layout google --depth 1 --size 300 --suffix .v
This will chop your image into 300x300 tiles and save each tile as a vips image inside outputdir. It's very fast and will work on images of any size.
To join, you need to write a program. I would use Ruby, I think. You'll have to join the tiles into a set of strips with left-right joins and save them to disc, then in a second pass, load the strips again and join them into the final image with a set of top-bottom joins.
We use the issue tracker on github for questions and bug reports, that might be a better places to ask any more questions you have:
https://github.com/jcupitt/nip2/issues?state=open
On tail recursion: nip2 uses graph reduction of a variant of Turner's combinators to evaluate programs (he was my PhD supervisor). Eliminating tail recursion is probably best done by switching to a pointer reversing reduction engine. This doesn't actually remove tail recursion, but it removes the stack, which is almost the same thing.
I guess you found list comprehensions and pattern matching too? They need a bit of work still, the parser has to be rewritten to support them fully. I'm rewriting libvips at the moment, I plan to work on nip2 again when that's done.
John
Oop, of course that command should be:
Deletevips dzsave hugeimage.tif outputdir --layout google --depth 1 --tile-size 300 --suffix .v
hm...
DeleteI thought that by "lazy evaluation" reason, image is not divided into parts before I did not try try to display the result of combining these parts.
I think, that 8 hours of my script - is normal result, but I'm getting a memory overflow.
Why nip2 don't free this memory?
all parts of my image together occupy less than 1 GB. If i take small part and join it with big image - all work okey.
So, i think that garbage coolector dont work... Why remain in the memory portion, which is obviously no longer referenced.
On C++ i can divide big image on many smaller... but nip2 is lazy language - so the image is not actually divided.
Why such funtion as copy_to_file (I can make a mistake in the title) do not force nip2 perform operations on the part of the image?
how i can force the computation, can i save part of image on disk forcibly?
>> Eliminating tail recursion is probably best done by switching to a pointer reversing reduction engine. This doesn't actually remove tail recursion, but it removes the stack, which is almost the same thing.
Delete<< can you get simple example of program, which dont use stack?
I have such example:
Test P = class {
foo x y = 1 + y;
Result = foldr foo 0 [0..300000];
}
C stack overflow. Expression too complex.
why foldr use the stack?
Ah that's true. Yes, this would be a good thing to fix.
DeleteHello :) can i save image on disk by im_system image on nip2 script?
DeleteYes, I think so.
DeleteThe "official" way to save an image is to have it as the result of main and then use the -o switch to write that to a file.
but it is not working:
Deletemy_save Img = im_system Img.value "nip2 -e 'im_copy %s' -o /home/result1.png"
what am I doing wrong?
Show, please, how i can solve this task for image 10 x 500000 px, for A and B Parts Size of 1 and 2 px:
ReplyDelete[code]
MyMix Img = class {
check_uint x descript
= "ok", x > 0
= ("error: " ++ descript ++ " is not valid scope") / 0;
mix a b
= [], a == [] || b == []
= a, b == []
= b, a == []
= hd b : hd a : mix (tl a) (tl b);
list2img l
= "error: list2img, bad argument", l == []
= hd l, l == [hd l]
= join_tb (hd l) (list2img (tl l));
AH = Number "размер части А: " 100;
BH = Number "размер части В: " 200;
_A = AH.value;
_B = BH.value;
_CH1 = check_uint _A "часть A";
_CH2 = check_uint _B "часть В";
_AParts = improc.getABParts 0 _A Img _B;
_BParts = improc.getABParts _A _B Img _A;
_BAParts = mix _AParts _BParts;
Result = list2img _BAParts;
}
[/code]
I make decision with libvips on C++, but i save parts of result in harddisk, but can i do it with nip2 script?
I'll experiment and see if I can do this, but I think this may be beyond nip2.
DeleteIt's good for large images, not so good for large numbers of small images.
Here's a chop and join that I think go as quickly as nip2 can go:
Deletehttp://www.vips.ecs.soton.ac.uk/development/join.def
They don't have many features, and they avoid the nip2 class system so they don't need to create a class for every image. But they won't do 300,000 tiles. Maybe 30,000 if you are patient.
I think I would use Ruby for a job like this. The ruby-vips binding is rather nice.
I see join.def.
Deletestore part of the image in the list - is this normal?
After this part of the image will not be deleted (the memory will not be released) until the list goes out of scope, because this part will be a reference from the list. And this will lead to a wild overspending memory. Is not it?
Most images don't really exist, they are just a closure. An image of any size is just a few kb of memory.
Deletecan i call the nip2-plugin via command line interface?
ReplyDeleteYes, nip2 has a CLI interface. There's a section in the manual about it:
ReplyDeletehttp://www.vips.ecs.soton.ac.uk/supported/7.38/doc/html/nipguide/nipguidese13.html#x21-390004.5
hm, i read this attention: "These notes are for the Unix command-line, but they should work for Windows as well. ", but test it.
ReplyDeleteIt is not working.
in /bin directory you can find nip2.exe and nip2-cli.exe files.
first, eat my command without responce, second say about errors:
nip2 -e "2 + 2"
<- clean line
nip2 -eb "2 + 2"
<- clean line
nip2 "2 + 2"
<- clean line
nip2-cli.exe "2 + 2"
nip2.exe: unable to make $HOME\.nip2-7.38.1 : File exists
So, i think, that it is not working, i test several examples and...tty to start a script from file, but i have not any results.
You say, that i can save image on disk by im_system:
>The "official" way to save an image is to have it as the result of main and then use the -o switch to write that to a file.
I wont to call it from nip2 (not CLI, not C++), i can't set the arguments for this function, so i say about example. Pleeease :).
Hi, it's working here in Wine. I'll try to find a real Windows machine to test on.
DeleteYes, you need to use nip2-cli.exe since Windows does not allow .exes to be both command-line and GUI.
Hi there,
ReplyDeleteHow to stretch an image, contrary to shrinking? I thought im_stretch3 would do it, but I missunderstood it I guess..
Also, would affine transformations always result in smaller images? I did several JPEGs and all have 10-40kb cuts.
Hi, sorry, I only just found your comment. Blogger isn't mailing me when comments are submitted for moderation, for some reason.
ReplyDeleteUse an affine transform with a scale >1 and pick a suitable interpolator.