libvips
Performance when required multiple different resizes
Hello, I am having some performance doubts. My use case is resizing a large image into 3 different ones on a single core VPS.
It goes like this, a single jpg is read from memory, resized and saved the resized pyvips.Image is kept for later reuse, before saving it to a jpg. Then the same is done for the middle size sample, reusing the previous resized for performance. And finally the same is done for the thumbnail sized sample, also reusing the previous resized for performance.
Here is my benchmark. https://gist.github.com/fredgido/c09a4ed2456a14e8e1b4afba10589e75 , used sample image linked in code. On my laptop PIL 1.7433s vs VIPS 1.9279s when doing it like the way I said above, more detail on timmings on bellow the code text file on the gist. From those you can conclude it takes half of the time copying memory.
Doing like PIL it looked like to me like it was rereading the jpg and breaking performance. I tried a solution that was copy_memory but this still makes it take half of the time copying the memory around.
Am I missing something or it is related to how pyvips works?
As i see pyvips is faster, but on this middle stage reuse is making me think I am missing something. And so I am here asking help making faster than PIL.
Hi @fredgido,
Because pyvips is demand-driven, your trick of reusing the previous result is not a great fit. I would use thumbnail and run the whole thing three times. I noticed you're missing a run_time = bench() before the first call to vips, so the time you get is not quite right.
Here's an adapted version of your benchmark:
https://gist.github.com/jcupitt/1420f8046f98c1866857711e55cf6d48
I couldn't connect to https://files.catbox.moe/fxseh8.jpg, so I substituted this image, hopefully it's similar: http://www.rollthepotato.net/~john/nina.jpg
On this desktop, I see:
$ ./pil-vips.py ~/pics/nina.jpg
took 0.5819024660000001 thread time
took 0.5819 cpu time
took 0.5819 real time
original conversion with pil
took 0.11730546600000002 thread time
took 0.1173 cpu time
took 0.1173 real time
preview conversion with pil
took 0.0024904620000000266 thread time
took 0.0025 cpu time
took 0.0025 real time
thumbnail conversion with pil
took 0.701758878 thread time
took 0.7018 cpu time
took 0.7018 real time
total time for pil
memory copy took: 0.1700
took 0.014496911000000057 thread time
took 1.6757 cpu time
took 0.3362 real time
original conversion with vips
memory copy took: 0.0248
took 0.004016895999999992 thread time
took 0.3635 cpu time
took 0.0523 real time
preview conversion with vips
memory copy took: 0.0011
took 0.002831407999999924 thread time
took 0.0084 cpu time
took 0.0046 real time
thumbnail conversion with vips
took 0.021415886999999967 thread time
took 2.0477 cpu time
took 0.3932 real time
total time for vips
took 0.00630381999999996 thread time
took 0.3229 cpu time
took 0.1351 real time
original conversion with vips thumbnail
took 0.004640265000000032 thread time
took 0.0534 cpu time
took 0.0407 real time
preview conversion with vips thumbnail
took 0.004415751000000023 thread time
took 0.0364 cpu time
took 0.0319 real time
thumbnail conversion with vips thumbnail
took 0.01543783200000004 thread time
took 0.4127 cpu time
took 0.2077 real time
total time for vips thumbnail
ie. pillow-simd is 0.70s, pyvips resize is 0.39s, pyvips thumbnail is 0.21s.
That's with libvips threading enabled. You can time with just one core like this:
$ VIPS_CONCURRENCY=1 ./pil-vips.py ~/pics/nina.jpg
took 0.579884229 thread time
took 0.5799 cpu time
took 0.5799 real time
original conversion with pil
took 0.11691954900000001 thread time
took 0.1169 cpu time
took 0.1169 real time
preview conversion with pil
took 0.002483568000000047 thread time
took 0.0025 cpu time
took 0.0025 real time
thumbnail conversion with pil
took 0.699347224 thread time
took 0.6993 cpu time
took 0.6994 real time
total time for pil
memory copy took: 0.5162
took 0.024345118000000054 thread time
took 1.0600 cpu time
took 0.9404 real time
original conversion with vips
memory copy took: 0.1162
took 0.0037943790000000144 thread time
took 0.2356 cpu time
took 0.2336 real time
preview conversion with vips
memory copy took: 0.0021
took 0.0026850790000000346 thread time
took 0.0071 cpu time
took 0.0065 real time
thumbnail conversion with vips
took 0.030899612999999992 thread time
took 1.3027 cpu time
took 1.1805 real time
total time for vips
took 0.004217889999999946 thread time
took 0.2566 cpu time
took 0.2336 real time
original conversion with vips thumbnail
took 0.004278608000000017 thread time
took 0.0476 cpu time
took 0.0455 real time
preview conversion with vips thumbnail
took 0.003553360000000061 thread time
took 0.0305 cpu time
took 0.0300 real time
thumbnail conversion with vips thumbnail
took 0.012116503 thread time
took 0.3347 cpu time
took 0.3092 real time
total time for vips thumbnail
ie. pillow-simd is 0.70s, pyvips resize is 1.18, pyvips thumnail is 0.31s.
