Imageflow benchmark

[aggarwaldev]

Hi,

Thank you for this awesome library.

I recently come across another library that claims upto 17x faster performance than ImageMagick. Here's the link: https://github.com/imazen/imageflow

Can you please add imageflow to benchmark contenders for better comparison with sharp?

[lovell]

There was some discussion on Twitter about this - see https://twitter.com/jcupitt65/status/1252905648368619520

All the code for the benchmarks is at https://github.com/lovell/sharp/tree/master/test/bench - happy to accept a PR if you're able.

[lovell]

I created a work-in-progress bench-add-imageflow branch for this and added the latest v0.0.1-rc8 of @imazen/imageflow via commit 5cddb81.

Some preliminary results when run on a ~3 year old laptop with low battery:

jpeg jimp-buffer-buffer x 0.65 ops/sec ±6.09% (8 runs sampled)
jpeg mapnik-buffer-buffer x 2.96 ops/sec ±2.58% (19 runs sampled)
jpeg imageflow-buffer-buffer x 3.65 ops/sec ±11.73% (21 runs sampled)
jpeg gm-buffer-buffer x 6.97 ops/sec ±3.97% (38 runs sampled)
jpeg sharp-buffer-buffer x 15.95 ops/sec ±10.44% (52 runs sampled)

It looks like imageflow ships with and uses mozjpeg for JPEG output, which might explain why its ops/sec figure above is rather low. I'm sure the way in which I'm calling it could be wrong, and it's only on v0.0.1-rc8 so I'd imagine there's still work to be done.

[jcupitt]

imazen have a docker image for benchmarking, which is where they get their timings. You can test like this:

$ docker pull imazen/imageflow_bench_ubuntu20
...
$ docker run -it --rm --entrypoint /bin/bash imazen/imageflow_bench_ubuntu20
imageflow@f4d55028ee4e:~$ vipsheader u1.jpg 
u1.jpg: 5104x3380 uchar, 3 bands, srgb, jpegload
imageflow@f4d55028ee4e:~$ time ./bin/imageflow_tool v0.1/ir4 --in u1.jpg --out if.jpg --command "width=128&height=85&quality=89" >& /dev/null

real	0m0.217s
user	0m0.196s
sys	0m0.021s
imageflow@f4d55028ee4e:~$ time vipsthumbnail u1.jpg --size 128 --strip --output v.jpg[Q=89]

real	0m0.212s
user	0m0.200s
sys	0m0.013s
imageflow@f4d55028ee4e:~$ ls -l v.jpg if.jpg 
-rw-r--r-- 1 imageflow 1001 5490 Jun 27 07:33 if.jpg
-rw-r--r-- 1 imageflow 1001 5519 Jun 27 07:42 v.jpg

So they are pretty much the same speed for small JPEG thumbnails. iflow wins this test in the imazen benchmarks because they use the --linear flag to vipsthumbnail, which turns off jpeg shrink-on-load. I don't think many sharp users will be using linear light resampling.

For a larger output size:

imageflow@f4d55028ee4e:~$ time vipsthumbnail u1.jpg --size 2000 --output v.jpg[optimize-coding,strip,Q=89]

real	0m0.394s
user	0m0.575s
sys	0m0.041s
imageflow@f4d55028ee4e:~$ time ./bin/imageflow_tool v0.1/ir4 --in u1.jpg --out if.jpg --command "width=2000&height=1324&quality=89" >& /dev/null

real	0m1.252s
user	0m1.207s
sys	0m0.044s
imageflow@f4d55028ee4e:~$ ls -l v.jpg if.jpg 
-rw-r--r-- 1 imageflow 1001 565222 Jun 27 07:43 if.jpg
-rw-r--r-- 1 imageflow 1001 741079 Jun 27 07:43 v.jpg

iflow is /much/ slower, but perhaps that's mozjpeg, as Lovell says. The file size difference does look like mozjpeg.

libvips is a bit quicker at PNG images:

imageflow@f4d55028ee4e:~$ time vipsthumbnail u1.png --size 128 --output v.jpg[optimize-coding,strip,Q=89]

real	0m0.693s
user	0m0.653s
sys	0m0.040s
imageflow@f4d55028ee4e:~$ time ./bin/imageflow_tool v0.1/ir4 --in u1.png --out if.jpg --command "width=128&height=85&quality=89" >& /dev/null

real	0m0.915s
user	0m0.850s
sys	0m0.065s

Perhaps because libvips can run PNG decode and image resize in parallel.

[lovell]

Here's a patch to add imageflow to the perf tests in case anyone wants to revisit this, however the project seems rather stagnant so I'll close for now.

--- a/test/bench/perf.js
+++ b/test/bench/perf.js
@@ -15,6 +15,7 @@ const imagemagick = require('imagemagick');
 const mapnik = require('mapnik');
 const jimp = require('jimp');
 const squoosh = require('@squoosh/lib');
+const imageflow = require('@imazen/imageflow');
 
 const fixtures = require('../fixtures');
 
@@ -253,6 +254,43 @@ async.series({
           });
       }
     });
+    // imageflow
+    jpegSuite.add('imageflow-file-file', {
+      defer: true,
+      fn: function (deferred) {
+        new imageflow.Steps(new imageflow.FromFile(fixtures.inputJpg))
+          .constrainWithin(width, height, { down_filter: 'lanczos' })
+          .encode(
+            new imageflow.FromFile(outputJpg),
+            new imageflow.MozJPEG(80, false)
+          )
+          .execute()
+          .then(function () {
+            deferred.resolve();
+          })
+          .catch(function (err) {
+            throw err;
+          });
+      }
+    }).add('imageflow-buffer-buffer', {
+      defer: true,
+      fn: function (deferred) {
+        new imageflow.Steps(new imageflow.FromBuffer(inputJpgBuffer))
+          .constrainWithin(width, height, { down_filter: 'lanczos' })
+          .encode(
+            new imageflow.FromBuffer(null, 'key'),
+            new imageflow.MozJPEG(80, false)
+          )
+          .execute()
+          .then(function (buffer) {
+            assert.notStrictEqual(null, buffer);
+            deferred.resolve();
+          })
+          .catch(function (err) {
+            throw err;
+          });
+      }
+    });
     // sharp
     jpegSuite.add('sharp-buffer-file', {
       defer: true,