toy
`invalid byte sequence in UTF-8` on Linux
I installed image_optim on my Raspberry and find that I can't use image_optim because of an encoding error that sadly comes without any stacktrace. I remembered that Ruby had some encoding issues in older versions (?1.9?), so I made sure that I have ruby2.2 installed; I removed the existing image_optim and re-installed with sudo gem2.2 install image_optim_pack image_optim. But the error still persists:
flow@raspi:/tmp/kb-test$ image_optim dont-touch-me.jpg
invalid byte sequence in UTF-8
That's all I get as output, except there's now a mysteriously-named file in the directory:
flow@raspi:/tmp/kb-test$ ls -AlF
total 188
-rw-rw-r-- 1 flow flow 0 Jul 25 15:32 ??5@@???@8
...
The questionmarks are really replacement characters; using my Linux Mint file browser over sshfs I can see that the first byte is 0x01, and the one right before the second @ is 0x04.
The weird thing is that I can install and run image_optim on my Linux Mint box without errors, so the error should be caused by a difference between the two systems, not image_optim per se.
Here's what uname -a gives me:
Linux raspi 4.1.15-v7+ #830 SMP Tue Dec 15 17:02:45 GMT 2015 armv7l armv7l armv7l GNU/Linux
How can I proceed from here?
You may need to use image_optim2.2 as normally the binaries are getting the suffix if the ruby binary has the suffix. Also I suggest adding --verbose to the command which besides other things will not suppress the exception backtrace.
Thanks for your consideration.—I seem not to have an image_optim2.2 executable, at least not on the path; since I removed the previous installation and did the new one with gem2.2 install, it should use the most recent Ruby, I guess.
From the stacktrace below it would appear that the real issue is hidden in these two lines:
image_optim_pack: advpng, jpegtran, jhead, jpeg-recompress, optipng,
pngcrush, jpegoptim, gifsicle
from /var/lib/gems/2.2.0/gems/image_optim_pack-0.5.0.20170712/vendor/linux-x86_64 failed
image_optim_pack: advpng, jpegtran, jhead, jpeg-recompress, optipng,
pngcrush, jpegoptim, gifsicle
from /var/lib/gems/2.2.0/gems/image_optim_pack-0.5.0.20170712/vendor/linux-i686 failed
Maybe there are no suitable binaries for the Raspi ARM architecture?
OTOH the first indicted line, (91 of /var/lib/gems/2.2.0/gems/image_optim-0.25.0/lib/image_optim/bin_resolver/bin.rb) reads:
88 when :jhead, :'jpeg-recompress'
89 capture("#{escaped_path} -V 2> #{Path::NULL}")[/\d+(\.\d+)+/]
90 when :jpegtran
91 capture("#{escaped_path} -v - 2>&1")[/version (\d+\S*)/, 1]
92 when :pngcrush
93 capture("#{escaped_path} -version 2>&1")[/pngcrush (\d+(\.\d+)+)/, 1]
94 when :pngout
95 date_regexp = /[A-Z][a-z]{2} (?: |\d)\d \d{4}/
and indeed, when I replace that RegEx stuff with a constant string '2.2' (or anything), the encoding error goes away! image_optim still fails with svgo worker: svgo not found; please provide proper binary (so, no binaries for ARM?), but the invalid byte sequence does look like a bug to me.
Full stacktrace with --verbose:
flow@raspi:/tmp/kb-test$ image_optim --verbose the-entire-internet-in-december-1969.jpg
image_optim v0.25.0
config:
verbose: true
nice: 10
threads: 4
pack: true
skip_missing_workers: true
allow_lossy: false
cache_dir:
cache_worker_digests: false
image_optim_pack: advpng, jpegtran, jhead, jpeg-recompress, optipng, pngcrush, jpegoptim, gifsicle from /var/lib/gems/2.2.0/gems/image_optim_pack-0.5.0.20170712/vendor/linux-x86_64 failed
image_optim_pack: advpng, jpegtran, jhead, jpeg-recompress, optipng, pngcrush, jpegoptim, gifsicle from /var/lib/gems/2.2.0/gems/image_optim_pack-0.5.0.20170712/vendor/linux-i686 failed
invalid byte sequence in UTF-8
/var/lib/gems/2.2.0/gems/image_optim-0.25.0/lib/image_optim/bin_resolver/bin.rb:91:in `[]'
/var/lib/gems/2.2.0/gems/image_optim-0.25.0/lib/image_optim/bin_resolver/bin.rb:91:in `version_string'
/var/lib/gems/2.2.0/gems/image_optim-0.25.0/lib/image_optim/bin_resolver/bin.rb:77:in `detect_version'
/var/lib/gems/2.2.0/gems/image_optim-0.25.0/lib/image_optim/bin_resolver/bin.rb:20:in `initialize'
/var/lib/gems/2.2.0/gems/image_optim_pack-0.5.0.20170712/lib/image_optim/pack.rb:33:in `new'
/var/lib/gems/2.2.0/gems/image_optim_pack-0.5.0.20170712/lib/image_optim/pack.rb:33:in `block in bins'
/var/lib/gems/2.2.0/gems/image_optim_pack-0.5.0.20170712/lib/image_optim/pack.rb:32:in `map'
/var/lib/gems/2.2.0/gems/image_optim_pack-0.5.0.20170712/lib/image_optim/pack.rb:32:in `bins'
/var/lib/gems/2.2.0/gems/image_optim_pack-0.5.0.20170712/lib/image_optim/pack.rb:39:in `all_bins_working?'
/var/lib/gems/2.2.0/gems/image_optim_pack-0.5.0.20170712/lib/image_optim/pack.rb:107:in `debug_message'
/var/lib/gems/2.2.0/gems/image_optim_pack-0.5.0.20170712/lib/image_optim/pack.rb:90:in `block in path'
/var/lib/gems/2.2.0/gems/image_optim_pack-0.5.0.20170712/lib/image_optim/pack.rb:89:in `each'
/var/lib/gems/2.2.0/gems/image_optim_pack-0.5.0.20170712/lib/image_optim/pack.rb:89:in `find'
/var/lib/gems/2.2.0/gems/image_optim_pack-0.5.0.20170712/lib/image_optim/pack.rb:89:in `path'
/var/lib/gems/2.2.0/gems/image_optim-0.25.0/lib/image_optim/bin_resolver.rb:86:in `init_pack'
/var/lib/gems/2.2.0/gems/image_optim-0.25.0/lib/image_optim/bin_resolver.rb:25:in `initialize'
/var/lib/gems/2.2.0/gems/image_optim-0.25.0/lib/image_optim.rb:84:in `new'
/var/lib/gems/2.2.0/gems/image_optim-0.25.0/lib/image_optim.rb:84:in `initialize'
/var/lib/gems/2.2.0/gems/image_optim-0.25.0/lib/image_optim/runner.rb:54:in `new'
/var/lib/gems/2.2.0/gems/image_optim-0.25.0/lib/image_optim/runner.rb:54:in `initialize'
/var/lib/gems/2.2.0/gems/image_optim-0.25.0/bin/image_optim:17:in `new'
/var/lib/gems/2.2.0/gems/image_optim-0.25.0/bin/image_optim:17:in `<top (required)>'
/usr/local/bin/image_optim:23:in `load'
/usr/local/bin/image_optim:23:in `<main>'
I suggest to inspect the value of escaped_path as it should be the source of invalid bytes