Premature termination of Bzip2::Reader - critical!

[maasha]

I am experiencing nasty premature termination of Bzip2::Reader. I am including a test script and a data file located here:

ftp://ftp_20130926_12853:[email protected]

Notice the difference in output:

./test.rb | wc 13206 16508 952823

bzcat bzip_test.bz2 | wc 14604484 18255605 994881701

Versions:

Linux orsted 3.2.0-4-amd64 #1 SMP Debian 3.2.46-1 x86_64 GNU/Linux ruby 1.9.2p180 (2011-02-18 revision 30909) [x86_64-linux] bzip2-ruby (0.2.7)

[cawel]

I am experiencing an abrupt termination after about (only) 900 KB has been uncompressed. Any development regarding that issue?

The impression I get is as if my bz2 source file would have a special character in it which makes the Bzip2::Reader stop reading/uncompressing the source file. But I know my bz2 source file is valid since tar -xvf source_file.bz2 works flawlessly.