fs2 icon indicating copy to clipboard operation
fs2 copied to clipboard

gzip compression not backwards compatible with gzip 1.3.12

Open jgogstad opened this issue 5 years ago • 0 comments

Gzip compressed files produced with fs2.compression.gzip are not recognized by gzip 1.3.12. This is primarily an issue for users on CentOS 6 (CentOS 7 ships a newer version that handles this).

Here's how to reproduce:

Produce a compressed file:

import java.nio.file.Paths

import cats.effect.{Blocker, ExitCode, IO, IOApp}
import cats.syntax.all._

object WriteCompressedFile extends IOApp {
  override def run(args: List[String]): IO[ExitCode] = Blocker[IO].use { b =>
    fs2.Stream.emit("lorem ipsum")
      .covary[IO]
      .through(fs2.text.utf8Encode)
      .through(fs2.compression.gzip())
      .through(fs2.io.file.writeAll(Paths.get("/tmp/bar/foo.gz"), b)).compile.drain.as(ExitCode.Success)
  }
}

Uncompress on CentOS 6

├─ /tmp/bar
╰ file foo.gz
foo.gz: gzip compressed data, has CRC, from MacOS, original size modulo 2^32 37

├─ /tmp/bar
╰ docker run --rm -it -v $(pwd):/foo centos:centos6 bash
[root@52c59137bb4b /]# cd foo
[root@52c59137bb4b foo]# ls -l
total 4
-rw-r--r-- 1 root root 57 Jun  5 08:21 foo.gz
[root@52c59137bb4b foo]# file foo.gz
foo.gz: gzip compressed data, has CRC, from MacOS
[root@52c59137bb4b foo]# gunzip foo.gz
gzip: foo.gz is a a multi-part gzip file -- not supported
[root@52c59137bb4b foo]# gzip -d foo.gz
gzip: foo.gz is a a multi-part gzip file -- not supported
[root@52c59137bb4b foo]#

Related issue on CentOS:

  • https://serverfault.com/questions/727752/how-to-upgrade-gzip-from-1-3-12-to-1-6-on-centos-6
  • https://forums.centos.org/viewtopic.php?p=260691#p260691

Doesn't seem likely to change there (snipped from last link):

What we ship is 1.3.12 and will be for the lifespan of the distro. CentOS 7 uses 1.5.8 and will do for its life too.

Not sure if it's feasible to be backwards compatible with gzip à 2007, but it's good to document incompatibilities anyway.

Feel free to close if this is out of scope

jgogstad avatar Jun 05 '20 08:06 jgogstad