datafusion icon indicating copy to clipboard operation
datafusion copied to clipboard

feature flag to statically link liblzma

Open universalmind303 opened this issue 2 years ago • 3 comments

Is your feature request related to a problem or challenge?

datafusion automatically dynamically links lzma if you have the compression feature enabled due to the dependency on xz2

Describe the solution you'd like

add a new feature to statically link lzma, or at the very least document the workaround provided below.

Describe alternatives you've considered

You can manually statically link by adding the following to your cargo.toml

lzma-sys = { version = "*", features = ["static"] }

Additional context

some bugs caused by this

  • https://github.com/GlareDB/glaredb/pull/2631
  • https://github.com/lancedb/lance/issues/1912

universalmind303 avatar Feb 17 '24 15:02 universalmind303

Can we add xz2 = { version = "0.1", optional = true, features = ["static"] } into https://github.com/apache/arrow-datafusion/blob/main/datafusion/core/Cargo.toml#L100 ? 😕 cc @alamb

guojidan avatar Feb 18 '24 01:02 guojidan

It would certainly be good to add a feature flag (though it might be better to simply always statically link?)

I think statically linking by default makes sense to me and is consistent with most of the rest of the rust ecosystem / crates.

alamb avatar Feb 19 '24 07:02 alamb

@alamb i agree. Statically linking by default makes a lot of sense & greatly simplifies things for downstream users. Happy to make this change if you want to assign issue to me.

universalmind303 avatar Feb 21 '24 17:02 universalmind303