Haocheng Liu comments

Results 6 comments of


                                            Haocheng Liu

[add_columns] implements add columns task through ray

I believe https://github.com/lancedb/lance/pull/3369 addresses this issue. Please feel free to reopen if I miss something.

Fuzz testing and finish implementation for large_string / large_binary in 2.1

This problem has two parts. The above CR will support the generic string compression and the next one will support FSST.

Fuzz testing and finish implementation for large_string / large_binary in 2.1

I think this issue could be closed? @westonpace

Up to 20x : Binary Copy Capability For Compaction

Hey, the zonemap/bloomfilter uses a notion called `zone size` which is independent of the fragments so that it provides maximum flexibility. The TLDR version is that users can arbitrarily specify...

Up to 20x : Binary Copy Capability For Compaction

> - @HaochengLIU my understanding is that, although a zone has a fixed size, it won't go beyond a fragment right? For example if I have zone size 100, and...

Up to 20x : Binary Copy Capability For Compaction

> - Footer Finalization: After all data is copied to a new file, the footer is written using the standard FileWriter::finish() method. This function takes the accumulated column metadata, page...