KnightChess comments

Results 71 comments of


                                            KnightChess

[SUPPORT] Querying Hudi tables with Spark+Velox（C++）, ObjectSizeCalculator.getObjectSize hangs causing about a 50-second delay in queries

@majian1998 #10504 look like we met the same problem, but we not use velox also met it.

[FEATURE][Spark] Support partial sort and combine for reducing shuffle data size

we also have this demand. I met in hudi table which will use `countByKey` to evaluation work load.

[FEATURE][Spark] Support partial sort and combine for reducing shuffle data size

any plan to support it?

[FEATURE][Spark] Support partial sort and combine for reducing shuffle data size

how about use `ExternalAppendOnlyMap` to archive it, but it's may spill to disk, we can add a param to controll it. And I think if map side can be combine,...

[FEATURE][Spark] Support partial sort and combine for reducing shuffle data size

I think the merging should work on the client side, not on the RSS server. It maximizes the utilization of executor resources, rather than placing the computation burden on the...

[FEATURE][Spark] Support partial sort and combine for reducing shuffle data size

And furthermore, I think the RSS server is best left to handle data transmission efficiently without any content processing. It should return whatever it receives from user tasks. All data...

[SUPPORT]hudi way of doing bucket index cannot be used to improve query engines queries such join and filter

> @KnightChess do you have intreast to push-forward this feature? @danny0405 yes, I follow up this problem

Add in-code docs for Rust

does it meat add doc for every `Moudles`, `Sturct`, `Method` in code, like this? ```rust /// Table interface for engine integrate #[derive(Clone, Debug)] pub struct Table { pub base_url: Arc,...

Add in-code docs for Rust

@xushiyan got it. I've been familiarizing myself with this project recently, and I can help contribute to this document.

Support partition pruning during file listing

Hi, I'd like to work for this feature if no one plan work for it.