jizezhang

Hello, I am facing the same issue. Were you able to resolve this? Thanks a lot!

Hello, I am facing the same issue. May I ask what is the fix for this?

take

Hi @alamb just double checking, I noticed this comment https://github.com/apache/datafusion/pull/18630/files#r2526799488, but regardless we still want to integrate to remove dependency on `CoalescerBatchesExec`? Thanks!

Hi @alamb wanted to check my understanding and discuss the approach a little bit. Please let me know your thoughts and correct me if anything sounds off. I thought about...

Hi @alamb, I wonder if I may confirm the behavior of one test https://github.com/apache/datafusion/blob/fc77be94570e3ada7e28db8c5412125f54e0b96d/datafusion/core/tests/custom_sources_cases/provider_filter_pushdown.rs#L267 with you wrt integrating with `BatchCoalescer`. I noticed that this test runs two versions of the...

Actually the behavior of one other test `test_preserve_order_with_spilling` may probably be also affected by this change. When reserving memory for an array, e.g. here https://github.com/apache/datafusion/blob/d24eb4a23156b7814836e765d5890186ab40682f/datafusion/physical-plan/src/sorts/stream.rs#L247-L250 the buffer size is computed...

Hi @alamb, I was running benchmarks with my PR https://github.com/apache/datafusion/pull/19002 (and some variations for comparison), though it seemed that the results somewhat varied a bit across runs, e.g. even with...

@alamb I put out a second PR https://github.com/apache/datafusion/pull/19239 to remove `RepartitionExec` from the optimizer rule. Please take a look when you get a chance. Thank you.

> It seems like after [#19239](https://github.com/apache/datafusion/pull/19239) all we have left now is to integrate `BatchCoalescer` into the `AsyncFuncExec` and we really could remove the `CoalesceBatches` optimization rule 🤔 Yes, is...