Bartosz Konieczny

Hi @bithw1 I will take a look at your example today or tomorrow. Thank you.

Re @bithw1 I didn't succeed to reproduce the same generated code. Do you have any specific setup? The code I get every time looks like: ```java /* 001 */ public...

OK, that explains why we got different results :) I launched the code against _Spark 3 preview 2_ and also got a single scan stage: ```java [2020-01-07 06:20:10,399] org.apache.spark.internal.Logging DEBUG...

Let's confirm first if the output is really different for you if you launch the code against 2.4.0. > above is 2.4.0, we should see line 549, which is: >...

Hi @bithw1 , Thank you for the suggestion! Indeed, the post you're quoting is quite old and I'm totally agree that it would be interesting to deep delve into that...

Hi @bithw1 I published the first post from the series https://www.waitingforcode.com/apache-spark-sql/reorder-join-optimizer/read. I will add all of them under "Spark SQL reorder join" tag on this page https://www.waitingforcode.com/tags/spark-sql-reorder-join Cheers, Bartosz.

You're welcome @bithw1 Thank you for an interesting topic :)

Thank you @igreenfield for such an amazing question! I was looking for the reasons in the documentation and old PR but found any information about that. I've just posted a...

Hi @igreenfield , I've some answers from the mailing list: * https://mail-archives.apache.org/mod_mbox/spark-dev/201911.mbox/%3CCAJLcJd8oyMj58W8g6VPipgs73LEJn8h8x3DPCOUCgePrQtNSzQ%40mail.gmail.com%3E * https://mail-archives.apache.org/mod_mbox/spark-dev/201911.mbox/%3CCAPh_B%3DY%3DNU-3QWUcGJECcBJ3a1XxOOc9VF6tGBd7kL6YUCGW%3Dg%40mail.gmail.com%3E Long story short, it's all about the compilation performance :) Regarding your point about 64KB limitation,...

Re @igreenfield At that moment I don't have much time so I won't be able to help you. Sorry for that, late January it should be better. Meantime, maybe you...