Support IN and NOT IN operator in partition filter

[kazdy]

Description of the improvement

I'd like to use IN and NOT IN operators when for partition filter.

SQL equivalent is:

SELECT * FROM huditable WHERE part IN ('2022-01-02', '2022-01-03');
SELECT * FROM huditable WHERE part NOT IN ('2022-01-02', '2022-01-03');

Expected behavior

IN and NOT IN are honored by the PartitionPruner and PartitionFilter.

Additional context

Blocked on https://github.com/apache/hudi-rs/issues/160

[kazdy]

I'll contribute this one

[jonathanc-n]

@kazdy Hullo, are you still working on this? I can help build upon it, but if my recent PR gets merged there will be a lot of conflicts so we could possibly open another pr after. This is a good step towards getting all the predicate functionality

[kazdy]

Hey, I parked it for some time. I'll get back to it next month I hope, feel free to make your changes and I'll resolve conflicts

[xushiyan]

Sounds good. We'll merge https://github.com/apache/hudi-rs/pull/203 first

[xushiyan]

hey @kazdy #160 is closed now. Do you want to resume this, and any ETA? thanks.

[kazdy]

@xushiyan I want to resume this, I just don't know when I'll have time, so no ETA sorry

[xushiyan]

@xushiyan I want to resume this, I just don't know when I'll have time, so no ETA sorry

Understood, @kazdy , no worries. We will evaluate the priority in a week, and decide picking it up for 0.3.0 or pushing to a future release.

[jonathanc-n]

@kazdy if you do not have the time, I can probably help take it over. Currently on break this week, can pick it up next week if thats fine with you?

[kazdy]

@jonathanc-n feel free to implement it!