cloudflow icon indicating copy to clipboard operation
cloudflow copied to clipboard

Add Flink 1.12 support to Cloudflow (reason: we would like to use Azure Data Lake Store Gen2)

Open tringuyen-yw opened this issue 5 years ago • 5 comments

Is your feature request related to a problem? Please describe. Please add support for Flink 1.12 in Cloudflow.

Flink 1.12 add the ability to use ADLS Gen2 for checkpointing, which is very interesting for us: FLINK-18562 Add Support for Azure Data Lake Store Gen 2 in Flink File System

tringuyen-yw avatar Oct 16 '20 14:10 tringuyen-yw

As of Cloudflow 2.0.10, Flink streamlet docker images are shipped with Flink 1.11

That's not correct, at the moment we only support Flink 1.10. We would first have to see that everything works correctly with Flink 1.12. We have tried before with Flink 1.11 but there were some issues...

RayRoestenburg avatar Oct 18 '20 13:10 RayRoestenburg

(I might just be confused by the phrasing of your feature request, I guess you mean that you would like Flink 1.12 support?)

RayRoestenburg avatar Oct 18 '20 13:10 RayRoestenburg

@tringuyen-yw It looks like Flink 1.12 is not released yet: https://cwiki.apache.org/confluence/display/FLINK/1.12+Release Expected release date November?

RayRoestenburg avatar Oct 21 '20 15:10 RayRoestenburg

I think migration to 1.11 is an important milestone, especially if we decide to use Flink kubernetes native.

blublinsky avatar Oct 23 '20 17:10 blublinsky

Support for Flink 1.12 would be an exciting addition.

synchris avatar Dec 01 '21 15:12 synchris