dagger icon indicating copy to clipboard operation
dagger copied to clipboard

feat: Capability in dagger to consume from ACL enabled kafka clusters

Open mayankrai09 opened this issue 3 years ago • 0 comments

we are working on a centralized Kafka cluster that would contain data from other entities (Gojek/Gofin..etc). For compliance reasons, the cluster needs to be ACL enabled. Also, there is a need to run daggers that would consume from this ACL-enabled cluster. We are planning to use securtiy_protocol: SASL_PLAINTEXT sasl_mechanism: SCRAM-SHA-512/SCRAM-SHA-256 [We took the decision to use SCRAM for the ease of adding new users] Apart from these configs we would also need to have jaas-config in the consumer side containing the user credentials. We would be mounting the jaas-config to that particular flink cluster which would be consuming from ACL-enabled Kafka clusters.

mayankrai09 avatar Sep 21 '22 17:09 mayankrai09