faust icon indicating copy to clipboard operation
faust copied to clipboard

Add features to support ssl_context for producer if broker_producer is used.

Open youngsol opened this issue 2 years ago • 4 comments

Checklist

  • [X] I have included information about relevant versions
  • [X] I have verified that the issue persists when using the master branch of Faust.

Steps to reproduce

Tell us what you did to cause something to happen.

Expected behavior

broker_producer config can be set but different security protocol cannot be applied.

Tell us what you expected to happen.

ssl context for producer can be set since broker/consumer machines are different.

Actual behavior

same ssl context is used.

Tell us what happened instead.

Full traceback

Paste the full traceback (if there is any)

Versions

  • Python version 3.8
  • Faust version 1.10.4
  • Operating system
  • Kafka version
  • RocksDB version (if applicable)

youngsol avatar Apr 07 '23 12:04 youngsol

Faust version 1.10.4

Try uninstalling Faust and reinstalling Faust-streaming

wbarnha avatar Apr 07 '23 12:04 wbarnha

Oops, that's right but I still don't see the producer_ssl_config in document neither in code. faust-streaming==0.10.10

https://github.com/faust-streaming/faust/blob/d9d2d3eef0538c3296f9b7bd2b32777ec4c1cc4d/faust/transport/drivers/aiokafka.py#L1078-L1119

Producer and consumer shares same ssl_context config

youngsol avatar Apr 07 '23 12:04 youngsol

Could you provide an example of what you're trying to do? Typically I've had no issue using https://faust-streaming.github.io/faust/userguide/settings.html#std-setting-ssl_context to configure my Faust SSL contexts.

wbarnha avatar Apr 07 '23 16:04 wbarnha

app = faust.App(
    'test',
    broker_consumer=[server1,server2],
    broker_producer=[server3,server4],
    broker_credentials = faust.SSLCredentials(context="some_context")
)

There will be a problem if consumers and producers are set to use different security protocol since [server1.server2] and [server3,server4] are different kafka clusters

youngsol avatar Apr 07 '23 17:04 youngsol