self-hosted
self-hosted copied to clipboard
Published
1 month ago
•
getsentry
getsentry
snuba erroring out with SASL connection to Kafka not being supported only PLAIN?
Self-Hosted Version
24.9.0
CPU Architecture
x86_64
Docker Version
26.0.2 build 3c863ff
Docker Compose Version
v2.26.1
Steps to Reproduce
in the docker compose file:
- remove the kafka service (as we are using out own kafka setup)
- under the x-snuba-defaults change the kafka settings from:
DEFAULT_BROKERS: "kafka:9092"to have KAFKA settings:
DEFAULT_BROKERS: "kafka.ourdomain.com:9094"
KAFKA_SECURITY_PROTOCOL: "SASL_PLAINTEXT"
KAFKA_SASL_MECHANISM: "SCRAM-SHA-512"
KAFKA_SASL_USERNAME: "<username>"
KAFKA_SASL_PASSWORD: "<password>"
- Comment out wherever the service depends_on kafka service.
Tried even setting DEFAULT_BROKERS to be SASL_PLAINTEXT://kafka.ourdomain.com:9094 as we use 9094 port instead of 9092... even tried for the sake of it setting 9092 but the same error which would be the SASL not being supported in snuba container...
Expected Result
Expected result was to connect Sentry setup to our instance of Kafka and be able to open Sentry and login.
Actual Result
installation goes through, but once we run docker compose up, we are getting bunch of errors from snuba about not being able to connect to Kafka:
snuba-generic-metrics-counters-consumer-1 | note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
snuba-generic-metrics-counters-consumer-1 | Traceback (most recent call last):
snuba-generic-metrics-counters-consumer-1 | File "/usr/local/bin/snuba", line 33, in <module>
snuba-generic-metrics-counters-consumer-1 | sys.exit(load_entry_point('snuba', 'console_scripts', 'snuba')())
snuba-generic-metrics-counters-consumer-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
snuba-generic-metrics-counters-consumer-1 | File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
snuba-generic-metrics-counters-consumer-1 | return self.main(*args, **kwargs)
snuba-generic-metrics-counters-consumer-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^
snuba-generic-metrics-counters-consumer-1 | File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1055, in main
snuba-generic-metrics-counters-consumer-1 | rv = self.invoke(ctx)
snuba-generic-metrics-counters-consumer-1 | ^^^^^^^^^^^^^^^^
snuba-generic-metrics-counters-consumer-1 | File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
snuba-generic-metrics-counters-consumer-1 | return _process_result(sub_ctx.command.invoke(sub_ctx))
snuba-generic-metrics-counters-consumer-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
snuba-generic-metrics-counters-consumer-1 | File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
snuba-generic-metrics-counters-consumer-1 | return ctx.invoke(self.callback, **ctx.params)
snuba-generic-metrics-counters-consumer-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
snuba-generic-metrics-counters-consumer-1 | File "/usr/local/lib/python3.11/site-packages/click/core.py", line 760, in invoke
snuba-generic-metrics-counters-consumer-1 | return __callback(*args, **kwargs)
snuba-generic-metrics-counters-consumer-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^
snuba-generic-metrics-counters-consumer-1 | File "/usr/src/snuba/snuba/cli/rust_consumer.py", line 242, in rust_consumer
snuba-generic-metrics-counters-consumer-1 | exitcode = rust_snuba.consumer( # type: ignore
snuba-generic-metrics-counters-consumer-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
snuba-generic-metrics-counters-consumer-1 | pyo3_runtime.PanicException: called `Result::unwrap()` on an `Err` value: KafkaError (Client creation error: No provider for SASL mechanism SCRAM-SHA-512: recompile librdkafka with libsasl2 or openssl support. Current build options: PLAIN)
snuba-outcomes-consumer-1 | 0: rust_begin_unwind
snuba-outcomes-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/std/src/panicking.rs:597:5
snuba-outcomes-consumer-1 | 1: core::panicking::panic_fmt
snuba-outcomes-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/core/src/panicking.rs:72:14
snuba-outcomes-consumer-1 | 2: core::result::unwrap_failed
snuba-outcomes-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/core/src/result.rs:1652:5
snuba-outcomes-consumer-1 | 3: core::result::Result<T,E>::unwrap
snuba-outcomes-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/core/src/result.rs:1077:23
snuba-outcomes-consumer-1 | 4: rust_arroyo::processing::StreamProcessor<rust_arroyo::backends::kafka::types::KafkaPayload>::with_kafka
snuba-outcomes-consumer-1 | at /root/.cargo/git/checkouts/arroyo-2fe52eb044a47b23/b15d546/rust-arroyo/src/processing/mod.rs:212:33
snuba-outcomes-consumer-1 | 5: rust_snuba::consumer::consumer_impl
snuba-outcomes-consumer-1 | at ./rust_snuba/src/consumer.rs:257:21
snuba-outcomes-consumer-1 | 6: rust_snuba::consumer::consumer::{{closure}}
snuba-outcomes-consumer-1 | at ./rust_snuba/src/consumer.rs:48:9
snuba-outcomes-consumer-1 | 7: pyo3::marker::Python::allow_threads
snuba-outcomes-consumer-1 | at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-0.18.3/src/marker.rs:473:9
snuba-outcomes-consumer-1 | 8: rust_snuba::consumer::consumer
snuba-outcomes-consumer-1 | at ./rust_snuba/src/consumer.rs:47:5
snuba-outcomes-consumer-1 | 9: rust_snuba::consumer::_::__pyfunction_consumer
snuba-outcomes-consumer-1 | at ./rust_snuba/src/consumer.rs:27:1
snuba-outcomes-consumer-1 | 10: pyo3::impl_::trampoline::fastcall_with_keywords::{{closure}}
snuba-outcomes-consumer-1 | at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-0.18.3/src/impl_/trampoline.rs:41:35
snuba-outcomes-consumer-1 | 11: pyo3::impl_::trampoline::trampoline_inner::{{closure}}
snuba-outcomes-consumer-1 | at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-0.18.3/src/impl_/trampoline.rs:204:54
snuba-outcomes-consumer-1 | 12: std::panicking::try::do_call
snuba-outcomes-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/std/src/panicking.rs:504:40
snuba-outcomes-consumer-1 | 13: std::panicking::try
snuba-outcomes-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/std/src/panicking.rs:468:19
snuba-outcomes-consumer-1 | 14: std::panic::catch_unwind
snuba-outcomes-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/std/src/panic.rs:142:14
snuba-outcomes-consumer-1 | 15: pyo3::impl_::trampoline::trampoline_inner
snuba-outcomes-consumer-1 | at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-0.18.3/src/impl_/trampoline.rs:204:9
snuba-outcomes-consumer-1 | 16: pyo3::impl_::trampoline::fastcall_with_keywords
snuba-outcomes-consumer-1 | at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-0.18.3/src/impl_/trampoline.rs:52:1
snuba-outcomes-consumer-1 | 17: rust_snuba::consumer::_::<impl rust_snuba::consumer::consumer::MakeDef>::DEF::trampoline
snuba-outcomes-consumer-1 | at ./rust_snuba/src/consumer.rs:27:1
snuba-outcomes-consumer-1 | 18: <unknown>
snuba-outcomes-consumer-1 | 19: PyObject_Vectorcall
snuba-outcomes-consumer-1 | 20: _PyEval_EvalFrameDefault
snuba-outcomes-consumer-1 | 21: <unknown>
snuba-outcomes-consumer-1 | 22: <unknown>
snuba-outcomes-consumer-1 | 23: _PyEval_EvalFrameDefault
snuba-outcomes-consumer-1 | 24: <unknown>
snuba-outcomes-consumer-1 | 25: <unknown>
snuba-outcomes-consumer-1 | 26: <unknown>
snuba-outcomes-consumer-1 | 27: _PyEval_EvalFrameDefault
snuba-outcomes-consumer-1 | 28: <unknown>
snuba-outcomes-consumer-1 | 29: <unknown>
snuba-outcomes-consumer-1 | 30: _PyEval_EvalFrameDefault
snuba-outcomes-consumer-1 | 31: <unknown>
snuba-outcomes-consumer-1 | 32: _PyObject_FastCallDictTstate
snuba-outcomes-consumer-1 | 33: _PyObject_Call_Prepend
snuba-outcomes-consumer-1 | 34: <unknown>
snuba-outcomes-consumer-1 | 35: _PyObject_MakeTpCall
snuba-outcomes-consumer-1 | 36: _PyEval_EvalFrameDefault
snuba-outcomes-consumer-1 | 37: <unknown>
snuba-outcomes-consumer-1 | 38: PyEval_EvalCode
snuba-outcomes-consumer-1 | 39: <unknown>
snuba-outcomes-consumer-1 | 40: <unknown>
snuba-outcomes-consumer-1 | 41: <unknown>
snuba-outcomes-consumer-1 | 42: _PyRun_SimpleFileObject
snuba-outcomes-consumer-1 | 43: _PyRun_AnyFileObject
snuba-outcomes-consumer-1 | 44: <unknown>
snuba-outcomes-consumer-1 | 45: Py_BytesMain
snuba-outcomes-consumer-1 | 46: <unknown>
snuba-outcomes-consumer-1 | 47: __libc_start_main
snuba-outcomes-consumer-1 | 48: _start
it comes from almost all the snuba components the same error.
Is snuba image really not supporting Kafka SASL? And I thought snuba is supposed to be connecting only to clickhouse? Why does it even need kafka configuration?
Thanks for the help!
Event ID
No response
It's not supported, see here https://github.com/getsentry/snuba/issues/6358#issuecomment-2407417060
I'm not an employee, so I suggest creating another issue at the snuba repo.
Duplicate of https://github.com/getsentry/snuba/issues/6441. Let's keep only one issue open.
