snuba
snuba copied to clipboard
getsentry
`snuba-billing-consumer` container panicked caused by "called `Result::unwrap()` on an `Err` value: KafkaError (Consumer commit error: CoordinatorLoadInProgress)"
Environment
24.8.0, self-hosted
Steps to Reproduce
See here https://github.com/getsentry/self-hosted/issues/3330#issuecomment-2387709775
snuba-billing-consumer container panicked. I don't know what happened previously since this is not from my instance.
snuba-outcomes-billing-consumer-1 | 2024-09-27 07:18:18,891 Initializing Snuba...
snuba-outcomes-billing-consumer-1 | 2024-09-27 07:18:28,312 Snuba initialization took 9.421803737000005s
snuba-outcomes-billing-consumer-1 | 2024-09-27 07:18:31,491 Initializing Snuba...
snuba-outcomes-billing-consumer-1 | 2024-09-27 07:18:45,525 Snuba initialization took 14.035213774999988s
snuba-outcomes-billing-consumer-1 | {"timestamp":"2024-09-27T07:18:45.580742Z","level":"INFO","fields":{"message":"Storage: outcomes_raw, ClickHouse Table Name: outcomes_raw_local, Message Processor: MessageProcessorConfig { python_class_name: \"OutcomesProcessor\", python_module: \"snuba.datasets.processors.outcomes_processor\" }, ClickHouse host: clickhouse, ClickHouse port: 9000, ClickHouse HTTP port: 8123, ClickHouse database: default"},"target":"rust_snuba::consumer"}
snuba-outcomes-billing-consumer-1 | {"timestamp":"2024-09-27T07:18:45.581731Z","level":"INFO","fields":{"message":"Starting consumer for \"outcomes_raw\"","storage":"outcomes_raw"},"target":"rust_snuba::consumer"}
snuba-outcomes-billing-consumer-1 | {"timestamp":"2024-09-27T07:18:46.569645Z","level":"INFO","fields":{"message":"Kafka consumer member id: \"rdkafka-0a097bef-1201-4913-8b0e-308f82d88d47\""},"target":"rust_arroyo::backends::kafka"}
snuba-outcomes-billing-consumer-1 | {"timestamp":"2024-09-27T07:18:46.569868Z","level":"INFO","fields":{"message":"New partitions assigned: {Partition { topic: Topic(\"outcomes-billing\"), index: 0 }: 3230605}"},"target":"rust_arroyo::processing"}
snuba-outcomes-billing-consumer-1 | {"timestamp":"2024-09-27T07:18:49.584362Z","level":"INFO","fields":{"message":"Partitions to revoke: [Partition { topic: Topic(\"outcomes-billing\"), index: 0 }]"},"target":"rust_arroyo::processing"}
snuba-outcomes-billing-consumer-1 | {"timestamp":"2024-09-27T07:18:49.584407Z","level":"INFO","fields":{"message":"Joining strategy"},"target":"rust_arroyo::processing"}
snuba-outcomes-billing-consumer-1 | {"timestamp":"2024-09-27T07:18:49.584476Z","level":"WARN","fields":{"message":"Timeout Some(0ns) reached while waiting for tasks to finish"},"target":"rust_arroyo::processing::strategies::reduce"}
snuba-outcomes-billing-consumer-1 | {"timestamp":"2024-09-27T07:18:49.584825Z","level":"INFO","fields":{"message":"Committing offsets"},"target":"rust_arroyo::processing"}
snuba-outcomes-billing-consumer-1 | {"timestamp":"2024-09-27T07:18:49.591273Z","level":"INFO","fields":{"message":"Partition revocation complete."},"target":"rust_arroyo::processing"}
snuba-outcomes-billing-consumer-1 | {"timestamp":"2024-09-27T07:18:49.879681Z","level":"INFO","fields":{"message":"Kafka consumer member id: \"rdkafka-0a097bef-1201-4913-8b0e-308f82d88d47\""},"target":"rust_arroyo::backends::kafka"}
snuba-outcomes-billing-consumer-1 | {"timestamp":"2024-09-27T07:18:49.879810Z","level":"INFO","fields":{"message":"New partitions assigned: {Partition { topic: Topic(\"outcomes-billing\"), index: 0 }: 3230616}"},"target":"rust_arroyo::processing"}
snuba-outcomes-billing-consumer-1 | {"timestamp":"2024-09-27T07:18:52.684338Z","level":"INFO","fields":{"message":"Partitions to revoke: [Partition { topic: Topic(\"outcomes-billing\"), index: 0 }]"},"target":"rust_arroyo::processing"}
snuba-outcomes-billing-consumer-1 | {"timestamp":"2024-09-27T07:18:52.684381Z","level":"INFO","fields":{"message":"Joining strategy"},"target":"rust_arroyo::processing"}
snuba-outcomes-billing-consumer-1 | {"timestamp":"2024-09-27T07:18:52.684389Z","level":"WARN","fields":{"message":"Timeout Some(0ns) reached while waiting for tasks to finish"},"target":"rust_arroyo::processing::strategies::reduce"}
snuba-outcomes-billing-consumer-1 | {"timestamp":"2024-09-27T07:18:52.684490Z","level":"INFO","fields":{"message":"Partition revocation complete."},"target":"rust_arroyo::processing"}
snuba-outcomes-billing-consumer-1 | {"timestamp":"2024-09-27T07:18:53.000686Z","level":"INFO","fields":{"message":"Kafka consumer member id: \"rdkafka-0a097bef-1201-4913-8b0e-308f82d88d47\""},"target":"rust_arroyo::backends::kafka"}
snuba-outcomes-billing-consumer-1 | {"timestamp":"2024-09-27T07:18:53.001073Z","level":"INFO","fields":{"message":"New partitions assigned: {Partition { topic: Topic(\"outcomes-billing\"), index: 0 }: 3230616}"},"target":"rust_arroyo::processing"}
snuba-outcomes-billing-consumer-1 | thread '<unnamed>' panicked at /root/.cargo/git/checkouts/arroyo-2fe52eb044a47b23/b15d546/rust-arroyo/src/backends/kafka/mod.rs:125:52:
snuba-outcomes-billing-consumer-1 | called `Result::unwrap()` on an `Err` value: KafkaError (Consumer commit error: CoordinatorLoadInProgress (Broker: Coordinator load in progress))
snuba-outcomes-billing-consumer-1 | stack backtrace:
snuba-outcomes-billing-consumer-1 | 0: rust_begin_unwind
snuba-outcomes-billing-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/std/src/panicking.rs:597:5
snuba-outcomes-billing-consumer-1 | 1: core::panicking::panic_fmt
snuba-outcomes-billing-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/core/src/panicking.rs:72:14
snuba-outcomes-billing-consumer-1 | 2: core::result::unwrap_failed
snuba-outcomes-billing-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/core/src/result.rs:1652:5
snuba-outcomes-billing-consumer-1 | 3: core::result::Result<T,E>::unwrap
snuba-outcomes-billing-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/core/src/result.rs:1077:23
snuba-outcomes-billing-consumer-1 | 4: rust_arroyo::backends::kafka::commit_impl
snuba-outcomes-billing-consumer-1 | at /root/.cargo/git/checkouts/arroyo-2fe52eb044a47b23/b15d546/rust-arroyo/src/backends/kafka/mod.rs:125:5
snuba-outcomes-billing-consumer-1 | 5: <rust_arroyo::backends::kafka::KafkaConsumer<C> as rust_arroyo::backends::Consumer<rust_arroyo::backends::kafka::types::KafkaPayload,C>>::commit_offsets
snuba-outcomes-billing-consumer-1 | at /root/.cargo/git/checkouts/arroyo-2fe52eb044a47b23/b15d546/rust-arroyo/src/backends/kafka/mod.rs:496:9
snuba-outcomes-billing-consumer-1 | 6: rust_arroyo::processing::StreamProcessor<TPayload>::_run_once
snuba-outcomes-billing-consumer-1 | at /root/.cargo/git/checkouts/arroyo-2fe52eb044a47b23/b15d546/rust-arroyo/src/processing/mod.rs:309:17
snuba-outcomes-billing-consumer-1 | 7: rust_arroyo::processing::StreamProcessor<TPayload>::run_once::{{closure}}
snuba-outcomes-billing-consumer-1 | at /root/.cargo/git/checkouts/arroyo-2fe52eb044a47b23/b15d546/rust-arroyo/src/processing/mod.rs:244:49
snuba-outcomes-billing-consumer-1 | 8: core::ops::function::FnOnce::call_once
snuba-outcomes-billing-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/core/src/ops/function.rs:250:5
snuba-outcomes-billing-consumer-1 | 9: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
snuba-outcomes-billing-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/core/src/panic/unwind_safe.rs:271:9
snuba-outcomes-billing-consumer-1 | 10: std::panicking::try::do_call
snuba-outcomes-billing-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/std/src/panicking.rs:504:40
snuba-outcomes-billing-consumer-1 | 11: std::panicking::try
snuba-outcomes-billing-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/std/src/panicking.rs:468:19
snuba-outcomes-billing-consumer-1 | 12: std::panic::catch_unwind
snuba-outcomes-billing-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/std/src/panic.rs:142:14
snuba-outcomes-billing-consumer-1 | 13: rust_arroyo::processing::StreamProcessor<TPayload>::run_once
snuba-outcomes-billing-consumer-1 | at /root/.cargo/git/checkouts/arroyo-2fe52eb044a47b23/b15d546/rust-arroyo/src/processing/mod.rs:244:9
snuba-outcomes-billing-consumer-1 | 14: rust_arroyo::processing::StreamProcessor<TPayload>::run
snuba-outcomes-billing-consumer-1 | at /root/.cargo/git/checkouts/arroyo-2fe52eb044a47b23/b15d546/rust-arroyo/src/processing/mod.rs:418:29
snuba-outcomes-billing-consumer-1 | 15: rust_snuba::consumer::consumer_impl
snuba-outcomes-billing-consumer-1 | at ./rust_snuba/src/consumer.rs:266:25
snuba-outcomes-billing-consumer-1 | 16: rust_snuba::consumer::consumer::{{closure}}
snuba-outcomes-billing-consumer-1 | at ./rust_snuba/src/consumer.rs:48:9
snuba-outcomes-billing-consumer-1 | 17: pyo3::marker::Python::allow_threads
snuba-outcomes-billing-consumer-1 | at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-0.18.3/src/marker.rs:473:9
snuba-outcomes-billing-consumer-1 | 18: rust_snuba::consumer::consumer
snuba-outcomes-billing-consumer-1 | at ./rust_snuba/src/consumer.rs:47:5
snuba-outcomes-billing-consumer-1 | 19: rust_snuba::consumer::_::__pyfunction_consumer
snuba-outcomes-billing-consumer-1 | at ./rust_snuba/src/consumer.rs:27:1
snuba-outcomes-billing-consumer-1 | 20: pyo3::impl_::trampoline::fastcall_with_keywords::{{closure}}
snuba-outcomes-billing-consumer-1 | at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-0.18.3/src/impl_/trampoline.rs:41:35
snuba-outcomes-billing-consumer-1 | 21: pyo3::impl_::trampoline::trampoline_inner::{{closure}}
snuba-outcomes-billing-consumer-1 | at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-0.18.3/src/impl_/trampoline.rs:204:54
snuba-outcomes-billing-consumer-1 | 22: std::panicking::try::do_call
snuba-outcomes-billing-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/std/src/panicking.rs:504:40
snuba-outcomes-billing-consumer-1 | 23: std::panicking::try
snuba-outcomes-billing-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/std/src/panicking.rs:468:19
snuba-outcomes-billing-consumer-1 | 24: std::panic::catch_unwind
snuba-outcomes-billing-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/std/src/panic.rs:142:14
snuba-outcomes-billing-consumer-1 | 25: pyo3::impl_::trampoline::trampoline_inner
snuba-outcomes-billing-consumer-1 | at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-0.18.3/src/impl_/trampoline.rs:204:9
snuba-outcomes-billing-consumer-1 | 26: pyo3::impl_::trampoline::fastcall_with_keywords
snuba-outcomes-billing-consumer-1 | at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-0.18.3/src/impl_/trampoline.rs:52:1
snuba-outcomes-billing-consumer-1 | 27: rust_snuba::consumer::_::<impl rust_snuba::consumer::consumer::MakeDef>::DEF::trampoline
snuba-outcomes-billing-consumer-1 | at ./rust_snuba/src/consumer.rs:27:1
snuba-outcomes-billing-consumer-1 | 28: <unknown>
snuba-outcomes-billing-consumer-1 | 29: PyObject_Vectorcall
snuba-outcomes-billing-consumer-1 | 30: _PyEval_EvalFrameDefault
snuba-outcomes-billing-consumer-1 | 31: <unknown>
snuba-outcomes-billing-consumer-1 | 32: <unknown>
snuba-outcomes-billing-consumer-1 | 33: _PyEval_EvalFrameDefault
snuba-outcomes-billing-consumer-1 | 34: <unknown>
snuba-outcomes-billing-consumer-1 | 35: <unknown>
snuba-outcomes-billing-consumer-1 | 36: <unknown>
snuba-outcomes-billing-consumer-1 | 37: _PyEval_EvalFrameDefault
snuba-outcomes-billing-consumer-1 | 38: <unknown>
snuba-outcomes-billing-consumer-1 | 39: <unknown>
snuba-outcomes-billing-consumer-1 | 40: _PyEval_EvalFrameDefault
snuba-outcomes-billing-consumer-1 | 41: <unknown>
snuba-outcomes-billing-consumer-1 | 42: _PyObject_FastCallDictTstate
snuba-outcomes-billing-consumer-1 | 43: _PyObject_Call_Prepend
snuba-outcomes-billing-consumer-1 | 44: <unknown>
snuba-outcomes-billing-consumer-1 | 45: _PyObject_MakeTpCall
snuba-outcomes-billing-consumer-1 | 46: _PyEval_EvalFrameDefault
snuba-outcomes-billing-consumer-1 | 47: <unknown>
snuba-outcomes-billing-consumer-1 | 48: PyEval_EvalCode
snuba-outcomes-billing-consumer-1 | 49: <unknown>
snuba-outcomes-billing-consumer-1 | 50: <unknown>
snuba-outcomes-billing-consumer-1 | 51: <unknown>
snuba-outcomes-billing-consumer-1 | 52: _PyRun_SimpleFileObject
snuba-outcomes-billing-consumer-1 | 53: _PyRun_AnyFileObject
snuba-outcomes-billing-consumer-1 | 54: <unknown>
snuba-outcomes-billing-consumer-1 | 55: Py_BytesMain
snuba-outcomes-billing-consumer-1 | 56: <unknown>
snuba-outcomes-billing-consumer-1 | 57: __libc_start_main
snuba-outcomes-billing-consumer-1 | 58: _start
snuba-outcomes-billing-consumer-1 | note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
snuba-outcomes-billing-consumer-1 | {"timestamp":"2024-09-28T10:58:13.636959Z","level":"INFO","fields":{"message":"Caught error, terminating strategy"},"target":"rust_arroyo::processing"}
snuba-outcomes-billing-consumer-1 | {"timestamp":"2024-09-28T10:58:13.645814Z","level":"INFO","fields":{"message":"Partitions to revoke: [Partition { topic: Topic(\"outcomes-billing\"), index: 0 }]"},"target":"rust_arroyo::processing"}
snuba-outcomes-billing-consumer-1 | {"timestamp":"2024-09-28T10:58:13.650165Z","level":"INFO","fields":{"message":"Joining strategy"},"target":"rust_arroyo::processing"}
snuba-outcomes-billing-consumer-1 | {"timestamp":"2024-09-28T10:58:13.650206Z","level":"WARN","fields":{"message":"Timeout Some(0ns) reached while waiting for tasks to finish"},"target":"rust_arroyo::processing::strategies::reduce"}
snuba-outcomes-billing-consumer-1 | {"timestamp":"2024-09-28T10:58:13.658885Z","level":"INFO","fields":{"message":"Committing offsets"},"target":"rust_arroyo::processing"}
snuba-outcomes-billing-consumer-1 | thread '<unnamed>' panicked at /root/.cargo/git/checkouts/arroyo-2fe52eb044a47b23/b15d546/rust-arroyo/src/backends/kafka/mod.rs:125:52:
snuba-outcomes-billing-consumer-1 | called `Result::unwrap()` on an `Err` value: KafkaError (Consumer commit error: CoordinatorLoadInProgress (Broker: Coordinator load in progress))
snuba-outcomes-billing-consumer-1 | stack backtrace:
snuba-outcomes-billing-consumer-1 | 0: rust_begin_unwind
snuba-outcomes-billing-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/std/src/panicking.rs:597:5
snuba-outcomes-billing-consumer-1 | 1: core::panicking::panic_fmt
snuba-outcomes-billing-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/core/src/panicking.rs:72:14
snuba-outcomes-billing-consumer-1 | 2: core::result::unwrap_failed
snuba-outcomes-billing-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/core/src/result.rs:1652:5
snuba-outcomes-billing-consumer-1 | 3: core::result::Result<T,E>::unwrap
snuba-outcomes-billing-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/core/src/result.rs:1077:23
snuba-outcomes-billing-consumer-1 | 4: rust_arroyo::backends::kafka::commit_impl
snuba-outcomes-billing-consumer-1 | at /root/.cargo/git/checkouts/arroyo-2fe52eb044a47b23/b15d546/rust-arroyo/src/backends/kafka/mod.rs:125:5
snuba-outcomes-billing-consumer-1 | 5: <rust_arroyo::backends::kafka::OffsetCommitter<C> as rust_arroyo::backends::CommitOffsets>::commit
snuba-outcomes-billing-consumer-1 | at /root/.cargo/git/checkouts/arroyo-2fe52eb044a47b23/b15d546/rust-arroyo/src/backends/kafka/mod.rs:135:9
snuba-outcomes-billing-consumer-1 | 6: <rust_arroyo::processing::Callbacks<TPayload> as rust_arroyo::backends::AssignmentCallbacks>::on_revoke
snuba-outcomes-billing-consumer-1 | at /root/.cargo/git/checkouts/arroyo-2fe52eb044a47b23/b15d546/rust-arroyo/src/processing/mod.rs:161:35
snuba-outcomes-billing-consumer-1 | 7: <rust_arroyo::backends::kafka::CustomContext<C> as rdkafka::consumer::ConsumerContext>::rebalance
snuba-outcomes-billing-consumer-1 | at /root/.cargo/git/checkouts/arroyo-2fe52eb044a47b23/b15d546/rust-arroyo/src/backends/kafka/mod.rs:229:17
snuba-outcomes-billing-consumer-1 | 8: rdkafka::consumer::base_consumer::BaseConsumer<C>::handle_rebalance_event
snuba-outcomes-billing-consumer-1 | at /root/.cargo/git/checkouts/rust-rdkafka-34d944143d921bb0/e69c2aa/src/consumer/base_consumer.rs:196:17
snuba-outcomes-billing-consumer-1 | 9: rdkafka::consumer::base_consumer::BaseConsumer<C>::poll_queue
snuba-outcomes-billing-consumer-1 | at /root/.cargo/git/checkouts/rust-rdkafka-34d944143d921bb0/e69c2aa/src/consumer/base_consumer.rs:146:25
snuba-outcomes-billing-consumer-1 | 10: rdkafka::consumer::base_consumer::BaseConsumer<C>::poll
snuba-outcomes-billing-consumer-1 | at /root/.cargo/git/checkouts/rust-rdkafka-34d944143d921bb0/e69c2aa/src/consumer/base_consumer.rs:118:9
snuba-outcomes-billing-consumer-1 | 11: <rdkafka::consumer::base_consumer::BaseConsumer<C> as core::ops::drop::Drop>::drop
snuba-outcomes-billing-consumer-1 | at /root/.cargo/git/checkouts/rust-rdkafka-34d944143d921bb0/e69c2aa/src/consumer/base_consumer.rs:731:21
snuba-outcomes-billing-consumer-1 | 12: core::ptr::drop_in_place<rdkafka::consumer::base_consumer::BaseConsumer<rust_arroyo::backends::kafka::CustomContext<rust_arroyo::processing::Callbacks<rust_arroyo::backends::kafka::types::KafkaPayload>>>>
snuba-outcomes-billing-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/core/src/ptr/mod.rs:497:1
snuba-outcomes-billing-consumer-1 | 13: core::ptr::drop_in_place<rust_arroyo::backends::kafka::KafkaConsumer<rust_arroyo::processing::Callbacks<rust_arroyo::backends::kafka::types::KafkaPayload>>>
snuba-outcomes-billing-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/core/src/ptr/mod.rs:497:1
snuba-outcomes-billing-consumer-1 | 14: core::ptr::drop_in_place<alloc::boxed::Box<dyn rust_arroyo::backends::Consumer<rust_arroyo::backends::kafka::types::KafkaPayload,rust_arroyo::processing::Callbacks<rust_arroyo::backends::kafka::types::KafkaPayload>>>>
snuba-outcomes-billing-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/core/src/ptr/mod.rs:497:1
snuba-outcomes-billing-consumer-1 | 15: core::ptr::drop_in_place<rust_arroyo::processing::StreamProcessor<rust_arroyo::backends::kafka::types::KafkaPayload>>
snuba-outcomes-billing-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/core/src/ptr/mod.rs:497:1
snuba-outcomes-billing-consumer-1 | 16: rust_arroyo::processing::StreamProcessor<TPayload>::run
snuba-outcomes-billing-consumer-1 | at /root/.cargo/git/checkouts/arroyo-2fe52eb044a47b23/b15d546/rust-arroyo/src/processing/mod.rs:432:5
snuba-outcomes-billing-consumer-1 | 17: rust_snuba::consumer::consumer_impl
snuba-outcomes-billing-consumer-1 | at ./rust_snuba/src/consumer.rs:266:25
snuba-outcomes-billing-consumer-1 | 18: rust_snuba::consumer::consumer::{{closure}}
snuba-outcomes-billing-consumer-1 | at ./rust_snuba/src/consumer.rs:48:9
snuba-outcomes-billing-consumer-1 | 19: pyo3::marker::Python::allow_threads
snuba-outcomes-billing-consumer-1 | at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-0.18.3/src/marker.rs:473:9
snuba-outcomes-billing-consumer-1 | 20: rust_snuba::consumer::consumer
snuba-outcomes-billing-consumer-1 | at ./rust_snuba/src/consumer.rs:47:5
snuba-outcomes-billing-consumer-1 | 21: rust_snuba::consumer::_::__pyfunction_consumer
snuba-outcomes-billing-consumer-1 | at ./rust_snuba/src/consumer.rs:27:1
snuba-outcomes-billing-consumer-1 | 22: pyo3::impl_::trampoline::fastcall_with_keywords::{{closure}}
snuba-outcomes-billing-consumer-1 | at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-0.18.3/src/impl_/trampoline.rs:41:35
snuba-outcomes-billing-consumer-1 | 23: pyo3::impl_::trampoline::trampoline_inner::{{closure}}
snuba-outcomes-billing-consumer-1 | at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-0.18.3/src/impl_/trampoline.rs:204:54
snuba-outcomes-billing-consumer-1 | 24: std::panicking::try::do_call
snuba-outcomes-billing-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/std/src/panicking.rs:504:40
snuba-outcomes-billing-consumer-1 | 25: std::panicking::try
snuba-outcomes-billing-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/std/src/panicking.rs:468:19
snuba-outcomes-billing-consumer-1 | 26: std::panic::catch_unwind
snuba-outcomes-billing-consumer-1 | at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/std/src/panic.rs:142:14
snuba-outcomes-billing-consumer-1 | 27: pyo3::impl_::trampoline::trampoline_inner
snuba-outcomes-billing-consumer-1 | at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-0.18.3/src/impl_/trampoline.rs:204:9
snuba-outcomes-billing-consumer-1 | 28: pyo3::impl_::trampoline::fastcall_with_keywords
snuba-outcomes-billing-consumer-1 | at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-0.18.3/src/impl_/trampoline.rs:52:1
snuba-outcomes-billing-consumer-1 | 29: rust_snuba::consumer::_::<impl rust_snuba::consumer::consumer::MakeDef>::DEF::trampoline
snuba-outcomes-billing-consumer-1 | at ./rust_snuba/src/consumer.rs:27:1
snuba-outcomes-billing-consumer-1 | 30: <unknown>
snuba-outcomes-billing-consumer-1 | 31: PyObject_Vectorcall
snuba-outcomes-billing-consumer-1 | 32: _PyEval_EvalFrameDefault
snuba-outcomes-billing-consumer-1 | 33: <unknown>
snuba-outcomes-billing-consumer-1 | 34: <unknown>
snuba-outcomes-billing-consumer-1 | 35: _PyEval_EvalFrameDefault
snuba-outcomes-billing-consumer-1 | 36: <unknown>
snuba-outcomes-billing-consumer-1 | 37: <unknown>
snuba-outcomes-billing-consumer-1 | 38: <unknown>
snuba-outcomes-billing-consumer-1 | 39: _PyEval_EvalFrameDefault
snuba-outcomes-billing-consumer-1 | 40: <unknown>
snuba-outcomes-billing-consumer-1 | 41: <unknown>
snuba-outcomes-billing-consumer-1 | 42: _PyEval_EvalFrameDefault
snuba-outcomes-billing-consumer-1 | 43: <unknown>
snuba-outcomes-billing-consumer-1 | 44: _PyObject_FastCallDictTstate
snuba-outcomes-billing-consumer-1 | 45: _PyObject_Call_Prepend
snuba-outcomes-billing-consumer-1 | 46: <unknown>
snuba-outcomes-billing-consumer-1 | 47: _PyObject_MakeTpCall
snuba-outcomes-billing-consumer-1 | 48: _PyEval_EvalFrameDefault
snuba-outcomes-billing-consumer-1 | 49: <unknown>
snuba-outcomes-billing-consumer-1 | 50: PyEval_EvalCode
snuba-outcomes-billing-consumer-1 | 51: <unknown>
snuba-outcomes-billing-consumer-1 | 52: <unknown>
snuba-outcomes-billing-consumer-1 | 53: <unknown>
snuba-outcomes-billing-consumer-1 | 54: _PyRun_SimpleFileObject
snuba-outcomes-billing-consumer-1 | 55: _PyRun_AnyFileObject
snuba-outcomes-billing-consumer-1 | 56: <unknown>
snuba-outcomes-billing-consumer-1 | 57: Py_BytesMain
snuba-outcomes-billing-consumer-1 | 58: <unknown>
snuba-outcomes-billing-consumer-1 | 59: __libc_start_main
snuba-outcomes-billing-consumer-1 | 60: _start
snuba-outcomes-billing-consumer-1 | note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
Expected Result
Not panicked.
Actual Result
Panicked.
