sentry-ruby icon indicating copy to clipboard operation
sentry-ruby copied to clipboard

Events sent from Resque workers can be randomly dropped

Open st0012 opened this issue 3 years ago • 4 comments

The SDK's background worker relies on an at_exit callback to finish its queue before the process exits to make sure no events are dropped unsent. But because Resque workers exit with exit! by default, which skips all at_exit callbacks, some events may not be sent before a worker exits. So events manually sent from a Resque worker may not be sent because of this.

Note that sentry-resque's built-in capturing call has hint: { background: false }, so those events will be sent synchronously and aren't affected by this issue.

(The exit! behavior caused many issues and the latest report is https://github.com/resque/resque/issues/1664)

st0012 avatar Mar 21 '22 20:03 st0012

I think this issue can be solve from 2 directions:

  1. Tell users to set RUN_AT_EXIT_HOOKS=1.
  2. Force all Resque events to be send synchronously.

I think 2) is the better option as it's within our control and sending Resque events synchronously shouldn't bring too much performance impact to users' apps.

st0012 avatar Mar 21 '22 20:03 st0012

hmm, so we'll need to send the transactions sync too? We haven't exposed this option so far right?

sl0thentr0py avatar Mar 22 '22 11:03 sl0thentr0py

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you label it Status: Backlog or Status: In Progress, I will leave it alone ... forever!


"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀

github-actions[bot] avatar Apr 13 '22 00:04 github-actions[bot]

I think this issue can be solve from 2 directions:

  1. Tell users to set RUN_AT_EXIT_HOOKS=1.
  2. Force all Resque events to be send synchronously.

I think 2) is the better option as it's within our control and sending Resque events synchronously shouldn't bring too much performance impact to users' apps.

Are these still the recommended ways to get around this issue? We're seeing all transaction events be dropped on sentry-resque 5.8.0 and resque 2.5.0. RUN_AT_EXIT_HOOKS=1 definitely solves it but I'm wondering if there's a better way.

Is there a way for us to force Sentry's events buffer to flush so that we can write it into/around a Resque hook?

SimantovYousoufov avatar Apr 04 '23 19:04 SimantovYousoufov