Recent Serverless SDK sending more traffic?

[samjarman]

Is there an existing issue for this?

• [X] I have checked for existing issues https://github.com/getsentry/sentry-javascript/issues
• [X] I have reviewed the documentation https://docs.sentry.io/
• [X] I am using the latest SDK release https://github.com/getsentry/sentry-javascript/releases

How do you use Sentry?

Self-hosted/on-premise

Which SDK are you using?

@sentry/serverless

SDK Version

7.101.1

Framework Version

No response

Link to Sentry event

No response

SDK Setup

import * as Sentry from '@sentry/serverless';
import { NodeOptions } from '@sentry/serverless/types/awslambda';

export const sentryConfig: NodeOptions = {
	dsn: process.env.SENTRY_DSN,
	environment: process.env.ENV,
};

Sentry.AWSLambda.init(sentryConfig);

export default Sentry;

And then elsewhere...

import Sentry from '../../helpers/sentry';
const myFunction = { ....}
export const handler = Sentry.AWSLambda.wrapHandler(myFunction);

Steps to Reproduce

Since moving from V7.74.1 up to the latest, one of the updates (I can't yet tell which) is causing higher than expected traffic to our self-hosted sentry instance. When we send constant traffic to our lambda, that causes constant errors, its observed that a higher amount of data hits our sentry instance in the latest SDKs. (Maybe the default sampling rate changed or something?)

In worse cases, or when the lambdas are under load, this has started to overwhelmed our sentry instances. This means that other lambdas also start taking longer, as (I think) the flushing is retrying, and taking up to 2 seconds which lines up with the comment in this docs: https://docs.sentry.io/platforms/node/guides/aws-lambda/

Is there any advice/wisdom for setting up Sentry to report quickly and minimally, particularly to not block lambda's returning (especially when they're powering APIs)?

Expected Result

I would only expect Sentry to communicate from the SDK out in the case of a caught or uncaught exception, or in the case of Sentry.captureException(...).

Actual Result

...Sorry my report doesn't really follow this format

[AbhiPrasad]

Are you able to inspect the network traffic going to sentry and seeing what is inside the envelopes? Is it more errors/transactions? Or something else?

[samjarman]

@AbhiPrasad Any ideas how I would do that?

[AbhiPrasad]

Given you're running self hosted sentry, are you able to look at the logs in your Relay?

On the SDK side, you can add

If you add the following:

```ts
Sentry.getClient().on('beforeEnvelope', (envelope) => {
  const body = env[1];
  const item = body[0];
  console.log(item);
});

you can also log out all the envelopes that are sent from your Sentry client. This will help see what is being flushed from the SDK.

[samjarman]

Hi @AbhiPrasad I added the logging and its looks as expected in version 7.101.1.

It doesn't help me trying to track down my issue but that's on me 😅

Questions:

1. Does Sentry send anything in the case of an errorless/not throwing lambda function run? (Any metrics telemetry which might wait up to 20 seconds to get sent out) (I don't see anything beforeEnvelope - are there other events to log?)

2. Same question as above: Is there any advice/wisdom for setting up Sentry to report quickly and minimally, particularly to not block lambda's returning (especially when they're powering APIs)?

[AbhiPrasad]

Does Sentry send anything in the case of an errorless/not throwing lambda function run

if you have performance monitoring set up it will send transactions. Everything should go through envelopes though, we don't send anything outside of that.

Is there any advice/wisdom for setting up Sentry to report quickly and minimally, particularly to not block lambda's returning

Unfortunately we always have to block exit to make sure things get flushed successfully. No good work around for this :(

[getsantry[bot]]

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you remove the label Waiting for: Community, I will leave it alone ... forever!

---

"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀