Structured Logs: follow-up

[JoshuaMoelans]

The following items are all potential follow-ups of the initial Structured Logs implementation introduced in https://github.com/getsentry/sentry-native/pull/1271 .

• add debug output for structured logs when debug option is enabled
  • https://github.com/getsentry/sentry-native/pull/1466
• increase logs queue size
  • https://github.com/getsentry/sentry-native/issues/1469
• length format specifier handling
  • see comment
  • https://github.com/getsentry/sentry-native/issues/1401
• replace cond_wake with level-triggered flag
  • see comment
  • https://github.com/getsentry/sentry-native/issues/1402
• Set sentry.origin for logs coming from integrations
  • For now, only if created from downstream (e.g. Unreal), since native itself just has the manual API.
  • https://develop.sentry.dev/sdk/telemetry/logs/#sdk-integration-attributes
• Logger integrations
  • logging sentry__logger_func() calls can be added as an option, but need to disable logging when in the sentry_logs.c code to avoid recursion.
  • community requests for C/C++ logging libraries (feel free to add requests below!)
• Named parameter API
  • to allow for better searchability for sentry.message.parameter.X where X is no longer confined to being numerical, we can add an alternative API that allows passing in a sentry_value_t list containing (key, value) pair objects. Proof-of-concept already lives here https://github.com/getsentry/sentry-native/pull/1297

Done

• Using SENTRY_SDK_NAME instead of always writing sentry.native when setting the sdk name attribute
  • this changes to native.android or native.xbox in our SDK, and gets set to native.unreal/unity/godot from the respective GDX SDKs
  • https://github.com/getsentry/sentry-native/pull/1399
• ensure logs get sent/cached to disk on crash
  • inspired by https://github.com/getsentry/rfcs/pull/148
  • https://github.com/getsentry/sentry-native/issues/1403
  • https://github.com/getsentry/sentry-native/pull/1404
• Allow passing in custom attributes
  • The spec allows for key-value paired attributes which are added as key : {"value": value, "type": type} to the 'attributes' item in the envelope. This is possible using before_send_log, but not convenient (as one would need to inspect the log to add log-specific attributes).
  • https://github.com/getsentry/sentry-native/issues/1405
  • https://github.com/getsentry/sentry-native/pull/1435
• Batch Processor Minimize Data Loss
  • We already flush logs on shutdown (and on crash), but we didn't yet add this for calling sentry_flush.
  • https://github.com/getsentry/sentry-native/issues/1425
  • https://github.com/getsentry/sentry-native/pull/1434

[linear[bot]]

NATIVE-127

[supervacuus]

Suggestion: If this is intended as a meta issue, please break it out into separate issues as soon as someone starts working on them (ideally before planning during the weekly sync). Especially if you expect future appendage, it will be hard to decipher the link back and find descriptions when PRs only link back. Additionally, in terms of prioritization, all follow-ups will then not be represented by only one linear item.

[limbonaut]

I'm not sure if this was discussed already. Looking at the current API there is no possibility to supply custom attributes, which are part of the protocol. The examples in the developer docs suggests adding attributes as part of the API. In Godot SDK, it would look like this:

SentrySDK.logger.info("Collected coin %d", [coin.id], {
  "level.name": current_level.name,
  "coin.id": coin.id,
  "stats.collected_coins": GameStats.num_coins
}

Currently, it doesn't seem possible to implement attributes using native (no such issues in cocoa for example).

EDIT: better example.

[JoshuaMoelans]

Hi @limbonaut , thanks for raising this. We kind of provide this through our before_send_log hook (see the example in our docs), but I agree that this is not the most straightforward way to implement this (and doesn't easily give you control over which logs get which attributes).

We'll need to think about what makes the most sense, since we currently use sentry_log_X(string, args...) as a printf-like interface. We could either provide another sentry_log_X_attr(attr, string, args...) interface or just keep the existing API and allow users to append a sentry_value_t object with key-value pairs for the attributes.

[limbonaut]

Hi @JoshuaMoelans 👋 You could also consider adding a low-level call without changing a thing about the shortcuts:

sentry_log(level, string, attributes)

This should be enough for hybrid SDKs to implement the protocol. In Godot, for example, we have a formatting syntax, that I'd say is expected to be supported. So being able to support the local implementation of parameters is also important.

[supervacuus]

Hi @limbonaut.

We actually already implemented a version that would provide the above (up to a point), but the essential question that implementation left open was: who owns the formatter? (and as a corollary: which interpolation scheme should the Native SDK default to, if any, given that C string formatting is entirely positional?)

For downstream usage, for instance, various format-string encodings could be possible. The question then is: do we defer the actual string interpolation to a hook that can be used downstream? Or does the backend do this? And of course: how far do we take this? attributes would most likely be a sentry_value_t. Who decodes nesting and similar accessors in a given string interpolation scheme?

To sum it up, yes, this was a genuine concern on our minds, but since these questions remained unanswered, we wanted to wait for actual feedback from usage.

[supervacuus]

To be clear, we can provide an interface to attach attributes that do not need to be reflected in the string interpolation of the provided format string. However, if we tackle the topic, we should ensure that we can provide a solution that somehow converges without requiring a significant set of variants or complicated arguments.

[supervacuus]

In any case, just saw that @JoshuaMoelans created a new issue, maybe let's continue the discussion for this particular topic there.