Ability to get logs by "topic"

[lgalabru]

As of today, when debugging, we only have the options to get INFO/WARN/ERRO or DEBUG/INFO/WARN/ERRO. When enabling DEBUG logs, we're producing gigabytes of logs per hour which is a bit cumbersome. It would be useful to tag our logs with some kind of topic and be able to select some of them at runtime, with something like STACKS_LOG_TOPIC=atlas,pox,bitcoin,net,http,clarity,etc.

[psq]

even better if this was dynamic and did not require a restart, but filtering would be nice already!

[jcnelson]

Don't we already have this -- i.e. look at the log message's file name?

[kantai]

Don't we already have this -- i.e. look at the log message's file name?

I think that would be a pretty painful alternative: I may be interested in something like "debug logs and above for pox and atlas", "warn and above for net", and "info for everything else" and topic logging models usually allow you to do something like that. Grepping log outputs for something like that would be painful.

The other nice thing is that in many cases this allows the stacks-node to never produce the logging information at all. Debug logs on net and marf (marf in particular), are really really long, and trace logs on marf even add pretty costly computation. Support for topics would have the benefit that it could alleviate load on log aggregation and the stacks-node itself.

Fortunately, the logging framework we use already has a lot of support for this kind of thing -- we'd just need to construct multiple "logger" objects and make them available to logging sites as we update them.

[jcnelson]

Closing criteria for this issue:

• Get a PR merged that updates our logging macros to accept topics
• Start using them

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed. Please reopen if needed.