Component-based log levels

[CassOnMars]

Presently, our logger defaults to Info, using --debug as a flag to turn on debug logging. This helps cut out a lot of noisy logs, but there are times where debug logs are really helpful. https://github.com/QuilibriumNetwork/monorepo/pull/453 found a case where debug logs are on always, which isn't intended, but it also was a case where log level varied by component. This issue is a note that it would be good for us to have configurable logging (long term, preferably service-driven so we can change log level dynamically) at the component level so we can avoid having extra noise while still having debug logs for specific parts of the protocol.

[blacks1ne]

If the idea is to let the node operator enable Debug logging level for selected node components (with the default logging level being Info), would it be sufficient to introduce the <component>DebugEnabled boolean configuration parameter to the Logger section and increase the logging level in the corresponding component's constructor? For example, for a Worker Manager, the current constructor looks as follows:

func NewWorkerManager(
	store typesStore.WorkerStore,
	logger *zap.Logger,
	config *config.Config,
	proposeFunc func(
		coreIds []uint,
		filters [][]byte,
		serviceClients map[uint]*grpc.ClientConn,
	) error,
	decideFunc func(
		reject [][]byte,
		confirm [][]byte,
	) error,
) typesWorker.WorkerManager {
	return &WorkerManager{
		store:            store,
		logger:           logger.Named("worker_manager"),
		workersByFilter:  make(map[string]uint),
		filtersByWorker:  make(map[uint][]byte),
		allocatedWorkers: make(map[uint]bool),
		serviceClients:   make(map[uint]*grpc.ClientConn),
		config:           config,
		proposeFunc:      proposeFunc,
		decideFunc:       decideFunc,
	}
}

It could be replaced with the following one

func NewWorkerManager(
	store typesStore.WorkerStore,
	logger *zap.Logger,
	config *config.Config,
	proposeFunc func(
		coreIds []uint,
		filters [][]byte,
		serviceClients map[uint]*grpc.ClientConn,
	) error,
	decideFunc func(
		reject [][]byte,
		confirm [][]byte,
	) error,
) typesWorker.WorkerManager {
	compLogger := logger.Named("worker_manager")
	if config.Logger.WorkerManagerDebugEnabled {
		compLogger = compLogger.WithOptions(zap.IncreaseLevel(zap.DebugLevel))
	}
	return &WorkerManager{
		store:            store,
		logger:           compLogger,
		workersByFilter:  make(map[string]uint),
		filtersByWorker:  make(map[uint][]byte),
		allocatedWorkers: make(map[uint]bool),
		serviceClients:   make(map[uint]*grpc.ClientConn),
		config:           config,
		proposeFunc:      proposeFunc,
		decideFunc:       decideFunc,
	}
}

Would that be appropriate implementation or would we benefit from another approach (e.g. increasing logging to a custom logging level via <component>LoggingLevel config parameter or totally disabling logging for component)?

[CassOnMars]

Yes, this is along the lines of what I was thinking, albeit it would grow extremely verbose given the number of touch points logging gets injected to.

[PalanQu]

I have an approach, we can define which components are, and add the logLevel config for each component in config.yml, like

p2p:
  logLevel: debug

engine:
  provingKeyId: default-proving-key
  logLevel: info

db:
  path: .config/store
  workerPathPrefix: .config/worker-store/%d

other_component:
  logLevel: info

Create logger and assign the log level before creating each component, and pass the logger in this component.