structlog icon indicating copy to clipboard operation
structlog copied to clipboard

`CallsiteParameterAdder` use `additional_ignores` for foreign events

Open fraser-langton opened this issue 2 months ago • 0 comments

I was configuring some stdlib logs (eg requests) and I want the log to emit with the callsite from my code not the lib, and so I added additional_ignores=["urllib3", "requests"] alas it doesn't do it if not _from_structlog

https://github.com/hynek/structlog/blob/main/src/structlog/processors.py#L906-L928

    def __call__(
        self, logger: logging.Logger, name: str, event_dict: EventDict
    ) -> EventDict:
        record: logging.LogRecord | None = event_dict.get("_record")
        from_structlog: bool = event_dict.get("_from_structlog", False)

        # If the event dictionary has a record, but it comes from structlog,
        # then the callsite parameters of the record will not be correct.
        if record is not None and not from_structlog:
            for mapping in self._record_mappings:
                event_dict[mapping.event_dict_key] = record.__dict__[
                    mapping.record_attribute
                ]

            return event_dict

        frame, module = _find_first_app_frame_and_name(
            additional_ignores=self._additional_ignores
        )
        for parameter, handler in self._active_handlers:
            event_dict[parameter.value] = handler(module, frame)

        return event_dict

To remedy this I just did my own subclass

class CallsiteParameterAdder(_CallsiteParameterAdder):
    _all_parameters: ClassVar[set[CallsiteParameter]] = set(CallsiteParameter)

    def __init__(
        self,
        parameters: Collection[CallsiteParameter] = _all_parameters,
        additional_ignores: list[str] | None = None,
    ) -> None:
        additional_ignores = additional_ignores or []
        super().__init__(parameters, [__name__, *additional_ignores])

    def __call__(self, logger: logging.Logger, name: str, event_dict: EventDict) -> EventDict:
        frame, module = _find_first_app_frame_and_name(additional_ignores=self._additional_ignores)
        for parameter, handler in self._active_handlers:
            event_dict[parameter.value] = handler(module, frame)

        return event_dict

I am wondering if the comment If the event dictionary has a record, but it comes from structlog, then the callsite parameters of the record will not be correct. in the source is still true, it may be down to how I configure structlog in services but it isn't an issue.

eg here is the stack in structlog.processors.CallsiteParameterAdder

  File "/temp.py", line 16, in <module>
    requests.post("https://google.com")
  File "/.venv/lib/python3.12/site-packages/requests/api.py", line 115, in post
    return request("post", url, data=data, json=json, **kwargs)
  File "/.venv/lib/python3.12/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/.venv/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/.venv/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/.venv/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
    resp = conn.urlopen(
  File "/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 791, in urlopen
    response = self._make_request(
  File "/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 547, in _make_request
    log.debug(
  File "/python/cpython-3.12.7-macos-aarch64-none/lib/python3.12/logging/__init__.py", line 1527, in debug
    self._log(DEBUG, msg, args, **kwargs)
  File "/python/cpython-3.12.7-macos-aarch64-none/lib/python3.12/logging/__init__.py", line 1684, in _log
    self.handle(record)
  File "/python/cpython-3.12.7-macos-aarch64-none/lib/python3.12/logging/__init__.py", line 1700, in handle
    self.callHandlers(record)
  File "/python/cpython-3.12.7-macos-aarch64-none/lib/python3.12/logging/__init__.py", line 1762, in callHandlers
    hdlr.handle(record)
  File "/python/cpython-3.12.7-macos-aarch64-none/lib/python3.12/logging/__init__.py", line 1028, in handle
    self.emit(record)
  File "/python/cpython-3.12.7-macos-aarch64-none/lib/python3.12/logging/__init__.py", line 1160, in emit
    msg = self.format(record)
  File "/python/cpython-3.12.7-macos-aarch64-none/lib/python3.12/logging/__init__.py", line 999, in format
    return fmt.format(record)
  File "/.venv/lib/python3.12/site-packages/structlog/stdlib.py", line 1160, in format
    ed = p(logger, meth_name, ed)  # type: ignore[arg-type]
  File "/.venv/lib/python3.12/site-packages/structlog/processors.py", line 896, in __call__
    traceback.print_stack()

in condensed form you can see how just ignoring logging + additional_ignores suffices

temp.py
requests
urllib3
logging
structlog

fraser-langton avatar Nov 17 '25 07:11 fraser-langton