[QUESTION]: How can I log intermediate LLM Responses

[Powerkrieger]

Question

I get an "LLM Response has improper format" error from my extractor step in the pipeline, but I dont really know what the model actually responded with. Is there a way to tell the pipeline to increase the logging level to show the LLM responses?

[stellasia]

Hey @Powerkrieger ,

So it seems you can get access to the raw response with this process:

1. Set on_error="IGNORE"
2. Set log level to DEBUG (example here)

By default, logs are truncated so that they are still readable. You can configure this by settings the two env vars: LOGGING__MAX_LIST_LENGTH and LOGGING__MAX_STRING_LENGTH (default values are 5 and 200 respectively). Please note that these config are undocumented features, so we might change the behavior in the future.

[Powerkrieger]

Thanks @stellasia , you seem to always respond with constructive solutions.

Im kindof wondering why the improperly formatted response is not printed in the RAISE scenario, by default. That, along with the missing documentation, makes it really hard to understand whats going wrong. And just printing it in that scenario would probably solve some frustration. If its because that is normaly a sign of a LLM that is too weak, then that could also be mentioned in the error message.