bug: Guardrail server failing when streaming is enabled
Did you check docs and existing issues?
- [x] I have read all the NeMo-Guardrails docs
- [x] I have updated the package to the latest version before submitting this issue
- [ ] (optional) I have used the develop branch
- [x] I have searched the existing issues of NeMo-Guardrails
Python version (python --version)
Python 3.12.5
Operating system/version
MacOS 15.3.2
NeMo-Guardrails version (if you must use a specific version and not the latest
No response
Describe the bug
models:
- type: main engine: openai model: gpt-4.1-2025-04-14 parameters: temperature: 0.2 stream: true
for this config trying to work with the nemoguardrails server with stream: true it is failing with the issue below:
File "/Users/Swayam/Documents/Projects/pandemos/client/.venv/lib/python3.12/site-packages/nemoguardrails/library/self_check/input_check/actions.py", line 72, in self_check_input response = await llm_call(llm, prompt, stop=stop) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/Swayam/Documents/Projects/pandemos/client/.venv/lib/python3.12/site-packages/nemoguardrails/actions/llm/utils.py", line 122, in llm_call raise LLMCallException(e) nemoguardrails.actions.llm.utils.LLMCallException: LLM Call Exception: 'AsyncStream' object has no attribute 'model_dump'
Steps To Reproduce
- Create an OpenAI config with stream true
- Start the nemoguardrails server
- Try to ask any query or use a custom openai client with local base url to ask AsyncOpenAI streaming queries.
Expected Behavior
- Streaming response should be returned neatly.
Actual Behavior
- Internal server error occurs.