strands-agents
[BUG] OpenAI Implementation does not support passing custom HTTP client
Checks
- [ ] I have updated to the lastest minor and patch version of Strands
- [x] I have checked the documentation and this is not expected behavior
- [x] I have searched ./issues and there are no duplicates of my issue
Strands Version
1.11.0
Python Version
3.11.13
Operating System
15.7.1
Installation Method
pip
Steps to Reproduce
import asyncio
import httpx
from strands import Agent
from strands.models.openai import OpenAIModel
client = httpx.AsyncClient()
agent = Agent(
model=OpenAIModel(
model_id="gpt-4o-mini-2024-07-18",
client_args={
"api_key": "your-openai-key",
"http_client": client
}
)
)
async def chat(prompt: str):
result = await agent.invoke_async(prompt)
print(result)
async def main():
await chat("this works")
await chat("this doesn't work")
if __name__ == "__main__":
asyncio.run(main())
Expected Behavior
As per the documentation, I should be able to pass any parameters supported by the OpenAI client in the constructor of the OpenAIModel.
That should include the http_client parameter, that let me pass a custom HTTPx client to the OpenAI client, as documented in here.
Actual Behavior
When passing a custom HTTPx client, only the first request will pass through, because that client will be automatically closed by Strands right after processing it. Subsequent requests fails.
Additional Context
No response
Possible Solution
No response
Related Issues
No response
We have a note about this in the source:
We initialize an OpenAI context on every request so as to avoid connection sharing in the underlying http client. The asyncio event loop does not allow connections to be shared. For more details, please refer to https://github.com/encode/httpx/discussions/2959.
With that said, the recommendation here is to make sure your custom client is compatible with the following openai call:
async with openai.AsyncOpenAI(**self.client_args) as client:
...
Hi @pgrayy ,
I'm not sure what you mean by "compatible custom client". The custom HTTP client I'm passing is a param (http_client) that is part of the list of client_args used to initialized the OpenAI AsyncOpenAI, as the code show. This parameter is officially supported by OpenAI SDK, see here for an example on how to use it.
The problem is that in Strands, the same list of client_args is passed on each iteration to instantiate the OpenAI Client. But on the second call, the custom HTTP client it contains will be in a closed state, hence why it fails.
How OpenAI recommend to do this is to reinstantiate the HTTP client on every turn, though Strands does not allow me to do that. For instance, if I could pass a HTTP client factory instead of a httpx.AsyncClient, it might be possible. But right now, I don't have a way to pass a custom HTTP client to OpenAI with Strands.
The factory approach would be our recommendation for now. For http_client, you could pass in a factory that returns a new instance of your client on each invocation. I'll see if I can come up with an example.
So a factory pattern wouldn't work here unfortunately. With the OpenAIModel implementation as given, the only option right now would be to derive an httpx.AsyncClient that doesn't close:
class CustomClient(httpx.AsyncClient):
...
Other solutions would require some updates in Strands that we would need to flesh out. Some ideas include:
- Allow customers to update model config in a BeforeModelCallEvent hook.
- Support a factory pattern for the http_client in the OpenAIModel provider.
I think that factory pattern does make the most sense. I'll bring it up with the team. Thank you for raising the issue.
Sounds good, let me try to disable closing the client then, I'll keep you posted, and +1 for the suggested updates in Strands.
Ok thanks Patrick, instantiating a custom "unclosable" httpx client does the trick, though I think we both agree that the factory approach would be the more reliable in the long-term, to prevent other failures like this. Should we keep this issue opened until we come up with a better solution?
Had same issue. Custom unclosable client fixed the issue. waiting for official fix. Thanks
