httpcore icon indicating copy to clipboard operation
httpcore copied to clipboard

Don't hold lock unless necessary in PoolByteStream.close()

Open leventov opened this issue 4 months ago • 1 comments

Summary

Don't hold lock unless necessary in PoolByteStream.close().

This lock causes significant contention and slowdown in LiteLLM when used with streaming but sync HTTP transport.

Testing: this patch was validated through load testing that decreased lock contention (validated with py-spy) and significantly decreased wall clock completion time of concurrent streaming LLM requests via LiteLLM.

Checklist

  • [x] I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)

leventov avatar Sep 23 '25 11:09 leventov

Maybe related issue:

  • #1029

ilovelinux avatar Nov 17 '25 21:11 ilovelinux