glide icon indicating copy to clipboard operation
glide copied to clipboard
verified publisher icon EinStack

🔧 Use ObjectPools for chat request/response schemas

Open roma-glushko opened this issue 1 year ago • 1 comments

Use ObjectPools to optimize memory allocation of

  • chat request/response schema allocations
  • chat stream chunk schema allocations

roma-glushko avatar May 19 '24 20:05 roma-glushko

@roma-glushko Could you assign this one to me? I've got a PR up for the first part, and will start working on streaming.

tom-fitz avatar Jun 12 '24 22:06 tom-fitz