Mohammad Siavashi

icon indicating an email [email protected]

icon company KTH University icon location Sweden icon location PhD student in networked and computer systems at KTH University.

Results 1 issues of Mohammad Siavashi

Question: Support for Continuous Batching and Asynchronous Requests

1 comment

Hi. I'm new to this LLM world. I have a few questions regarding the engine. Does it support continuous batching? I'm asking because I'm trying to set a request per...

enhancement