LitServe
LitServe copied to clipboard
dynamic inference worker
Allow LitServer to send requests to either streaming or non-streaming inference worker dynamically based on the request.
cc: @lantiga