DeepSpeed-MII
DeepSpeed-MII copied to clipboard
[FEATURE] Access to logits and final hidden layer
Hello, I don't believe there is any way to access logits or the final hidden layer embedding in the current API and would like to request this feature. Thank you
This would definitely make the framework infinitely better than vLLM, which suffers the same issue.