DeepSpeed-MII icon indicating copy to clipboard operation
DeepSpeed-MII copied to clipboard

[FEATURE] Access to logits and final hidden layer

Open lshamis opened this issue 1 year ago • 1 comments

Hello, I don't believe there is any way to access logits or the final hidden layer embedding in the current API and would like to request this feature. Thank you

lshamis avatar Apr 17 '24 17:04 lshamis

This would definitely make the framework infinitely better than vLLM, which suffers the same issue.

SulRash avatar May 07 '24 12:05 SulRash