DeepSpeed-MII icon indicating copy to clipboard operation
DeepSpeed-MII copied to clipboard

Logits Processors

Open psitronic opened this issue 1 year ago • 0 comments

Is it planned to add a functionality to provide functions (external logits processors) that modify logits based on previously generated tokens? E.g., to apply an output formatter. Something similar to a logits_processors in e.g. vLLM.

psitronic avatar Oct 21 '24 08:10 psitronic