DeepSpeed-MII
DeepSpeed-MII copied to clipboard
Logits Processors
Is it planned to add a functionality to provide functions (external logits processors) that modify logits based on previously generated tokens? E.g., to apply an output formatter. Something similar to a logits_processors in e.g. vLLM.