DeepSpeed-MII
DeepSpeed-MII copied to clipboard
Does deepspeed-mii support prefix_allowed_tokens_fn?
I use transformers pipeline to generate json dictionaries and I need to specify a prefix_allowed_tokens_fn such that the tokens that can be generated at some steps are fixed. By looking into the source code, it doesn't seem like the deepspeed pipeline support this. Could someone verify whether it is supported please?