Anjishnu
Anjishnu
#2122 seems to be an interesting feature to add proper support for raising an error message about incorrect input sizes. Would like to take this up at some point if...
_cache_state["logits"] would be a vector of size 32000 lets say if you are considering llama2. If I convert the options to tokens, lets say John=32, Amy=27941, etc. and then use...
Oh good point @marcotcr ! I was not accounting for token healing. If I look at the probabilities of `' dog'` and `' cat'` for your example, then I am...