rellm
rellm copied to clipboard
r2d4
Exact structure out of any language model completion.
Very interesting library @r2d4 ! I am trying to use the example in the README but with the model being on the GPU (as is required for many of the...
Matching a regex partially can lead to generating a token which causes the whole generated sequence to be invalid, even if a substring of the token would result in a...
I noticed one of the core parts of the strategy is to call generate one token at a time, but I was wondering how slow/fast this is compared to using...
Hello, Thank you for a very interesting solution! Do you have any ideas how `rellm` can be used to make controllable generation for multiple prompts?
