Chris Bates

No matter what I try I can't set the context_length of a GPTQ model. It's overridden by ExLLAMA, which then sets the cache size and context_length whatever it set as...

### Describe the issue as clearly as possible: After doing pip uninstall there's a bunch of numba jit stuff left behind in ``` /python3.11/site-packages/outlines/fsm/__pycache__/ /python3.11/site-packages/outlines/text/__pycache__/ ``` I noticed this when...

Support Chat Templates as a parameter to the generate methods

9

### Presentation of the new feature When using Outlines on various open models available on Huggingface I have often needed to add various chat template tags to the prompt I...

### What behavior of the library made you think about the improvement? Related to #756 (chat template support) At the moment generate.text requires manually passing in a stop_at parameter which...

stop_at doesn't stop generation early when using Exllamav2

3

### Describe the issue as clearly as possible: When I add a stop_at token to outlines.generate.text it seems to generate all tokens behind the scene, and then strips anything after...