Parth Sareen

Remove langchain, use ollama api

1

I had written this at a time where the Ollama api didn't exist. There is a lot of bloat around langchain and I'd like to get to something a bit...

Fixes: https://github.com/ollama/ollama/issues/7993

envconfig: allow setting context length through env var

9

From offline discussions: - Current API parameters improperly mix system and request level configurations - For now, let users set a context length through `OLLAMA_CONTEXT_LENGTH` - Users can still set...

sample: add sampling package for new engine

2

This package introduces a first pass at the sampler for the new engine. ## Usage ```go sampler, err := sample.NewSampler( req.Temperature, req.TopK, req.TopP, req.MinP, req.Seed, ) token, err := seq.sampler.Sample(logits[seq.iBatch*vocabSize...

- [ ] update Sampler interface to unmarshal json into sampling options - [ ] Error handling on unmarshal - [ ] update transform interface to be private

- [ ] rm uv sampler - [ ] rm rand - [ ] rm f64 transforms -> f32s - [ ] default seed should be -1

follow up to https://github.com/ollama/ollama/pull/8938

WIP tps - in isolation + temperature greedy: 125 - no temp apply top k: 78 -> 84 top p: 44 -> the equivalent 50% drop is noticed in the...

removed transforms from greedy sample to allow testing with the sampler package