microllama

microllama copied to clipboard

e.g. ```bash pip install spacy python -m spacy download en_core_web_sm ``` and in `Dockerfile`: ```dockerfile RUN pip install spacy RUN python -m spacy download en_core_web_sm ``` In `microllama.py`: ```python from...

tomdyson

Evaluate Pinecone as optional alternative vector store

2

Try out Pinecone as an optional alternative to FAISS. Expected pros: smaller container, lower memory use. Expected cons: slower indexing and querying because of network latency, cost for large document...

tomdyson

- Adds llm and llm-openai as dependencies. - Refactors answer() and streaming_answer() to use llm.get_model() and model.chat() instead of openai.ChatCompletion. - Updates README, Dockerfile, and deploy_instructions() to reflect new dependencies,...

tomdyson

Switch to `llm` for LLM abstraction layer

1

Currently, `microllama` uses `openai` and `langchain` directly for interacting with language models and managing embeddings/vector stores. We should investigate switching to Simon Willison's `llm` library (https://llm.datasette.io/) as a more general...

tomdyson

tomdyson