versatile-data-kit
versatile-data-kit copied to clipboard
Create a basic FASTAPI webservice with one endpoint
We need a fast api webservice with an /ask_question endpoint which should take in a json request body which looks like { "question " : "How many days a year do employees in Ireland have off", "user_group": "HR"} .
For each request we need to
- Embed the question using an embedding model
- Make a request to postgres for all chunks of text which are similar to this embedding
- concat all these chunks of text along with the user question into a single string and send it to the llm to answer.
The final text that is sent to the LLM might look something like
Please answer the customers question? When the answer has been pulled from a confluence chunk of text please also return the link to the article.
> How many days a year do employees in Ireland have off
Here is content pull from confluence that we think will be relevant to answering it.
para 1: (Link to the confluence page)
some txt
para 2: (Link to the confluence page)
other txt
para 3: (Link to the confluence page)
and a final piece of text about holidays in Ireland
defintion of done This only needs to be running locally. It doesn't need to be running in the CICD cluster
example of how it is started (having config file might be easier). Use sensible defaults (e.g default method to Postgres)
./run-demo.py --embedding-api API://xx --ingest-mehod postgres --postgres-config posgres://foo --llm-model llm.com