versatile-data-kit icon indicating copy to clipboard operation
versatile-data-kit copied to clipboard

Create a basic FASTAPI webservice with one endpoint

Open murphp15 opened this issue 2 years ago • 0 comments

We need a fast api webservice with an /ask_question endpoint which should take in a json request body which looks like { "question " : "How many days a year do employees in Ireland have off", "user_group": "HR"} .

For each request we need to

  1. Embed the question using an embedding model
  2. Make a request to postgres for all chunks of text which are similar to this embedding
  3. concat all these chunks of text along with the user question into a single string and send it to the llm to answer.

The final text that is sent to the LLM might look something like

Please answer the customers question? When the answer has been pulled from a confluence chunk of text please also return the link to the article. 
> How many days a year do employees in Ireland have off

Here is content pull from confluence that we think will be relevant to answering it. 

para 1: (Link to the confluence page)
some txt 
para 2: (Link to the confluence page) 
other txt 
para 3: (Link to the confluence page)
and a final piece of text about holidays in Ireland

defintion of done This only needs to be running locally. It doesn't need to be running in the CICD cluster

example of how it is started (having config file might be easier). Use sensible defaults (e.g default method to Postgres) ./run-demo.py --embedding-api API://xx --ingest-mehod postgres --postgres-config posgres://foo --llm-model llm.com

murphp15 avatar Jan 10 '24 13:01 murphp15