feat: Support SQLite as Offline Store

[HaoXuAI]

Is your feature request related to a problem? Please describe. Support sqlite database as batch data source for the offline store.

Describe the solution you'd like SQLite as an offline store.

Describe alternatives you've considered N/a

Additional context N/a

[franciscojavierarceo]

We already support this. Do you have something more specific you want?

[tokoko]

Do we? pretty sure Hao is referring to sqlite offline store engine. We don't currently have that, sqlite is only used on the online side.

[HaoXuAI]

Yep, i'm talking about sqlite as data source for offline store. Actually it's bit confusing that data source is in some way equals to offline store. I feels it should be a abstraction that can be used for both offline store and online store? @franciscojavierarceo @tokoko

[franciscojavierarceo]

Yeah that makes sense! And I do agree data source is weird to be limited exclusively to offline store.

[tokoko]

@franciscojavierarceo @HaoXuAI Before we go too far here, let me dig deeper in the use case. We already support duckdb which can be made to work even for local testing w/o extra infrastructure similar to how sqlite works. What do we gain by adding sqlite?

[franciscojavierarceo]

@tokoko

1. Feast is meant to be flexible, so if someone chooses to contribute an online or offline store our bias should be to accept the implementation and make sure it works before merging.

2. SQLite is one of the most popular local databases and it comes out of the box without any necessary installations, so supporting it seems rational for local development.

As LLMs become increasingly local, (2) plays a bigger role.

Separately, I do agree DuckDB seems like a tool we should certainly advise users towards.

[HaoXuAI]

Yeah I think it's still worth adding the SQLite, probably the most simple db in python world :)

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.