EXTREMELY EARLY DRAFT:: Grules / Dynamic Routing

[dave-gray101]

Pending the repair of channel refactor

This is a very early draft of some of my latest experiments for CI reasons // linking people demonstrations. It is not anywhere near ready for review // release, and this message will be updated eventually.

This feature branch is for a new dynamic routing layer. This is currently imagined as an advanced feature, potentially disabled by default. By introducing the grules rule engine, we will be able to have dynamic control between receiving a request and dispatching it to a backend.

Some examples of where I want to take this functionality:

• Dynamic API Key Processing - #2046 - rules will know what the key was, endpoint, source etc.
• Backend Memory Management - see temporary file backend_rules_onlytwo.json, the ability to unload backends deterministically
• Request Rewriting - Especially combined with the above two features, the core reason behind this work is the ability to have a request come in over HTTP, and if it's not immediately serviceable, enqueued onto MQTT for later processing
• "Rules Engine Backend" - Why use a rule engine instead of a scripting language / hardcoding all of this? I have another idea in mind which is to expose the grule system as its own "retro ai backend" - since we already have pretty effective JSON generation capabilities, something that combines our store endpoints and grules may provide a simple way to afford models symbolic manipulation tools.

[netlify[bot]]

✅ Deploy Preview for localai canceled.

Name	Link
🔨 Latest commit	ef5b82434859cc80ab7c01f60b41cd64f9ff63df
🔍 Latest deploy log	https://app.netlify.com/sites/localai/deploys/661eae90267a6d0008b9fc6f