agnaistic
Idea: Replace old context with summaries
A possible way of improving context memory in long chats would be noticing we're approaching the context limit and proposing (or even doing it automatically in the background) "compressing" a number of old message exchanges by a generated summary.
Say you're 500 interactions in, approaching your token limit (numbers made up). The app notices this and proposes replacing the oldest 200 interactions with one or two paragraphs in the prompt. Sure, some information would be lost, but that was "long ago" anyway.
It would be neat if this didn't actually delete the old messages. They could still be reused when a new summary is needed as the user approaches the context limit again. It would probably work better than using summaries to generate new summaries, although there are limits to this, too...
This idea is not super elaborate, just something I was considering for my own app before I discovered Agnaistic.
Heya, thanks for the idea. There was a recent discussion on discord about supporting some pipeline features and adding the filament project as a submodule in Agnai for people who locally install via the npm package or by cloning.
