Feature Request: Enhanced Remote Function cache control

[dangodai]

Describe the problem

SvelteKit querys implement client side caching for, as far as I can tell, the following reasons

1. De-duplicating identical requests made during the same page load. This is made necessary by the RF/async design that encourages calling queries many times. There is no problem here.

2. Improving performance by re-using results that we already have.

The problem emerges from reason (2). Fundamentally, client side caching is a difficult form of caching (which can already be a difficult problem) which many organizations choose to avoid. The trade off between performance and data correctness quite often favors the latter.

SvelteKit only gives a few options regarding the cache right now: Implicit invalidation via reference counting, explicit invalidation with .refresh(), and invalidation (implicit or explicit) following commands or form submissions.

This becomes a problem when client side navigation results in a cache hit and there is no built in option to fetch fresh data. Imagine some of the following scenarios

• The root layout requests currentUser(). Some client other than the current one (such as an admin, or the same user in another tab) subsequently modifies the user. The user then navigates to a page under that layout such as /profile/edit which is backed by currentUser() and is given a form with stale data.
• The layout displays the user's assignments for the day (or perhaps just a count, backed by the same query). Their manager gives them a new assignment. The user navigates to a dedicated page that shows their daily assignments, and doesn't see the new one.
• The layout shows a user's account balance, getBalance(). They open some products in new tabs and make a purchase in one of them, causing the account balance to change. They return to the original tab, and navigate to a page like /profile which shows getBalance() and are presented with a stale value.

I think the pattern is clear and comes up often (roles, notifications, etc.). There are likely other common causes, but this is the one I have been running into.

In a traditional website user's expect a page navigation to serve fresh results. When using SSR (CRS disabled) this is not a problem. When queries are not kept onscreen by a parent layout this is not a problem. When using load functions, this is not a problem (although keeping duplicate calls in sync is a problem). I think that the default approach by SK here leads you into caching problems that betray the user expectations.

While the context of Redis is different, typically being used by two application servers that can maintain a persistent connection, their article on client side caching may be good reading to demonstrate the issues and approaches: https://redis.io/docs/latest/develop/reference/client-side-caching/

Describe the proposed solution

I don't believe the tracking approach used by Redis is practical for SK between serverless deployments and the different in potential scale of open connections. However, the following options would provide meaningful control to SK users so they can make their own caching decisions:

1. Support invalidating queries that are added to the page during navigation. Controls to opt in/out of this per query may be desirable, as some requests are known to be okay to cache (such as immutable data, or data that can be presented stale).
2. Support TTLs
3. Have .refresh() return the query result instead of Promise<void>. This would allow parts of an application that want to always bypass the cache to do so (but requires careful coordinate from SK to still handle de-duplication as it does today), e.g.

<div>
    My up to date balance is {await getBalance().refresh()}
</div>

4. Support fully opting out of the client side cache, aside from duplicate requests inside of a batch of course.

Alternatives considered

Some alternatives considered or suggested to me by the helpful folks in the discord community

1. Nuclear option: Spam refreshAll() inside of a navigation hook. This works, but lacks any granularity. It is also not the most obvious way to opt out of caching.
2. Add an extra property to query arguments, such as the current route, to fetch fresh data when that value changes, e.g currentUser(page.route.id). This works, but requires bastardizing your API to work around the cache, as the function arguments and the cache key are one and the same.

Thanks for any consideration, or recommendation for alternative approaches.

Importance

would make my life easier

Additional Information

No response

[phi-bre]

Since the title doesn't specifically narrow the feature to just the client side, I would like to mention server-side caching as well.

While I'm not particularly worried about staleness during nested server side remote function calls, I do worry about memory usage, as ALL function call results are kept in memory for the entire duration of the request with no option to clear them or have them garbage collected at all. Which may not be a problem with short lived CRUD operations, but if you load data to populate an LLM prompt and wait for its response, the "request" can live for up to a minute or more.

[yayza]

Wouldn't a utility function like this work for all three scenarios:

export async function fresh<T>(query: RemoteQuery<T>) {
    if (browser && query.ready) {
        await query.refresh();
    }
    return await query;
}

eg. in a component where you always want the latest data

<script lang="ts">
    import { getCurrentUser } from "./user.remote";

    fresh(getCurrentUser());
</script>

{(await getCurrentUser()).balance}