Foreign keys: tracking incoming dependencies

[joepio]

Resources move, change and get deleted. This can be problematic when people depend on these Resources. Atomic Data is largely designed to re-use external content all the time, which means that these external dependencies become more important. For example, when a Commit is parsed and validated, the Properties mentioned in the commit will need to be available (either cached, or fetched). When these are unavailable, the Commit will fail.

A partial solution to this problem is using IPFS #42, which helps make resources immutable. However, this still does not fix the updates itself.

I think that we need two things:

• A protocol for sharing updates (perhaps simplpy posting commits to various commit endpoints)
• A protocol for following things (perhaps also by sharing a commit that has the target resource as a value) (overlap with notifications? #28)

Foreign keys for collection caching / cache invalidation

Calculating a collection can be an expensive endeavor. So ideally, we'd use caching to prevent doing these expensive calulations. One way of approaching this is by keeping track of the collections in which a resource is used. When the resource changes, we can 'invalidate' the cache. This could mean that we have a is-invalid property on cached collections which is set to true whenever an item holding a foreign-key is updated. The function generating the collection can then check for this property, and if it is true, it can skip the expensive step of filtering all the existing resources.

This approach, however, would miss new resources, or resources that first didn't match, but after some commit do match. In other words, when a new resource is added, it will not invalidate a cache, even if it should. For example, the new todos collection would not be invalidated when a new todo is added. What we could do, is for every commit, run all stored collections, and see if they match for that specific resource. Which... Kind of defeats the purpose of using foreign keys for this at all :').

Also, whenever the filters change in a Collection, this should mean that the Collections should be invalidated.

Subscribing to changes using an Endpoint

• When I want to subscribe to changes for Resource X, I find the /subscribe endpoint.
• I enter a couple of query params: resource (the resource to which is subscribed), commitEndpoint (the /commit endpoint to which the changes should be sent. Maybe we could later add subscription levels (e.g. 'delete only').

[joepio]

One of the problems in links between resources, is coming up with names (or creating properties) for the inverted direction. For example, I might have a favoriteBook, which might link back to me with isFavoriteOf. I don't like the idea of modellers having to think about these inverted relationships.

I think it might make sense to have an incoming property, which contains all the property => subject relationships. These relationships can be calculated by the server, which should keep track of incoming references. The property is the same as the outgoing property in the original resource.