offix icon indicating copy to clipboard operation
offix copied to clipboard

Performance tests of the datastore

Open wtrocki opened this issue 5 years ago • 5 comments

Performance tests of the datastore (fetching data by specific id) etc.

wtrocki avatar Jun 18 '20 11:06 wtrocki

All query operations are performed on-device storage so, it's generally fast, plus fetching by id uses the "id" index. Advanced filtering capabilities require O(n) time on Indexeddb where "n" is the number of items in the local database. This could be a problem is n becomes large. On Indexeddb, you can only query by indices and Index key ranges. No advanced query capabilities are available.

How large do you reckon "n" will get? Large enough to cause a performance issue? What do we do when the local database gets too large?

Eunovo avatar Jun 19 '20 16:06 Eunovo

Every operation that involves IndexedDB is generally very very slow. That is why people tend to have all db in memory but that brings the replication challenges etc.

How large do you reckon "n" will get? Large enough to cause a performance issue? What do we do when the local database gets too large?

From my experience, it is nothing unusual to have 30mb dataset. I would not expect this to be resolved for the datastore. I will be checking how we can architecture it to avoid this problems first.

wtrocki avatar Jun 19 '20 16:06 wtrocki

@wtrocki On the matter of performance in relationships, I think we can speed up an operation like the one we have below by indexing the userId field on Comment

user = User.query(where: { id: "..." })
Comment.query(where: { userId: user.getId() })

If we index all relationship fields, we will just modify our filters to use those indices. PS: Currently, the filters are not utilizing any index. Not even the id index. There will be separate issue and PR for this

Eunovo avatar Jun 24 '20 12:06 Eunovo

Yes. That will be good start.

wtrocki avatar Jun 24 '20 12:06 wtrocki

This issue is not immediately needed but important to factor

wtrocki avatar Jun 24 '20 12:06 wtrocki