aries-cloudagent-python icon indicating copy to clipboard operation
aries-cloudagent-python copied to clipboard

Pagination on created schemas and cred defs?

Open dbluhm opened this issue 2 years ago • 3 comments

See: https://github.com/hyperledger/aries-cloudagent-python/blob/anoncreds-rs/aries_cloudagent/anoncreds/issuer.py#L232-L255 And: https://github.com/hyperledger/aries-cloudagent-python/blob/anoncreds-rs/aries_cloudagent/anoncreds/issuer.py#L417-L446

This is similar in behavior to the current interface but as you can see it is using fetch_all to retrieve all of the matching records. In the event your issuer has created lots of schemas/cred defs, this might be a slow operation. Should it use pagination?

dbluhm avatar Jul 31 '23 18:07 dbluhm

Perhaps it is best to just use the other values to narrow the results rather than to paginate?

dbluhm avatar Jul 31 '23 18:07 dbluhm

Are you using pagination for other kind of records in ACA-Py? We had recently a discussion regarding this in an AFJ call and we figured out that, even if it's certainly possible to add offset and limit, special care must be taken in case new records are added between calls.

An issue we have is about sorting: it seems that Indy's WQL was not meant to store a massive amount of data (and possibly Askar inherited this design constraints as well). For server deployments like ACA-Py it's a bit tricky because we would probably like to sort records by creation date or last activity date or whatever, and this will not be possible if using encrypted tags.

genaris avatar Aug 01 '23 21:08 genaris

Askar does a better job at handling large numbers of records because of improvements in indexing and the like. That said, it does still use encrypted tags. Not using encrypted tags is a big change in the security model. Note that there is a way to specifically use unencrypted tags for specific purposes, and then to use those with ordering and filtering. Is that enough of a compromise, or would you wind up with plaintext tags everywhere?

I’ve heard that in AFJ you are considering going to database engine-managed encryption at rest. Any progress on that? What are the ramifications of enabling that support — impact on the security model, compatibility across DB engines in particular.

swcurran avatar Aug 08 '23 14:08 swcurran

Closing in favor of #3001

swcurran avatar Aug 14 '24 22:08 swcurran