google
docstore/gcpdatastore: Add support for Google Firestore in Datastore mode
Is your feature request related to a problem? Please describe.
The docstore API cannot be used with Google Cloud Firestore in Datastore mode.
The docstore/gcpfirestore driver is not sufficient for this -- trying to results in an error:
2019/12/11 19:51:32 could not save foo: docstore (code=FailedPrecondition): rpc error: code = FailedPrecondition desc = The Cloud Firestore API is not available for Datastore Mode projects.
See the attached example program.
It uses GCP application default credentials.
It expects a GCP project name in the GCP_PROJECT_NAME environment variable.
Describe the solution you'd like
A driver implementation for Google Cloud Firestore in Datastore mode.
Describe alternatives you've considered
Using a different backing service or cloud provider.
Additional context
The GCP Firestore documentation mentions Native Mode has a limit of 10k writes/s. This makes it unsuitable for write-heavy workloads at certain scales. While Docstore Mode does not have an exact figure for it's upper writes/s limit it indicates the limit isn't as low.
Quoting: https://cloud.google.com/datastore/docs/firestore-or-datastore
Use Firestore in Datastore mode for new server projects.
Which is why I chose it. Since one would expect this Go package to mostly be used on servers, I think Datastore is something that would benefit many.
Firestore in Datastore mode is the correct mode for most server applications, and is a closer match to the other DBs supported by docstore. This library should use Datastore mode. Firestore in Native mode is for mobile apps and client side Javascript.
https://cloud.google.com/datastore/docs/firestore-or-datastore#choosing_a_database_mode
We recommend the following when choosing a database mode:
- Use Firestore in Datastore mode for new server projects.
Firestore in Datastore mode allows you to use established Datastore server architectures while removing fundamental Datastore limitations. Datastore mode can automatically scale to millions of writes per second.
- Use Firestore in Native mode for new mobile and web apps.
Firestore offers mobile and web client libraries with real-time and offline features. Native mode can automatically scale to millions of concurrent clients.
It looks like the APIs supported for Datastore mode vs Native mode are completely independent (https://cloud.google.com/datastore/docs/firestore-or-datastore#feature_comparison). So this is basically a request for a new docstore/gcpdatastore driver.
