Using this library in a Kubernetes environment

[robross0606]

We've been exploring use of this library in a Kubernetes (K8s) environment, but the choice of Sqlite as a back end is possibly preventing that use:

• Scaling in a K8s environment involves using multiple "pods" with persistent "shared volume" storage.
• Kubernetes supports very few "shared volumes" that work in multi-node clusters and all of them are somewhat nfs-based.
  • nfs autoprovisioner
  • glusterfs
  • rook.io has a shared mode I haven't tried
• SQLite breaks significantly with NFS, and very easily with multiple processes accessing it (as this setup would require) See:
  • https://stackoverflow.com/questions/29244788/error-disk-i-o-error-on-a-newly-created-database
  • https://stackoverflow.com/questions/9907429/locking-sqlite-file-on-nfs-filesystem-possible
  • https://stackoverflow.com/questions/5043327/how-to-ensure-a-safe-file-sync-with-sqlite-and-nfs

Was any thought given here to this type of environment? I'm curious why sqlite was chosen over directly using the file system for persistence. Any thoughts on alternate back ends?

[damoclark]

The library was never intended for a distributed environment. It was built to serve a Node Red msg-queue node. From memory, I considered direct filesystem storage, but using sqlite saved me from all the low-level implementation details for maintaining a reliable persistent storage queue.