tinyrecord icon indicating copy to clipboard operation
tinyrecord copied to clipboard

Question: Can tinyrecord be used to make TinyDB "process safe"

Open urig opened this issue 3 years ago • 2 comments

Thank you for making tinyrecord free and open source. I would like to ask whether tinyrecord's transaction support can be used to make TinyDB "process safe"?

By "process safe" I mean: If two processes use the same TinyDB database file simultaneously and both processes always use transactions for writing, is it safe to assume no data will be lost?

urig avatar Jun 16 '22 14:06 urig

for process safety -- you would need to share a lock between multiple processes (not sure how you would do that, probably a multiprocessing.Lock), and subclass transaction to change the code.

probably the best way to do it would be for transaction to expose a lock_class attribute, so you can just do this:

class MyTxn(transaction):
    lock_class = MultiProcessLock

with MyTxn(table) as tr:
    ...

eugene-eeo avatar Oct 22 '22 13:10 eugene-eeo

Streamlit is becoming a to-go choice for MVP development with python. In that case, multiple sessions could be open and there would be "multiple clients (each of them single threaded)" concurrency issue.

Looking at this problem, I was thinking about a simple implementation using the filelock library (https://py-filelock.readthedocs.io/en/latest/index.html). It is a locking mechanism based on a physical file. Do you think it is a viable option?

msuguino avatar Jan 08 '24 12:01 msuguino