litefs icon indicating copy to clipboard operation
litefs copied to clipboard

Allow import via file copy (initial creation only)

Open benbjohnson opened this issue 3 years ago • 5 comments

It would be nice to allow users to simply copy an existing database file into LiteFS. Currently, this doesn't work as LiteFS only calculates checksums after transactions and the file copy doesn't operate transactionally.

It could be possible to implement this as long as there are no other users using the database at the same time. LiteFS can enforce the SHARED & RESERVED file locks during copy to ensure no other processes begin using a partially written database.

benbjohnson avatar Nov 18 '22 21:11 benbjohnson

This is trickier than one would think. This has been implemented as a command instead (ltx import).

benbjohnson avatar Dec 28 '22 20:12 benbjohnson

Hi @benbjohnson we have an application which allows importing its database on first startup.

I hoped this would be automatic on file write/replace, but with import command, application must me developed locally with LiteFS and be LifeFS aware.

Can you reconsider the automatic import, please? Is there any blocker that it isn't possible?

morki avatar Jan 18 '23 14:01 morki

@morki I think if this was restricted to only allow import on initial creation then it might not be as difficult. I'll reopen and reevaluate it.

benbjohnson avatar Jan 28 '23 18:01 benbjohnson

I didn't want to create another issue, but while using litefs import the journal_mode wasn't set to wal after import. I'm not sure if it was related or not, but the primary was also read-only at the time.

I manually set the journal_mode on the db pre-import and then the primary was writable and journal_mode was wal post-import. Not sure if that was the root cause, but I figured I mention it as a potential edge-case.

tjheeta avatar Apr 28 '23 23:04 tjheeta

@tjheeta hmm, that seems odd. I'm not sure how or why LiteFS would switch the journal mode. The only thing I could think of is if you did a VACUUM then that can reset the journal mode back to the rollback journal. Can you go ahead and open a separate issue for this? I can investigate.

benbjohnson avatar Apr 30 '23 02:04 benbjohnson