Check blocks are present during backup
a21737b8e89456040089858c6b35e2b35f12d906 changed Conserve so that it assumes that blocks referenced by previous indexes are present in the archive. That helps performance, but it does mean that blocks that are missing in the previous band will never get re-added by a later band: the files just remain missing, which is not great.
Perhaps this should be reverted, and instead Conserve should check the blocks are present before adding a new reference to them, perhaps with an in-memory cache of which ones have been seen. #106
If the previous index references an out-of-range address that won't be caught, but perhaps that's less likely: it requires a bug in the old version, not just loss of files.
Another possibility: add a --thorough option that could:
- Always read the content of files, rather than checking their timestamp.
- Always check the blocks are present, rather than assuming referenced blocks are present.
However, perhaps this puts the work on the user to run validate and interpret the results, and it might be better to just do the right thing without supervision.