Renaming of tmp folder to data fails due to antivirus scans on Windows

[kieranjol]

We have been bagging on network shares lately using bagit-python as opposed to working on local forensic workstations. As a result, some new issues have arisen, largely with bagit failing due to not having permission to rename the tmp folder to 'data'. We believe that this is due to antivirus scans (Windows Defender) on the network placing a lock on the rename process for a few seconds. PermissionError: [WinError 5] Access is denied: 'X:\\storage\\workingFiles\\bag_tests\\pythonforktest\\000929796\\tmpe1ehkzxc' -> 'data' Looking at old issues, I can see that this error has popped up here before: https://github.com/LibraryOfCongress/bagit-python/issues/134 so it might be a bit more widespread than I would have initially thought.

I have added some try/except code to a local fork of bagit-python and it has been handling these issues really well. Anecdotally, the issue arises for us about 20% of the time in a test of about 400 bags totalling 100gigs, and usually a retry after 10 seconds fixes the issue, with a rare few instances needed two retries. In the issue linked to above, this 'retry' solution appears to have worked as well as they were able to run bagit successfully again after the initial fail.

I will follow up with the pull request.

Best,

Kieran O'Leary Digital Repository Services Manager National Library of Ireland

[acdha]

Thanks, that's an interesting challenge since it's technically an environmental problem but a very real one anyone on Windows could encounter. Your code is very simple so it's probably better than adding a dependency on stamina but the Python packaging situation has improved massively since @edsu was originally working on bagit.py and I wonder whether we might want to start using retries with backoff, etc. to handle other errors as well. I'm thinking that the answer might be to merge #192 now but consider how often it is the case that a transient permission or I/O error could be retried within a modest period of time as I know people have had issues with things like network filesystems in the past, too.

[kieranjol]

Thank you so much for reviewing and merging!

On Thu 14 Aug 2025 at 14:13, Chris Adams @.***> wrote:

acdha left a comment (LibraryOfCongress/bagit-python#191) https://github.com/LibraryOfCongress/bagit-python/issues/191#issuecomment-3188411904

Thanks, that's an interesting challenge since it's technically an environmental problem but a very real one anyone on Windows could encounter. Your code is very simple so it's probably better than adding a dependency on stamina https://pypi.org/project/stamina/ but the Python packaging situation has improved massively since @edsu https://github.com/edsu was originally working on bagit.py and I wonder whether we might want to start using retries with backoff, etc. to handle other errors as well. I'm thinking that the answer might be to merge #192 https://github.com/LibraryOfCongress/bagit-python/pull/192 now but consider how often it is the case that a transient permission or I/O error could be retried within a modest period of time as I know people have had issues with things like network filesystems in the past, too.

— Reply to this email directly, view it on GitHub https://github.com/LibraryOfCongress/bagit-python/issues/191#issuecomment-3188411904, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAITFPWL7WGLVYLYTE4XE5L3NSDRJAVCNFSM6AAAAACD4GISX2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTCOBYGQYTCOJQGQ . You are receiving this because you authored the thread.Message ID: @.***>