dattobd icon indicating copy to clipboard operation
dattobd copied to clipboard

Will the system memory exhaust?

Open zhanghaiyang9999 opened this issue 4 years ago • 3 comments

After creating the snapshot, if the original volume data changes very frequently, will the system memory be exhausted? Because when one bio comes, need to clone one bio and add the bio to the queue via bio_queue_add.

If the cow file writing is slower than the original volume writing speed, the system memory must be used exhaust and cause system crash, right?

zhanghaiyang9999 avatar Oct 07 '21 03:10 zhanghaiyang9999

yes, but there is a limit. a given block will only be cowed once. So for example, the superblock gets rewritten to many times, it's only going to end up in the cow file the first time. that said if you take the worse case scenario and rewrite the entire disk while taking a snapshot, yes you could run into that problem, but you'll blow out the cow file before that happens.

nixomose avatar Oct 07 '21 11:10 nixomose

thanks, you are right, but maybe no need to rewitte entire disk, for example just rewrite 10G data, maybe enter the worse case? -------- 原始信息 --------发件人: nixomose @.> 日期: 2021/10/7 19:44 (GMT+08:00) 收件人: datto/dattobd @.> 抄送: zhanghaiyang @.>, Author @.> 主题: Re: [datto/dattobd] Will the system memory exhaust? (#261) yes, but there is a limit. a given block will only be cowed once. So for example, the superblock gets rewritten to many times, it's only going to end up in the cow file the first time. that said if you take the worse case scenario and rewrite the entire disk while taking a snapshot, yes you could run into that problem, but you'll blow out the cow file before that happens.

—You are receiving this because you authored the thread.Reply to this email directly, view it on GitHub, or unsubscribe.Triage notifications on the go with GitHub Mobile for iOS or Android.

[ { @.": "http://schema.org", @.": "EmailMessage", "potentialAction": { @.": "ViewAction", "target": "https://github.com/datto/dattobd/issues/261#issuecomment-937713753", "url": "https://github.com/datto/dattobd/issues/261#issuecomment-937713753", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { @.": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

zhanghaiyang9999 avatar Oct 07 '21 12:10 zhanghaiyang9999

yeah. We have seen the effect you are talking about but we've never run into the case where we ran out of memory before we ran out of cow file. maybe you could contrive such a situation, but we've never seen it in normal use.

nixomose avatar Oct 07 '21 12:10 nixomose