Why does NOVA perform CoW for append write?

[leftgeek]

It seems that NOVA also perform CoW for append write, is this necessary?

[Andiry]

It does not make any difference. Append write is always writing to new blocks.

Thanks, Andiry

On Wed, Jul 17, 2019 at 9:07 AM kongnull [email protected] wrote:

It seems that NOVA also perform CoW for append write, is this necessary?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/NVSL/linux-nova/issues/73?email_source=notifications&email_token=AAKBYEHPMGIB4KHN2COVNYDP747SDA5CNFSM4IESBJV2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4G7ZDBQA, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKBYEH6Y6CZ3JZHOIENZC3P747SDANCNFSM4IESBJVQ .

[leftgeek]

I mean... If the original file size is not 4KB-aligned, append write can simply overwrite the tail block instead of writing to a new block and copying the original data. Is this right?

[stevenjswanson]

COW should not be necessary for appends. We have been implemented this for some of our recent work, but I’m not sure we released that version. Juno, have we pushed the code for atomic append w/o COW?

-steve

-- Composed on (and maybe dictated to) my phone.

On Jul 17, 2019, at 16:44, kongnull [email protected] wrote:

I mean... If the original file size is not 4KB-aligned, append write can simply overwrite the tail block instead of writing to a new block and copying the original data. Is this right?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[juno-kim]

As far as I know, it is not released yet.

Thanks, Juno

On Wed, Jul 17, 2019 at 8:02 PM Steven Swanson [email protected] wrote:

COW should not be necessary for appends. We have been implemented this for some of our recent work, but I’m not sure we released that version. Juno, have we pushed the code for atomic append w/o COW?

-steve

-- Composed on (and maybe dictated to) my phone.

On Jul 17, 2019, at 16:44, kongnull [email protected] wrote:

I mean... If the original file size is not 4KB-aligned, append write can simply overwrite the tail block instead of writing to a new block and copying the original data. Is this right?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/NVSL/linux-nova/issues/73?email_source=notifications&email_token=AALTYUI5RHNMONE6IU7JBXTP77MNFA5CNFSM4IESBJV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2HFYWI#issuecomment-512646233, or mute the thread https://github.com/notifications/unsubscribe-auth/AALTYUOEKHCDNMCQXX3RAGDP77MNFANCNFSM4IESBJVQ .

[leftgeek]

Thanks, another similar question is that NOVA memsets the rest of the last, partially-written block (the portion beyonds file size) to zero (and flushes it). This seems also unnecessary?

[stevenjswanson]

This is a security issue. If you mmap the file you will get access to the unwritten space. If we don’t zero it, it could contain old data from a previously deleted file.

It also might be a POSIX compatibly issue. I would guess that areas is supposed to be zeroed for mmap.

-steve

-- Composed on (and maybe dictated to) my phone.

On Jul 17, 2019, at 20:26, kongnull [email protected] wrote:

Thanks, another similar question is that NOVA memsets the rest of the last, partially-written block (the portion beyonds file size) to zero (and flushes it). This seems also unnecessary?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[leftgeek]

Thanks a lot. You're right, I found an explanation at http://www.man7.org/linux/man-pages/man2/mmap.2.html

A file is mapped in multiples of the page size. For a file that is not a multiple of the page size, the remaining memory is zeroed when mapped, and writes to that region are not written out to the file.

I guess maybe we can delay the memset operation until a mmap() call?