containerd
Add ReaderFrom/WriterTo for Linux (splice)
On Linux, we can use splice to optimize copies to happen in kernel
when copying to other files.
Since this is already a pipe, this should always work when copying
to/from another file.
Updated this, added some new test cases and improved the implementation slightly:
- EINTR handling was breaking out of the loop instead of just retrying - fixed
- Before the max amount that could be copied before the new methods would return was
1<<62... which is a lot... but is not expected since we want to copy until EOF (splice copies 0 bytes), so the internal copy takes a special value,-1, to mean copy to EOF. This is important because we have special handling for when the reader passed in toReadFromis an*io.LimitedReaderand this was originally mixing "amount to copy" with "amount remaining to copy".
FYI, ended up making a new repo to play around with this a bit more, benchmarks, etc: https://github.com/cpuguy83/pipes Somewhat suprisingly I'm seeing an approx 2x speedup using splice.
I've made some changes to the implementation that make it a bit cleaner which I'll move here as well.
Somewhat suprisingly I'm seeing an approx 2x speedup using splice.
Silly question; the benchmark in the readme doesn't show a delta; is that because some option wasn't set?
(Performance improvement sounds great though!)
Hmm I'm not sure. I hadn't used benchstat before... benchcmp shows the delta.
@cpuguy83 @kzys @dmcgowan what's the status on this one? Do we want to have this merged and included in a release?
I went looking what changes are in main that are not yet released (following https://github.com/containerd/fifo/pull/32#issuecomment-1023065555), so was looking if we wanted to include this PR as well if we would be doing a new release.
