panic on slice bounds

[meox]

Seems that there is an issue on function

func (fr *regFileReader) Read(b []byte) (n int, err error)

at line 718

Jul 13 11:53:35 ngcore rc.local[1907]: panic: runtime error: slice bounds out of range [:6620516960021273003] with capacity 32768
Jul 13 11:53:35 ngcore rc.local[1907]: goroutine 892 [running]:
Jul 13 11:53:35 ngcore rc.local[1907]: bufio.(*Reader).Read(0xc000790cc0, {0xc0016f8000, 0x12e8, 0xc0013a9bf8})
Jul 13 11:53:35 ngcore rc.local[1907]:         /usr/lib/go/src/bufio/bufio.go:238 +0x2ed
Jul 13 11:53:35 ngcore rc.local[1907]: io.(*teeReader).Read(0xc000135720, {0xc0016f8000, 0x470, 0x8000})
Jul 13 11:53:35 ngcore rc.local[1907]:         /usr/lib/go/src/io/io.go:560 +0x37

[vbatts]

Right on! What were you running when this happened? Do you have a particular reproducer?

[meox]

not easy to reproduce, we got this error while using podman load where file is passed throughout the stadin

[vbatts]

@vrothberg have y'all seen this error before?

[vrothberg]

Thanks for the ping, @vbatts :wave:

I haven't but maybe @mtrmac @nalind or @giuseppe have. @meox can you share the output of podman version?

[meox]

Podman Version:

Client:       Podman Engine
Version:      4.0.3-dev
API Version:  4.0.3-dev
Go Version:   go1.17.10
Git Commit:   717edd7b844dcd66468f5d991991d87e9fc14c12
Built:        Wed Mar  2 16:30:27 2022
OS/Arch:      linux/amd64

[mtrmac]

@meox Do you have the full backtrace?

Either way, this looks like some unreasonable memory corruption.

• How did hdr.Size end up being on the order of 2^64?
• Even assuming the underlying stream was broken, wouldn’t we get to a local slice that had an even larger len, while capacity is 32k? I’d expect the Go runtime to refuse to allocate, and create, such a thing.

[meox]

This is the stacktrace:

panic: runtime error: slice bounds out of range [:6620516960021273003] with capacity 32768
goroutine 892 [running]:
bufio.(*Reader).Read(0xc000790cc0, {0xc0016f8000, 0x12e8, 0xc0013a9bf8})
        /usr/lib/go/src/bufio/bufio.go:238 +0x2ed
io.(*teeReader).Read(0xc000135720, {0xc0016f8000, 0x470, 0x8000})
        /usr/lib/go/src/io/io.go:560 +0x37
io.(*teeReader).Read(0xc000135740, {0xc0016f8000, 0x41638d, 0x8000})
        /usr/lib/go/src/io/io.go:560 +0x37
github.com/vbatts/tar-split/archive/tar.(*regFileReader).Read(0xc0006017b8, {0xc0016f8000, 0xc000022000, 0xc0016f8000})
       xxx/podman/4.0.1+gitAUTOINC+717edd7b84-r0/git/src/.gopath/src/github.com/container>
github.com/vbatts/tar-split/archive/tar.(*Reader).Read(0xc000194000, {0xc0016f8000, 0xc0009bbb20, 0x419654})
       xxx/podman/4.0.1+gitAUTOINC+717edd7b84-r0/git/src/.gopath/src/github.com/container>
io.copyBuffer({0x7f526c4bd240, 0xc0009bbb20}, {0x1c17c00, 0xc000194000}, {0xc0016f8000, 0x8000, 0x8000})
        /usr/lib/go/src/io/io.go:423 +0x1b2
io.CopyBuffer({0x7f526c4bd240, 0xc0009bbb20}, {0x1c17c00, 0xc000194000}, {0xc0016f8000, 0x0, 0x0})
        /usr/lib/go/src/io/io.go:396 +0x3c
github.com/vbatts/tar-split/tar/storage.(*bitBucketFilePutter).Put(0xc0016f8000, {0x0, 0x0}, {0x1c17c00, 0xc000194000})

[mtrmac]

So it’s not hdr.size, it’s the bufio.Reader.w offset https://github.com/golang/go/blob/085c61ae517110168841be0afeb8f883d66fe95a/src/bufio/bufio.go#L238 .

That doesn’t necessarily point at this package at all (for starters, it’s not creating a bufio.Reader).

[mtrmac]

… and not a single reference to line 718 that I could see.

[meox]

Am I the only one with this strange issue?

[vrothberg]

Can you share a reproducer? Or was it a one timer?

[vbatts]

Closing this issue as it's never come up again, and does not have a reproducer. Re-open if we can located a reproducer.