[Bug] Old system's toybox tar may not support file larger than 3.5 GB

[404neko]

Should give an option to use busybox first or deprecate toybox completly

[hg42]

busybox does not provide all we need (command line options etc.). Also, toybox interface is more stable.

what is "old systems toybox"? which version?

are you sure, it's not the file system?

[404neko]

The things' beginning is when I trying to restore a backup with NeoBackup, but fail at final, showed EPIPE on notification ( #736 ) Only part of files been restored. I try to decompress the backup archive, but it shows "tar: invalid tar format". I check the backup archive on my pc, it seems the file is not damaged. So I check the progress of decompressing, I found when decompressing a file sized 3.5GB, toybox tar will report "invalid tar format" Then I try busybox tar, all files decompressed successfully. Seems toybox tar can't handle huge file decompress.

The brand of the phone is Motorola, running Android 8.0, /sdcard's partition format is ext4, also the /data/data

So here is the reasons for my suggestion about deprecate toybox

[hg42]

we use an issue template for a reason.

which toybox? use toybox --version

which NB version?

go to NBs terminal tool and press SUPPORT, then add the resulting log file.

It seems the backup worked, then it is unlikely that restore doesn't?

Also tar does not know the file size (because the file is streamed into tar) and it is not interested in it. It simply gets the data from the stream and outputs each file.

So tar is probably not the reason.

You are not very clear about what commands you executed, please show the commands and the resulting output. Decompress would mean gzip -d... Do you mean unpack which would be tar? Is compression enabled?

Note we do not use tar options for compression, instead the archive file is streamed from SAF through decryption (if enabled) and through decompression (if enabled) and streamed to tar, all in parallel, no intermediate files.

If the toybox version really has a problem, we would need the version to add that bug to a list and it would be used for scoring the toybox version. Though the bug would need some more tests.

You can try another toybox... there's a Magisk module. But it may not be chosen, e.g. if it has a lower score because of a known bug and the other gets a higher score. In that case you can put a working toybox on /data/local which should get a higher score (though an apparently bug free version would still get a higher score [may be the /data/local version should be prefererred in all cases])

these toybox variants are tried:

            "/data/local/toybox",
            "toybox-ext",
            "toybox_vendor",
            "toybox-stock",
            "toybox",

[404neko]

I will try my best to provide info you need, I need do a series experiments, it will take some time

[hg42]

go to NBs terminal tool and press SUPPORT, then add the resulting log file

this would help to see the environment, without asking a bunch of questions

for experiments, you should set maxLogLines to 20000 and create such a log after each experiment

[hg42]

assuming you have a compressed archive but not encrypted...

cat archive-file.tar.gz | gzip -d | toybox tar -t

would do the streaming similar to how NB does it (though it doesn't use gzip command), but only show the file names that are in the archive. So you can do this without extracting it to the system and will still see error messages if it breaks.

If busybox tar would show more file names, then I would need to think again. I don't see a way this could happen given how tar works, but you never know.