banyan-cli
banyan-cli copied to clipboard
Unexpected process termination
Summary
dataprep process gets killed on large inputs
Problem
I ran dataprep on a 64 GiB input and recieved the following output:
dataprep pack -i input/w1_d1_s68719476736 -o eh -m m.json
[INFO] 🚀 Starting packing pipeline...
[INFO] 🔍 Deduplicating the filesystem at input/w1_d1_s68719476736
[INFO] 📁 Finding directories and symlinks to back up starting at input/w1_d1_s68719476736
[INFO] 💾 Total number of files to pack: 2
[INFO] 🔐 Compressing and encrypting each file as it is copied to the new filesystem at eh
⠁ [00:00:00] ████████████████████░░░░░░░░░░░░░░░░░░░░ 1/2 Killed
I suspect this indicates that there is a Segfault occurring somewhere
I tested this on our AX101 device.