mydumper OOM when dumping a 200 GB database from TiDB

[kennytm]

Command line:

./mydumper \
    --outputdir=backup \
    --host 127.0.0.1 -P 4000 -u root \
    --long-query-guard=3600 \
    --tidb-force-priority=LOW_PRIORITY \
    --verbose=3 \
    --regex "^(?!(INFORMATION_SCHEMA|PERFORMANCE_SCHEMA|METRICS_SCHEMA|INSPECTION_SCHEMA))" \
    -t 8 -F 64 --skip-tz-utc --compress-protocol

(Memory size: 16 GB.)

[kennytm]

Not quite reproducible locally. mydumper is constantly using 12 MB of memory. Will try again tomorrow.

[IANTHEREAL]

while dumping from TiDB, TiDB often OOM. The related issue https://github.com/pingcap/tidb/issues/16104

[kennytm]

The RAM usage seems to be schema-dependent. On Alpine Linux the following schema caused mydumper to use up to 600 MiB of RAM:

• 22 × bigint
• 19 × int
• 12 × varchar(255)
• 3 × varchar(30)
• 2 × varchar(10)
• 2 × timestamp
• 1 × json

[IANTHEREAL]

mydumper -t 4: 600 MiB vs dumpling -t 4: 33 MiB (Because of TIDB OOM, no export has been successful.)

[kennytm]

Not reproducible with generated data on Debian, mydumper only uses roughly 5.5 MB of memory.

Similarly Dumpling is using roughly 9.5 MB of memory (RSS).

Trying to use Alpine next. Maybe musl problem again.

---

Edit: Not reproducible on Alpine 3.11.6. Going to try https://hub.docker.com/r/pingcap/tidb-backup-manager next.

Edit 2: Not reproducible using tidb-backup-manager.