cli icon indicating copy to clipboard operation
cli copied to clipboard

[Bug] pscale dump command with --replica creates filenames with @replica

Open FancyFane opened this issue 1 year ago • 0 comments

Problem Description

When larger tables are dumped using the pscale dump command files are sometimes produced with an @replica designation. This causes issues when trying to import data, as this filename will target replicas for the import, producing an error:

[jason@blackbox pscale_dump_fane_test_main_fane_test_sharded_20240506_170354]$ pscale database restore-dump --dir ./ fane_test test --overwrite-tables
Starting to restore database fane_test from folder ./
⠋ Restoring database ...2024-05-06T17:05:42-05:00	error	error restoring	{"error": "supported only for primary tablet type, current type: replica (errno 1105) (sqlstate HY000)"}
Error: failed to restore database: supported only for primary tablet type, current type: replica (errno 1105) (sqlstate HY000)

Reproduction Steps

  1. Create a database with some reasonable amount of data in it
  2. Run a pscale dump command to export the data including the --replica flag.
pscale database dump --keyspace fane_test_sharded fane_test main --replica
Starting to dump all tables from database fane_test to folder /home/jason/pscale_dump_fane_test_main_fane_test_sharded_20240506_170354
Dumping is finished! (elapsed time: 6.636205788s)
  1. Change directory into the dump location and do a listing note the @replica within the filenames of some files.
[jason@blackbox ~]$ cd pscale_dump_fane_test_main_fane_test_sharded_20240506_170354/
[jason@blackbox pscale_dump_fane_test_main_fane_test_sharded_20240506_170354]$ ls -lah
total 266M
drwxr-xr-x  2 jason jason 4.0K May  6 17:04 .
drwxr-xr-x 45 jason jason  12K May  6 17:03 ..
-rw-r--r--  1 jason jason 130M May  6 17:03 [email protected]
-rw-r--r--  1 jason jason 130M May  6 17:04 [email protected]
-rw-r--r--  1 jason jason 7.2M May  6 17:04 fane_test_sharded.sbtest1.00003.sql
-rw-r--r--  1 jason jason  234 May  6 17:03 fane_test_sharded.sbtest1-schema.sql
-rw-r--r--  1 jason jason  207 May  6 17:03 fane_test_sharded.test-schema.sql
-rw-r--r--  1 jason jason    0 May  6 17:03 metadata
  1. Attempt to import into a test branch:
[jason@blackbox pscale_dump_fane_test_main_fane_test_sharded_20240506_170354]$ pscale database restore-dump --dir ./ fane_test test --overwrite-tables
Starting to restore database fane_test from folder ./
⠋ Restoring database ...2024-05-06T17:05:42-05:00	error	error restoring	{"error": "supported only for primary tablet type, current type: replica (errno 1105) (sqlstate HY000)"}
Error: failed to restore database: supported only for primary tablet type, current type: replica (errno 1105) (sqlstate HY000)

Work Around

There is a work-a-round if this occurs, you can simply rename the files removing the @replica from the filename and the import runs without issues.

[jason@blackbox pscale_dump_fane_test_main_fane_test_sharded_20240506_170354]$ mv [email protected] fane_test_sharded.sbtest1.00001.sql
[jason@blackbox pscale_dump_fane_test_main_fane_test_sharded_20240506_170354]$ mv [email protected] fane_test_sharded.sbtest1.00002.sql
[jason@blackbox pscale_dump_fane_test_main_fane_test_sharded_20240506_170354]$ pscale database restore-dump --dir ./ fane_test test --overwrite-tables
Starting to restore database fane_test from folder ./
Restore is finished! (elapsed time: 1m17.473330261s)

Version

[jason@blackbox ~]$ pscale version
pscale version 0.197.0 (build date: 2024-04-23T15:25:04Z commit: d8b065d)

FancyFane avatar May 06 '24 22:05 FancyFane