dimitri
row per range or prefetch rows in with section throws error
- [X] pgloader --version
i'm using mac apple silicon, and installed pgloader by brew but not work. and also installed by manual but it doesn't work too same issue
```
pgloader --version
pgloader version "3.6.6"
compiled with SBCL 2.2.6
```
-
[X] did you test a fresh compile from the source tree? YES
-
[X] did you search for other similar issues? yes but not found same issue
issue
if i add rows per range in with section it throws compilation error and prefetch option too. if i remove rows per range and run pgloader command work well i don't know why it is.
my command
load database
from mysql://root:password@localhost:3307/sendymain
into postgresql://postgres:password@localhost:5432/sendymain
with include drop, create tables, create indexes,
workers = 8, concurrency = 1,
multiple readers per thread, rows per range = 50000
SET PostgreSQL PARAMETERS
maintenance_work_mem to '128MB',
work_mem to '128MB'
SET MySQL PARAMETERS
net_read_timeout = '120',
net_write_timeout = '120';
output
0: (TRIVIAL-BACKTRACE:PRINT-BACKTRACE-TO-STREAM #<SB-IMPL::STRING-OUTPUT-STREAM {700C86BB23}>)
1: (TRIVIAL-BACKTRACE:PRINT-BACKTRACE #<CODE-DELETION-NOTE "deleting unreachable code" {700C86BAC3}> :OUTPUT NIL :IF-EXISTS :APPEND :VERBOSE NIL)
2: ((FLET "H0" :IN PGLOADER::MAIN) #<CODE-DELETION-NOTE "deleting unreachable code" {700C86BAC3}>)
3: (SB-KERNEL::%SIGNAL #<CODE-DELETION-NOTE "deleting unreachable code" {700C86BAC3}>)
4: (SB-KERNEL:WITH-SIMPLE-CONDITION-RESTARTS SIGNAL NIL #<CODE-DELETION-NOTE "deleting unreachable code" {700C86BAC3}>)
5: (SB-C:COMPILER-NOTIFY CODE-DELETION-NOTE :FORMAT-CONTROL "deleting unreachable code" :FORMAT-ARGUMENTS NIL)
6: (SB-C::REPORT-CODE-DELETION)
7: (SB-C::%COMPILE-COMPONENT #<SB-C:COMPONENT :NAME "<unknown>" {700C860E33}>)
8: (SB-C::COMPILE-COMPONENT #<SB-C:COMPONENT :NAME "<unknown>" {700C860E33}>)
9: (SB-C::%COMPILE (LAMBDA NIL (LET* ((PGLOADER.SOURCES:*DEFAULT-CAST-RULES* (QUOTE ((:SOURCE (:TYPE #1="int"
...
10: ((FLET "LAMBDA0" :IN "SYS:SRC;COMPILER;TARGET-MAIN.LISP"))
11: ((FLET SB-C::WITH-IT :IN SB-C::%WITH-COMPILATION-UNIT))
12: (SB-C:COMPILE-IN-LEXENV (LAMBDA NIL (LET* ((PGLOADER.SOURCES:*DEFAULT-CAST-RULES* (QUOTE ((:SOURCE (:TYPE
...
13: (COMPILE NIL (LAMBDA NIL (LET* ((PGLOADER.SOURCES:*DEFAULT-CAST-RULES*
...
14: (PGLOADER:RUN-COMMANDS #P"/Users/somsomplayground/migration/database.test.load" :START-LOGGER NIL :FLUSH-SUMMARY T :SUMMARY NIL :LOG-FILENAME NIL :LOG-MIN-MESSAGES NIL :CLIENT-MIN-MESSAGES NIL)
...
15: (PGLOADER::PROCESS-COMMAND-FILE ("database.test.load") :FLUSH-SUMMARY T)
16: (PGLOADER.MONITOR::CALL-WITH-MONITOR #<FUNCTION (LAMBDA NIL :IN PGLOADER::MAIN) {700B5966EB}>)
17: (PGLOADER::MAIN ("pgloader" "--debug" "database.test.load"))
18: ((LAMBDA NIL :IN "/Users/somsom/playground/migration/pgloader-bundle-3.6.6/dumper-2SKVI5f7.lisp"))
19: ((FLET SB-UNIX::BODY :IN SB-IMPL::START-LISP))
20: ((FLET "WITHOUT-INTERRUPTS-BODY-3" :IN SB-IMPL::START-LISP))
21: (SB-IMPL::%START-LISP)
;
; compilation unit aborted
; caught 1 fatal ERROR condition
2022-07-20T11:56:45.028628+09:00 INFO Stopping monitor
What I am doing here?
I kept running into "Heap exhausted during garbage collection" issues and whenever I tried to resolve it by experimenting with options in the "with" section, anytime I would introduce the = character, I would get the mysterious "deleting unreachable code" error.
After some painful debugging, I was able to get it to work by downgrading sbcl to 2.1.9 and building pgloader source. I'm on an Apple M1 MBP. After building, this is my version FWIW:
./build/bin/pgloader --version
pgloader version "3.6.2c52da1"
compiled with SBCL 2.1.9
I was struggling with this error all week as well. I just solved it and came to open an issue myself! What ended up working for me was modifying the default values directly in src/params.lisp and then recompiling from source.
pgloader version "3.6.2c52da1"
compiled with SBCL 2.2.6
