nodeSavePageWE
nodeSavePageWE copied to clipboard
Failure when scraping certain URLs
I'm able to scrape most URLs successfully, but some like https://about.gitlab.com/ are failing and hanging with the following traceback:
$ node --trace-warnings index.js
(node:17868) UnhandledPromiseRejectionWarning: Error: Protocol error (Runtime.callFunctionOn): Execution context was destroyed.
at /test_save_page/node_modules/puppeteer/lib/cjs/puppeteer/common/Connection.js:208:63
at new Promise (<anonymous>)
at CDPSession.send (/test_save_page/node_modules/puppeteer/lib/cjs/puppeteer/common/Connection.js:207:16)
at ExecutionContext._evaluateInternal (/test_save_page/node_modules/puppeteer/lib/cjs/puppeteer/common/ExecutionContext.js:200:50)
at ExecutionContext.evaluate (/test_save_page/node_modules/puppeteer/lib/cjs/puppeteer/common/ExecutionContext.js:106:27)
at DOMWorld.evaluate (/test_save_page/node_modules/puppeteer/lib/cjs/puppeteer/common/DOMWorld.js:79:24)
at processTicksAndRejections (internal/process/task_queues.js:93:5)
at emitUnhandledRejectionWarning (internal/process/promises.js:168:15)
at processPromiseRejections (internal/process/promises.js:247:11)
at processTicksAndRejections (internal/process/task_queues.js:94:32)
(node:17868) Error: Protocol error (Runtime.callFunctionOn): Execution context was destroyed.
at /test_save_page/node_modules/puppeteer/lib/cjs/puppeteer/common/Connection.js:208:63
at new Promise (<anonymous>)
at CDPSession.send (/test_save_page/node_modules/puppeteer/lib/cjs/puppeteer/common/Connection.js:207:16)
at ExecutionContext._evaluateInternal (/test_save_page/node_modules/puppeteer/lib/cjs/puppeteer/common/ExecutionContext.js:200:50)
at ExecutionContext.evaluate (/test_save_page/node_modules/puppeteer/lib/cjs/puppeteer/common/ExecutionContext.js:106:27)
at DOMWorld.evaluate (/test_save_page/node_modules/puppeteer/lib/cjs/puppeteer/common/DOMWorld.js:79:24)
at processTicksAndRejections (internal/process/task_queues.js:93:5)
(node:17868) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
at emitDeprecationWarning (internal/process/promises.js:180:11)
at processPromiseRejections (internal/process/promises.js:249:13)
at processTicksAndRejections (internal/process/task_queues.js:94:32)
I'm new to the JS ecosystem, so not sure how to debug further.