nodeSavePageWE icon indicating copy to clipboard operation
nodeSavePageWE copied to clipboard

Failure when scraping certain URLs

Open kristallizer opened this issue 5 years ago • 0 comments

I'm able to scrape most URLs successfully, but some like https://about.gitlab.com/ are failing and hanging with the following traceback:

$ node --trace-warnings index.js
(node:17868) UnhandledPromiseRejectionWarning: Error: Protocol error (Runtime.callFunctionOn): Execution context was destroyed.
    at /test_save_page/node_modules/puppeteer/lib/cjs/puppeteer/common/Connection.js:208:63
    at new Promise (<anonymous>)
    at CDPSession.send (/test_save_page/node_modules/puppeteer/lib/cjs/puppeteer/common/Connection.js:207:16)
    at ExecutionContext._evaluateInternal (/test_save_page/node_modules/puppeteer/lib/cjs/puppeteer/common/ExecutionContext.js:200:50)
    at ExecutionContext.evaluate (/test_save_page/node_modules/puppeteer/lib/cjs/puppeteer/common/ExecutionContext.js:106:27)
    at DOMWorld.evaluate (/test_save_page/node_modules/puppeteer/lib/cjs/puppeteer/common/DOMWorld.js:79:24)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at emitUnhandledRejectionWarning (internal/process/promises.js:168:15)
    at processPromiseRejections (internal/process/promises.js:247:11)
    at processTicksAndRejections (internal/process/task_queues.js:94:32)
(node:17868) Error: Protocol error (Runtime.callFunctionOn): Execution context was destroyed.
    at /test_save_page/node_modules/puppeteer/lib/cjs/puppeteer/common/Connection.js:208:63
    at new Promise (<anonymous>)
    at CDPSession.send (/test_save_page/node_modules/puppeteer/lib/cjs/puppeteer/common/Connection.js:207:16)
    at ExecutionContext._evaluateInternal (/test_save_page/node_modules/puppeteer/lib/cjs/puppeteer/common/ExecutionContext.js:200:50)
    at ExecutionContext.evaluate (/test_save_page/node_modules/puppeteer/lib/cjs/puppeteer/common/ExecutionContext.js:106:27)
    at DOMWorld.evaluate (/test_save_page/node_modules/puppeteer/lib/cjs/puppeteer/common/DOMWorld.js:79:24)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
(node:17868) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
    at emitDeprecationWarning (internal/process/promises.js:180:11)
    at processPromiseRejections (internal/process/promises.js:249:13)
    at processTicksAndRejections (internal/process/task_queues.js:94:32)

I'm new to the JS ecosystem, so not sure how to debug further.

kristallizer avatar Oct 19 '20 00:10 kristallizer