sofifa-web-scraper icon indicating copy to clipboard operation
sofifa-web-scraper copied to clipboard

Humanoid stopped working

Open Geffikk opened this issue 1 year ago • 1 comments

the humanoid stopped working. First when you implemented this update, everything worked fine, but yesterday it stopped working due to the following error. Is this problem only on my site ?

npm run download-urls

starting to download latest player urls...

C:\Projects\sofifa-web-scraper\node_modules\request-promise-core\lib\errors.js:14
        Error.captureStackTrace(this);
              ^
RequestError: TypeError [ERR_INVALID_CHAR]: Invalid character in header content ["User-Agent"]
    at new RequestError (C:\Projects\sofifa-web-scraper\node_modules\request-promise-core\lib\errors.js:14:15)
    at plumbing.callback (C:\Projects\sofifa-web-scraper\node_modules\request-promise-core\lib\plumbing.js:87:29)
    at Request.RP$callback [as _callback] (C:\Projects\sofifa-web-scraper\node_modules\request-promise-core\lib\plumbing.js:46:31)
    at self.callback (C:\Projects\sofifa-web-scraper\node_modules\request\request.js:185:22)
    at Request.emit (node:events:514:28)
    at Request.start (C:\Projects\sofifa-web-scraper\node_modules\request\request.js:753:10)
    at Request.end (C:\Projects\sofifa-web-scraper\node_modules\request\request.js:1505:10)
    ... 2 lines matching cause stack trace ...
    at process.processImmediate (node:internal/timers:478:21) {
  cause: TypeError [ERR_INVALID_CHAR]: Invalid character in header content ["User-Agent"]
      at ClientRequest.setHeader (node:_http_outgoing:655:3)
      at new ClientRequest (node:_http_client:286:14)
      at Object.request (node:https:378:10)
      at Request.start (C:\Projects\sofifa-web-scraper\node_modules\request\request.js:751:32)
      at Request.end (C:\Projects\sofifa-web-scraper\node_modules\request\request.js:1505:10)
      at end (C:\Projects\sofifa-web-scraper\node_modules\request\request.js:564:14)
      at Immediate.<anonymous> (C:\Projects\sofifa-web-scraper\node_modules\request\request.js:578:7)
      at process.processImmediate (node:internal/timers:478:21) {
    code: 'ERR_INVALID_CHAR'
  },
  error: TypeError [ERR_INVALID_CHAR]: Invalid character in header content ["User-Agent"]
      at ClientRequest.setHeader (node:_http_outgoing:655:3)
      at new ClientRequest (node:_http_client:286:14)
      at Object.request (node:https:378:10)
      at Request.start (C:\Projects\sofifa-web-scraper\node_modules\request\request.js:751:32)
      at Request.end (C:\Projects\sofifa-web-scraper\node_modules\request\request.js:1505:10)
      at end (C:\Projects\sofifa-web-scraper\node_modules\request\request.js:564:14)
      at Immediate.<anonymous> (C:\Projects\sofifa-web-scraper\node_modules\request\request.js:578:7)
      at process.processImmediate (node:internal/timers:478:21) {
    code: 'ERR_INVALID_CHAR'
  },
  options: {
    resolveWithFullResponse: true,
    jar: RequestJar {
      _jar: CookieJar {
        enableLooseMode: true,
        store: MemoryCookieStore { idx: {} }
      }
    },
    simple: false,
    gzip: true,
    encoding: null,
    headers: {
      Accept: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
      'Accept-Encoding': 'gzip, deflate, br',
      Connection: 'keep-alive',
      Host: 'sofifa.com',
      'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36\r'
    },
    method: 'GET',
    uri: 'https://sofifa.com/players?col=oa&sort=desc&offset=0',
    callback: [Function: RP$callback],
    transform: undefined,
    transform2xxOnly: false
  },
  response: undefined
}

Node.js v20.9.0

Process finished with exit code 1

Geffikk avatar Mar 27 '24 10:03 Geffikk

Not sure about this one, it could be humanoid issue specific to the system you're in. This has not been a problem for me, you could try using linux based vms or docker image as workaround.

prashantghimire avatar Mar 28 '24 02:03 prashantghimire