dragospopa420
dragospopa420
Solved this issue: Add on your class the following : ` headers = { 'User-Agent': 'some real user agent', 'Accept': '*/*', 'Accept-Encoding': 'gzip,deflate,br', 'Connection': 'keep-alive' }` And also on your...
`{ 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'Accept-Language': 'en', }` Those are the default ones...they have a million reasons to go wrong. I was thinking that a more general approach would fit better.
That BrightDataProxyMiddleware only gets activated if I specify in custom settings `BRIGHTDATA_ENABLED: True`, otherwise it doesn't do anything. Also just to be sure I changed the settings to those below...
I'm using csv-writer to export my data in csv. I'm also interested in parquet, tried all the possible parquet writers from NPM... for some reason the output is waaay bigger...
Thanks @ivanvs . Thanks @B4nan for transferring the issue Thanks @mtrunkat I've also had some time to test this image and seems to perform well. Haven't found anything wrong with...
> I was asking @vladfrangu to take a closer look last week. IIRC the reason why we use ubuntu was supporting chromium, rest of the browsers should be fine with...