Sam Texas comments

Results 11 comments of


                                            Sam Texas

Over time, more desktops are detecting as "mobile"...

I'm having this problem as well. In fact, it seems to alternate between mobile / desktop when I shift-reload pages.

Over time, more desktops are detecting as "mobile"...

@gregmuellegger -- I have a fork on my machine where I plan to rip out the current detection method and replace it with the python port of mobiledetect (https://github.com/serbanghita/Mobile-Detect). This...

Integrate parts of Scrapy CrawlSpider Rule flexibility into DDS

Is there still interest in this feature? My team might want to develop this feature.

Bringing some civility to the conversation

How do I fix it?

Cannot get this url: https://raw.githubusercontent.com/a2u/free-proxy-list/master/free-proxy-list.txt

Just poked this Sept 2022, seems to work again, FWIW

scrapy-selenium is yielding normal scrapy.Request instead of SeleniumRequest

I plus-oned this and then solved it for myself a little later. For me this is not in the context of testing, so I have no need for contracts (at...

scrapy-selenium is yielding normal scrapy.Request instead of SeleniumRequest

Hey @educatron thanks for the question - let's not hijack the thread here. I think you should take that question directly to the scrapy community. https://scrapy.org/community/

implement a porn site filter so we don't reveal those links publicly

maybe implement this on a pre_save signal to do the lookup and flag the entry as porn

[worker] - Stop sending already-caught exceptions to sentry. Log them instead

This also affects webhooks. The webhook should send back a status and the error message to the service that called it.

force lowercase (on domain) when saving urls

this is incorrect. only the domain should be lowercase. in any case (no pun intended), the uniqueness should only be on an index based on the lowercase-ness of everthing. I...