feat: Add spider RequestDeduplicationMiddleware

[jhg]

I created it to avoid error about memory limit when it needs to crawl websites with many links. The downloader middleware late much to drop request, but this spider middleware drop it earlier then, I guess, it uses less memory. I would like to share it, maybe it can be useful for someone else.

[jhg]

I see now the mistake writing the branch name, sorry.