Deleted user
Deleted user
直接去改ss-libev那个代码难度很大吗???自己实现一个加解密是不是就可以了?
Running just `$header=get_headers($url);` should be way more efficient.
Then you will have to use CURL, which should be way less efficient that the PHP function. This might be something you want to test.
Are you sure? https://itstillworks.com/getheaders-vs-curl-speed-php-12245951.html
Oh, OK, a misunderstanding. I though you said that get_headers($url) is using internally cURL. I think it should be way faster to use just get_headers() and run cURL only when...
Are you currently crawling CSS, JS, and all other files for the URLs? Using headers should eliminate going through unnecessary files when you know in advance the media types and...
CSS and JS should not be directly linked, but there there are often various downloads like PDFs etc. that are href-linked.
Here are the exact rules: https://www.sitemaps.org/protocol.html#escaping All the stuff have to be entity escaped and URLs URL-escaped and encoded. Rules are very clear.
If you encounter encoded and/or escaped URLs, you should decode and unescape them before adding them to crawl list. All the text should be encode when it's written to sitemap...
Licenses are important for image sitemaps as there are no other feasible methods for communicating image licenses to search engines. If they are site wide, there could be a command...