suckit
suckit copied to clipboard
Skallwar
Suck the InTernet
How may I filter for filetypes like *.png without using the regex?
I have been running `suckit https://cigeography.blogspot.com/ -i '.*\.png' -o .`: _thread '' panicked at 'Failed to parse url: http://The POLISH AIR FORCE, ARMY AVIATION and NAVAL AVIATION BRIGADE | Error:...
So my specific usecase here is attempting to mirror a site with a lot of directories of various languages, but skipping the static files at a higher level. Example Folder...
When the URL does not end with an slash, the downloaded file becomes `index_no_slash.html`. But the converted local links are wrong, because [`url.join() is called`](https://github.com/Skallwar/suckit/blob/84276b963128d417f476740ae5dcc8448fa99689/src/scraper.rs#L185), and since the URL does...
When there is a webpage served under `/folder/file1.html` as well as under `/folder`, this creates a conflict: In the first case, suckit creates a local folder, and in the second...
allow importing of cookies via --coookies to be sent along with each request
Fixes #68 ~~Still some work to do but downloaded site are noticeably prettier now~~
Usually, a website will use [`url(...)`](https://developer.mozilla.org/en-US/docs/Web/CSS/url) in the css to point out to another resource such as background image, logo, sprite, fonts, or even another css. Would love this resource...
[tl](https://github.com/y21/tl) could be a good replacement of [kuchiki](https://github.com/kuchiki-rs/kuchiki) It will be interesting to do some benchmarks to compare the two of them