creeper
creeper copied to clipboard
wspl
:paw_prints: Creeper - The Next Generation Crawler Framework (Go)
We probably use both HTML parser and JSON parser for crawling complex pages, I found that pattern files support HTML parser only, how could I use this framework to parse...
some site need auth login, can creeper support it ? 😈
There are some hindrance in implementing the functional part of `@next`. They have stumped me for a long time: * InitSelector's loop call * Wait until the page cycle ends...
if the website need login, this crawler does not work。 func (p *Page) Body() (string, error) { u, err := p.Url() if err != nil { return "", err }...
