spidergram
spidergram copied to clipboard
Shadow Dom, improved dynamic content support
The default page handler for the spider now uses getPageMarkup() in the BrowserTools utls to retrieve the markup of the current page. What it doesn't yet do is properly handle pages with shadow dom content, or handle dynamically-loaded content.
Because that function is a convenient wrapper for "whatever it takes to get the markup," we can use it to iterate without complicating the rest of the page handling.