CSS/img pulled from browser cache won't be reported in matrix

[gorhill]

That's because there is no onBeforeRequest() triggered for these. That sucks.

Potential fix: just like for scripts/plugins, inspect the DOM and collect occurrences from there as well.

[gorhill]

I started a fix for this, and now one of the side effect of the solution is that requests might be reported twice in the log, because one request might have been recorded through onBeforeRequest() and another through DOM analysis.

Need to find a way to mitigate this.

[gorhill]

I should give a try to webRequest.onResponseStarted or webRequest.onCompleted and check the details.fromCache flag, looks promising.

[gorhill]

By the way, a good way to verify the bug is:

• Go to http://arstechnica.com/
• Open the Statistics page and narrow the log to http://arstechnica.com/
• Put the text cursor in the address bar of http://arstechnica.com/ and hit Enter
  • Do not force refresh, as this soft clear the cache I believe
• The Statistics page should show that very few items were logged following the main_frame event.
  • Since currently items are logged only for when items are not fetched from the cache.

To confirm any fix works, check that all items on the page are logged, not just those items outside the cache.

[gorhill]

Alright, I am losing track of, and forgetting earlier findings and now I am back to square one.

webRequest.onResponseStarted/details.fromCache is no good, it works for objects pulled from the on-disk cache, not the in-memory cache. This bug here causes the following scenario to happen, which I should have documented first time I entered this bug:

• Go to http://arstechnica.com/
• Drill-in to one article on http://arstechnica.com/ (any will do)
• Wait for HTTPSB's "garbage collector" to kick in -- around 10 minutes (all recorded requests for http://arstechnica.com/ are flushed since the page hasn't been used for a while)
• Go back to http://arstechnica.com/ using the back button of the browser
• Matrix for http://arstechnica.com/ is now almost all empty, no image and css reported
  • This is because most of the data is pulled from the in-memory cache.

This is the bug. Still no solution.

[my-password-is-password]

What about using DOM Mutation Observers?

Found this https://code.google.com/p/mutation-summary/