matrix-org
Do we set URLs, Etags & caching headers to ensure search engines don't respider the same content?
Right now google is hammering matrix.org by indexing view.matrix.org. Do we ensure that the historical pages have nice unique URLs, Etags and have a suitably massive caching header to ensure that Google doesn't go and try to reindex the whole thing again a few days later?
ooi have we also considered having matrix-static blat historical stuff out to disk as, well, static HTML, rather than regenerating it each and every time?
The issue is that each event is it's own anchor, so we'd get a html bundle for each and every event in every room
i'm not sure I follow. i'm suggesting that the URL for "page 250 of matrix.org" remains the same (without any nasty cache-busting question marks etc), and we could even pregen the html of that page in future.
What would page 250 be though, one new message comes in and the whole page gets shifted by 1 event