Update list-of-update-sites page

[ctrueden]

The list-of-update-sites repository is still styled after the old MediaWiki.

We need to integrate this better with the new wiki.

• At minimum: that means updating the style to match the new site.
• But maybe better would be to migrate the sites.yml to this repository instead, and integrate it into the GitHub Actions. Then we wouldn't need to copy-paste the page style from _includes/page to the list-of-update-sites repository, but rather would benefit from it being more DRY here.

Possibly, the two above points should be two issues... for now, this issue exists as a reminder to evaluate how we want to proceed.

[remiberthoz]

Hi,

I'd be happy to tackle this one. I would go directly for the second bullet point suggestion. I would like to summarize the current setup to be sure I understand before working on it:

• The list of update sites is available for humans:

  • served at https://imagej.github.io/list-of-update-sites/
  • and assembled by the Jekyll build process
  • https://imagej.net/list-of-update-sites/ is simply a redirection to the static GitHub pages host

• The list of update sites is made available to the ImageJ updater:

  • The updater fetches the list of update sites when it starts (to add new entries if applicable) by downloading the wiki page.
    • A initial call to initializeAndAddSites(...) by DefaultUpdateService.java#L145, eventually calls getAvailableSites(...) which calls downloadWikiPage(...).
    • A MediaWikiClient is built and its method getPageSource(...) is called with title argument set to "List of update sites". This will generate a HTTP GET request to the URL: https://imagej.net/api.php?action=query&format=xml&titles=List of update sites&export=true&exportnowrap=true from the request(...) method of the MediaWikiClient.
    • Somehow, the imagej.net server replies to the request. I cannot get a response on my browser, which blocks me to investigate further... In fact I doubt that the ../api.php?... request is correct since, if I understand right, the ImageJ wiki is now a static webpage hosted on imagej.github.io: the PHP API cannot work there.
  • The wiki page is then parsed by parseWikiPage(...) to extract URLs from the table.
    • The parser seems to handle a mix of HTML and Markdown.
    • The parser returns a Map<String, UpdateSite> which maps URLs to rich objects containing update site info.
  • From here, the updater has all the information it needs to proceed.

I obviously miss a piece of information regarding the PHP request. I'll investigate more. If anyone has an idea of how this works, help is welcome :smile:!

---

Once clarified, I think it would be good to move the sites.yml file on this repo to be DRY, as you suggested. In a first round of changes, we could simply adjust the GitHub Action workflow to prepare a wiki page from the sites.yml with no modifications to the style (to not break the updater). Then we could serve the sites.yml in raw format on a static page. Then we could upgrade the updater to parse YAML instead of parsing the Wiki. This setup allows for changes to the Wiki style, with no changes to the updater parser.

A key element which I have not figured out is how to keep things compatible with the current updater, for IJ installations which will not upgrade. Changing the Wiki style seems like a breaking-change, unless we host two pages (one with the current style for the updater, one with the new style for humans).

[remiberthoz]

Ah I understood part of it. The URL was almost correct but spaces in the title "List of update sites" have to be replaces with + not %20: https://imagej.net/api.php?action=query&format=xml&titles=List+of+update+sites&export=true&exportnowrap=true

The table is a MediaWiki format, embedded in a XML document. This is indeed what is parsed, and what I mistook for a mix of HTML and Markdown. It makes much more sense now.