specref icon indicating copy to clipboard operation
specref copied to clipboard
verified publisher icon tobie

Retiring tr.rdf

Open xfq opened this issue 5 years ago • 7 comments

Background: https://lists.w3.org/Archives/Public/spec-prod/2020AprJun/0006.html

In the past, the only way to get data related to the specifications was tr.rdf. A few years ago, W3C released the W3C API to provide more complete data. Since W3C is now focusing the effort to improve the API, it would be great if w3c.js can be updated to use the W3C API.

A valid W3C API key[1] is required to query the W3C API, so we need a way to configure the API key somewhere in Specref (for example, it can be an environment variable queried by update-all).

PS: The W3C API is currently a plain REST/JSON API, but in the future it might be migrated to API Platform.

[1] That is, in order to use the API, you need to create a W3C account and a W3C API key.

xfq avatar Jun 16 '20 04:06 xfq

Thanks. What's your timeline for this and would you have resources to implement those changes in Specref?

tobie avatar Jun 16 '20 21:06 tobie

I looked at this a bit, and it's a significant amount of work as W3C's API doesn't appear to be designed for this use case. I don't have the resources to get this work done. So this won't happen unless someone steps up to do the work or is ready to pay for the implementation.

If tr.rdf is retired, Specref will just no longer be automatically updated for W3C specs. Other organizations (such as IETF or WHATWG) will continue to be automatically updated. So will WICG specs.

tobie avatar Dec 19 '20 11:12 tobie

@xfq In that case, either W3C should not retire tr.rdf until find the time/money to do the work to integrate the new API (which possibly includes improving the new API, given that @tobie says it's a poor fit for the use case).

Specref no longer being updated automatically just doesn't seem like an acceptable outcome.

frivoal avatar Dec 21 '20 00:12 frivoal

@tobie @frivoal we'll see if we can have someone update specref to switch to the W3C API instead of tr.rdf. Rest assure we won't retire tr.rdf without that switch. If you know other tools relying on tr.rdf, please let me know.

deniak avatar Dec 23 '20 06:12 deniak

Things to think about:

  • Updates are hourly, we risk going above the threshold allowed by the API.
  • We need to go through all specs every hour to see if they have been updated or if there are new ones.
  • This means n/20 calls to the API listing specs and then n calls to the API for specs, and again n calls to the API for different versions.
  • We also need to check aliases, and possibly obsoleted specs.
  • We need a migration story from the previous solution that preserves the existing specs and versions (possibly with a script that helps us do that).
  • There might be specific issues with referencing working groups name, I haven’t looked into that yet.

tobie avatar Dec 23 '20 10:12 tobie

For the record https://www.w3.org/2002/01/tr-automation/ has a number of tools working from tr.rdf input, so we should probably keep it, at least until we have a proper replacement for each of those.

caribouW3 avatar Feb 19 '21 12:02 caribouW3