Harvest metakg from the supported APIs and provide an endpoint to query aggregated metakg

[newgene]

Translator APIs in SmartAPI registry provide meta-knowledgegraph (or metakg) information. These APIs can be TRAPI APIs or those APIs annotated by x-bte extensions. We can regularly harvest metakg info from these APIs and index it in Elasticsearch, and then provide a searchable endpoint to query metakg info.

The existing /api/metakg endpoint is a proxy to http://api.bte.ncats.io/metakg, which holds all metakg in memory. As metakg grows bigger, the current implementation won't scale up, and its search capacity is limited too.

We plan to move this metakg functionability from BTE to SmartAPI, and provide a native metakg API endpoint.

[newgene]

This is related to this BTE issue: https://github.com/biothings/BioThings_Explorer_TRAPI/issues/537

Once this feature is implemented in SmartAPI, we will evaluate this BTE issue to bring the same feature into BTE (e.g. by calling SmartAPI's native /api/metakg endpoint)

[newgene]

metakg harvesting can be adopted from:

https://github.com/newgene/bte-py/tree/main/biothings_explorer/smartapi_kg

but should be much simplified for the parts needed here.

[NikkiBytes]

Complete, /metakg endpoint is functional: i.e. https://smart-api.info/api/metakg/?q=38e9e5169a72aee3659c9ddba956790d&bte=1.

If additional issues arise, a new issue can be opened.