Generate plugin READMEs from their info models

[colinhicks]

Currently, changes to plugin information models need to be mirrored, manually in the respective README.md.

This proposal outlines a possible approach to:

1. Generate the relevant portion of the README directly from the info model.
2. Integrate into the existing plugin release process.

A: Create a doc-gen lein plugin to render an information-model map into markdown

• Content should correspond to the plugin's catalog and lifecycle entries, per function, including for each its summary & attributes/parameters.
• Match the structure and format of the existing info model documentation (with perhaps a few tweaks).
• Parameterize the lein plugin so output can be configured per Onyx plugin project.clj.
• Feedback needed: This could leverage some existing code in onyx.static.doc-gen. Alternatively, the plugin could avoid its own onyx core dependency, at the cost of minor duplication.

B: Alter release_plugin.sh

• Run the lein plugin and combine the output with the other readme sections (see below).
• A simple implementation, e.g. cat readme-above.md $(plugin-output) readme-below.md > README.md, would avoid need for Pandoc.

C: Configure individual Onyx plugins

• In each project.clj, configure doc-gen plugin parameters. Something like:

:onyx-doc-gen {
  :information-model onyx.kafka.information-model/model
  ;; where :code is an example formatting hint to wrap the field value in backticks
  :catalog-entry-columns [[:key "Parameter"]
                          [:type "Type" :code]
                          [:default Default :code]
                          [:doc "Description"]]}

• Create (or use existing) plugin /doc directory, adding two files:
  • readme-above.md for documentation above the info model (plugin description, installation, etc.)
  • readme-below.md for documentation below the info model (development, contributing, license, etc.)

[MichaelDrogalis]

After discussion in Slack, plan is to go forward with the suggested approach. 👍

[colinhicks]

Quick update: the plugin implementation is nearly ready for initial review. Will ping Slack to discuss logistics. Probably on Monday.

Note the approach differs from the issue description, above, so reading that likely won't help explicate the code. Docs coming.

I cooked up some templating after all, so the generated sections can flow naturally around any written context; viz. ~~input (see the ::::::: sections) and output.~~ *updated input (see the ````onyx-gen-doc` sections) and output.

[MichaelDrogalis]

This is really neat! Wonderful work. :)

[mariusz-jachimowicz-83]

Looks awesome