Extract documentation from files

[elyisu]

Hi! Is there a way to extract model documentation (i.e., comments, field definition, etc) from the existing files? This would be very helpful to share model and table definition (as well as assertions, etc) across internal and external stakeholders.

[bigrogerio]

I think that this would be great! something similar to Sphynx

[Ekrekr]

For most cases I would propose using IAM to make the relevant parts of the project to the user!

1. Description fields https://cloud.google.com/dataform/docs/reference/dataform-core-reference#idocumentableconfig: a) Gets added to table metadata in BigQuery, for the most part.
2. Direct Table metadata in BigQuery: a) Visible in BigQuery.
3. Comments in the SQL itself. a) This would involve generating docs from SQL comments - which doesn't really seem to be a thing.

So 1) and 2) are achieved assuming that internal and external stakeholders are granted the https://cloud.google.com/bigquery/docs/access-control#bigquery.metadataViewer role, and for viewing the Dataform DAG, the https://cloud.google.com/dataform/docs/access-control#dataform.viewer role.

However, I know that you're wanting this in a standardized format in a reason 😛. Could you elaborate a bit on whether there's a standard structure that you'd expect model documentation to be exported in?