oso icon indicating copy to clipboard operation
oso copied to clipboard
verified publisher icon opensource-observer

Cleanup schemas across oss-directory to dbt models to API

Open ryscheng opened this issue 1 year ago • 5 comments

What is it?

As of this PR https://github.com/opensource-observer/oso/pull/1334, we now have a style guide. There are many instances where we violate this style guide. This is the meta-issue to just go through and clean it up

ryscheng avatar Apr 30 '24 21:04 ryscheng

artifacts/projects/collections models https://github.com/opensource-observer/oso/pull/1337

ryscheng avatar May 01 '24 01:05 ryscheng

artifacts_by_project and projects_by_collection here https://github.com/opensource-observer/oso/pull/1339

ryscheng avatar May 01 '24 01:05 ryscheng

more here https://github.com/opensource-observer/oso/pull/1340

ryscheng avatar May 01 '24 02:05 ryscheng

A summary of the changes we want to make:

  • Remove use of all positional arguments (e.g. GROUP BY 1,2,3)
  • Replace all single-letter and acryonym names with a full name
  • Move anything not labeled _v1 in marts back to an intermediate model
  • Make project/collection/artifact/user naming consistent. We decided to go with _id, _source, _namespace, _name.
  • We'll move the historical list of artifact names to a new table artifacts_history. We should probably do the same with project/collection slugs (which will be renamed to name)
  • Rename project_name to display_name
  • Currently we have a separate artifact for each artifact type, even if its the same address. We should go back to collapsing this into a single artifact. We'll have a new 1-to-many table called artifact_types to reflect the many types an artifact could have.

Anything I'm missing? @ccerv1

ryscheng avatar May 01 '24 17:05 ryscheng

Yes, I've reviewed and this covers everything. Let's do it.

ccerv1 avatar May 01 '24 20:05 ccerv1

oss-directory updated here https://github.com/opensource-observer/oss-directory/pull/276

ryscheng avatar May 02 '24 18:05 ryscheng

update directory marts to alias here https://github.com/opensource-observer/oso/pull/1352

ryscheng avatar May 02 '24 19:05 ryscheng

first major round of model renames and moves #1344

ccerv1 avatar May 02 '24 19:05 ccerv1

import oss-directory fixed here https://github.com/opensource-observer/oso/pull/1356

ryscheng avatar May 02 '24 22:05 ryscheng

renaming marts models columns here https://github.com/opensource-observer/oso/pull/1355

ryscheng avatar May 02 '24 22:05 ryscheng

publish oss-directory to npm here https://github.com/opensource-observer/oss-directory/pull/278

ryscheng avatar May 03 '24 01:05 ryscheng

Staging and mart models here https://github.com/opensource-observer/oso/pull/1359

ryscheng avatar May 03 '24 06:05 ryscheng

oss-directory should import descriptions https://github.com/opensource-observer/oso/pull/1360

ryscheng avatar May 03 '24 06:05 ryscheng

Gonna close this for now, and track the followon work here: https://github.com/opensource-observer/oso/issues/1365

ryscheng avatar May 06 '24 00:05 ryscheng