Extract CatalogClient from flytepropeller to flytestdlib
TL;DR
The CatalogClient implementation has been extracted from flytepropeller's source code and moved to flytestdlib as flyteadmin also needs access to the functionality.
Type
- [ ] Bug Fix
- [X] Feature
- [ ] Plugin
Are all requirements met?
- [X] Code completed
- [X] Smoke tested
- [X] Unit tests added
- [X] Code documentation added
- [ ] Any pending items have an associated Issue
Complete description
The whole latest implementation of the catalog package was moved from flytepropeller to here since flyteadmin will need access to a client interacting with datacatalog for evicting cached executions.
Additionally, the new functionality for acquiring/releasing reservations via artifact tag as well as the deletion of artifacts was implemented for the datacatalog client.
The PR was created as a draft until the flyteidl and flyteplugins PRs have been merged and respective versions have been published.
Tracking Issue
https://github.com/flyteorg/flyte/issues/2867
Follow-up issue
NA
Codecov Report
Merging #148 (d6ec6db) into master (80d1624) will decrease coverage by
0.49%. The diff coverage is64.50%.
@@ Coverage Diff @@
## master #148 +/- ##
==========================================
- Coverage 67.94% 67.45% -0.50%
==========================================
Files 69 75 +6
Lines 4077 4756 +679
==========================================
+ Hits 2770 3208 +438
- Misses 1148 1348 +200
- Partials 159 200 +41
| Flag | Coverage Δ | |
|---|---|---|
| unittests | 67.45% <64.50%> (-0.50%) |
:arrow_down: |
Flags with carried forward coverage won't be shown. Click here to find out more.
| Impacted Files | Coverage Δ | |
|---|---|---|
| catalog/client.go | 0.00% <0.00%> (ø) |
|
| catalog/config.go | 0.00% <0.00%> (ø) |
|
| catalog/noop_catalog.go | 0.00% <0.00%> (ø) |
|
| catalog/config_flags.go | 30.00% <30.00%> (ø) |
|
| catalog/datacatalog/datacatalog.go | 63.35% <63.35%> (ø) |
|
| catalog/datacatalog/transformer.go | 80.71% <80.71%> (ø) |
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.