flytestdlib icon indicating copy to clipboard operation
flytestdlib copied to clipboard

Extract CatalogClient from flytepropeller to flytestdlib

Open MorpheusXAUT opened this issue 3 years ago • 1 comments

TL;DR

The CatalogClient implementation has been extracted from flytepropeller's source code and moved to flytestdlib as flyteadmin also needs access to the functionality.

Type

  • [ ] Bug Fix
  • [X] Feature
  • [ ] Plugin

Are all requirements met?

  • [X] Code completed
  • [X] Smoke tested
  • [X] Unit tests added
  • [X] Code documentation added
  • [ ] Any pending items have an associated Issue

Complete description

The whole latest implementation of the catalog package was moved from flytepropeller to here since flyteadmin will need access to a client interacting with datacatalog for evicting cached executions. Additionally, the new functionality for acquiring/releasing reservations via artifact tag as well as the deletion of artifacts was implemented for the datacatalog client.

The PR was created as a draft until the flyteidl and flyteplugins PRs have been merged and respective versions have been published.

Tracking Issue

https://github.com/flyteorg/flyte/issues/2867

Follow-up issue

NA

MorpheusXAUT avatar Dec 15 '22 17:12 MorpheusXAUT

Codecov Report

Merging #148 (d6ec6db) into master (80d1624) will decrease coverage by 0.49%. The diff coverage is 64.50%.

@@            Coverage Diff             @@
##           master     #148      +/-   ##
==========================================
- Coverage   67.94%   67.45%   -0.50%     
==========================================
  Files          69       75       +6     
  Lines        4077     4756     +679     
==========================================
+ Hits         2770     3208     +438     
- Misses       1148     1348     +200     
- Partials      159      200      +41     
Flag Coverage Δ
unittests 67.45% <64.50%> (-0.50%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
catalog/client.go 0.00% <0.00%> (ø)
catalog/config.go 0.00% <0.00%> (ø)
catalog/noop_catalog.go 0.00% <0.00%> (ø)
catalog/config_flags.go 30.00% <30.00%> (ø)
catalog/datacatalog/datacatalog.go 63.35% <63.35%> (ø)
catalog/datacatalog/transformer.go 80.71% <80.71%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

codecov[bot] avatar Dec 15 '22 17:12 codecov[bot]