flytekit icon indicating copy to clipboard operation
flytekit copied to clipboard

[Core feature] Override task `secret_requests` using `with_overrides`

Open Mecoli1219 opened this issue 1 year ago • 2 comments

Tracking issue

https://github.com/flyteorg/flyte/issues/5085

Why are the changes needed?

with_overrides method should enable overriding secret_requests property.

What changes were proposed in this pull request?

Handling secret_requests inside with_overrides method, and adding the property override_security_context inside TaskNodeOverrides to send the overrides information to flyteAdmin.

How was this patch tested?

Check this PR: https://github.com/flyteorg/flyte/pull/5261

Check all the applicable boxes

  • [ ] I updated the documentation accordingly.
  • [ ] All new and existing tests passed.
  • [x] All commits are signed-off.

Related PRs

https://github.com/flyteorg/flyte/pull/5261

Docs link

Mecoli1219 avatar Apr 21 '24 07:04 Mecoli1219

Thank you for opening this pull request! 🙌

These tips will help get your PR across the finish line:

  • Most of the repos have a PR template; if not, fill it out to the best of your knowledge.
  • Sign off your commits (Reference: DCO Guide).

welcome[bot] avatar Apr 21 '24 07:04 welcome[bot]

from flytekit import task, Secret, workflow
from flytekit.testing import SecretsManager
import typing
import flytekit
import os
import typing
from collections import OrderedDict

import flytekit
import flytekit.configuration
from flytekit.configuration import Image, ImageConfig
from flytekit.core.workflow import workflow
from flytekit.tools.translator import get_serializable

serialization_settings = flytekit.configuration.SerializationSettings(
    project="proj",
    domain="dom",
    version="123",
    image_config=ImageConfig(Image(name="name", fqn="asdf/fdsa", tag="123")),
    env={},
)


@task(secret_requests=[Secret("group1", key="key1")])
def foo() -> str:
    return "hello" + flytekit.current_context().secrets.get("group1", "key1")


@workflow
def my_wf() -> typing.Tuple[str, str]:
    overrides1 = {
        "secret_requests": [Secret("group3", key="key3")],
    }
    overrides2 = {
        "secret_requests": [Secret("group2", key="key2"), Secret("group3", key="key3")],
    }
    str1 = foo().with_overrides(**overrides2)
    str2 = foo().with_overrides(**overrides2)
    str2 = foo().with_overrides(**overrides2)
    str2 = foo().with_overrides(**overrides2)
    str2 = foo().with_overrides(**overrides2)

    return str1, str2


sec = SecretsManager()
os.environ[sec.get_secrets_env_var("group1", "key1")] = "super-secret-value1"
os.environ[sec.get_secrets_env_var("group2", "key2")] = "super-secret-value2"
os.environ[sec.get_secrets_env_var("group3", "key3")] = "super-secret-value3" 


# model_wf = get_serializable(OrderedDict(), serialization_settings, my_wf)

Mecoli1219 avatar Apr 21 '24 09:04 Mecoli1219