Workflow Parameters

[sphadley]

How do you pass workflow parameters using create_namespaced_workflow. I don't see anything about in the documentation, and I don't see anything specifically handling in the kwargs parsing. Am I missing something? Is it supported?

[abaiju15]

@sphadley I'm actually wondering the same thing with the args. Were you at least able to run the example locally? When I try to submit a workflow, the workflow does get created but I'm also getting this:

Traceback (most recent call last):
  File "test.py", line 67, in <module>
    workflow = v1alpha1.create_namespaced_workflow(namespace, manifest)
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api/v1alpha1_api.py", line 311, in create_namespaced_workflow
    return self.create_namespaced_workflow_with_http_info(namespace, body, **kwargs)  # noqa: E501
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api/v1alpha1_api.py", line 408, in create_namespaced_workflow_with_http_info
    collection_formats=collection_formats)
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api_client.py", line 353, in call_api
    _preload_content, _request_timeout, _host)
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api_client.py", line 192, in __call_api
    return_data = self.deserialize(response_data, response_type)
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api_client.py", line 264, in deserialize
    return self.__deserialize(data, response_type)
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api_client.py", line 303, in __deserialize
    return self.__deserialize_model(data, klass)
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api_client.py", line 639, in __deserialize_model
    kwargs[attr] = self.__deserialize(value, attr_type)
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api_client.py", line 303, in __deserialize
    return self.__deserialize_model(data, klass)
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api_client.py", line 639, in __deserialize_model
    kwargs[attr] = self.__deserialize(value, attr_type)
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api_client.py", line 281, in __deserialize
    for sub_data in data]
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api_client.py", line 281, in <listcomp>
    for sub_data in data]
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api_client.py", line 303, in __deserialize
    return self.__deserialize_model(data, klass)
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api_client.py", line 639, in __deserialize_model
    kwargs[attr] = self.__deserialize(value, attr_type)
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api_client.py", line 303, in __deserialize
    return self.__deserialize_model(data, klass)
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api_client.py", line 641, in __deserialize_model
    instance = klass(**kwargs)
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/kubernetes/client/models/v1_container.py", line 123, in __init__
    self.name = name
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/kubernetes/client/models/v1_container.py", line 350, in name
    raise ValueError("Invalid value for `name`, must not be `None`")  # noqa: E501
ValueError: Invalid value for `name`, must not be `None`

Did you happen to run into this?

[binarycrayon]

@abaiju15 Long story short, the repo hasn't been updated for a while and I just completed an update in branch 2.10.2 https://github.com/argoproj-labs/argo-client-python/tree/argo/v2.10.2

I have not had a chance to release it to pypi yet, if you don't want to wait, you can try it by:

pip install -e git+https://github.com/argoproj-labs/argo-client-python.git@argo/v2.10.2#egg=argo-workflows

The new release has multiple integration tests to submit most of the example yaml via the sdk so you shouldn't run into above ValueError.

Please make sure you are running argo 2.10.x

[binarycrayon]

@sphadley would you mind be more specific about your use case? Are you talking about overriding a parameter upon submitting a workflow? How is your workflow constructed?

[abaiju15]

Thanks @binarycrayon - that worked for the example! In terms of my use case of being able to submit parameters with the workflow, I'm trying to basically add an arbitrary number of input parameters to be used within the workflow. Let's say I have a template with these arguments:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: service-
spec:
  entrypoint: workflow-start
  arguments:
    parameters:
    - name: tag
      value: ""
    - name: registry
      value: ""

and I want to be able to add in more parameters at runtime, so it would look something like this:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: service-
spec:
  entrypoint: workflow-start
  arguments:
    parameters:
    - name: tag
      value: ""
    - name: registry
      value: ""
    - name: newParam1
      value: "newValue1"
    - name: newParam2
      value: "newValue2"

How would this be possible without having to modify the workflow.yaml file ahead of time?

[sphadley]

@binarycrayon, @abaiju15 's use case matches mine almost exactly.

Thanks, Steven

[samuelamar15]

Hey @abaiju15!

I am getting the same issue as you except the workflow doesn't even get created. @binarycrayon i couldn't find the branch you suggested so i tried both the argo/v2.10.0 and latest release-3.5 with argo version 2.10.2 but i am still getting the following trace:

Traceback (most recent call last):
  File "C:/Users/Samuel/AppData/Roaming/JetBrains/PyCharmCE2020.2/scratches/scratch.py", line 35, in <module>
    res = thread.get()
  File "C:\Users\Samuel\AppData\Local\Programs\Python\Python37\Lib\multiprocessing\pool.py", line 683, in get
    raise self._value
  File "C:\Users\Samuel\AppData\Local\Programs\Python\Python37\Lib\multiprocessing\pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "c:\users\samuel\desktop\src\argo-workflows\argo\workflows\client\api_client.py", line 208, in __call_api
    return_data = self.deserialize(response_data, response_type)
  File "c:\users\samuel\desktop\src\argo-workflows\argo\workflows\client\api_client.py", line 280, in deserialize
    return self.__deserialize(data, response_type)
  File "c:\users\samuel\desktop\src\argo-workflows\argo\workflows\client\api_client.py", line 319, in __deserialize
    return self.__deserialize_model(data, klass)
  File "c:\users\samuel\desktop\src\argo-workflows\argo\workflows\client\api_client.py", line 660, in __deserialize_model
    instance = klass(**kwargs)
  File "c:\users\samuel\desktop\src\argo-workflows\argo\workflows\client\models\v1alpha1_workflow.py", line 68, in __init__
    self.metadata = metadata
  File "c:\users\samuel\desktop\src\argo-workflows\argo\workflows\client\models\v1alpha1_workflow.py", line 138, in metadata
    raise ValueError("Invalid value for `metadata`, must not be `None`")  # noqa: E501
ValueError: Invalid value for `metadata`, must not be `None`

Process finished with exit code 1

What exactly is the versions of argo, kubenetes and argo-python-client i shoud use in order to use the api with this python client?

I also noticed argo-server is using v1 api but this client uses v1alpha1, maybe this is the problem?

Thanks!

[binarycrayon]

@samuelamar15 @abaiju15 @sphadley Could you try argo-workflows==4.0.1 release? Thanks!

[samuelamar15]

Unfortunatly I still cannot use it because the 4.0.1 release is not compatible with the 0.3.0 of argo-dsl.

[binarycrayon]

Yes this is in the works, I'll release new argo-dsl today or tomorrow

On Sat, Dec 5, 2020 at 8:43 AM Samuel Amar [email protected] wrote:

Unfortunatly I still cannot use it because the 4.0.1 release is not compatible with the 0.3.0 of argo-dsl.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/argoproj-labs/argo-client-python/issues/11#issuecomment-739319128, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAACPY4BT6BVAIOSWDA7B7TSTJPLJANCNFSM4RN24KSQ .

[binarycrayon]

@samuelamar15 I just released argo-python-dsl 0.4.0 -- also included an example to submit with dsl https://github.com/argoproj-labs/argo-python-dsl#submitting-with-dsl

Would love to know if this works for you

[samuelamar15]

@binarycrayon seems like all is working well now 👍