Export values in py tasks with sub param

[jaymed]

I'm trying to call a python task with a parameter that contain a list of dictionary values by using _export variables. The variables used to define the values in the list of dictionaries are not being interpolated.

Please consider the following example. In this case I would like arg2 to be an array of dictionary values. In this case the arg1 param evaluates correctly but the value of subarg2 contains the literal definition of relevant_date instead of the interpolated value.

timezone: UTC
_export:
  relevant_date: ${moment(session_time).utc().format('YYYYMMDD')}

+setup:
  py>: tasks.MyWorkflow.step1
  arg1: ${relevant_date}
  arg2:
    subarg2: ${relevant_date}

The output looks like this:

20190207
{u'subarg2': u"${moment(session_time).utc().format('YYYYMMDD')}"}

The sample python task looks like this:

import digdag

class MyWorkflow(object):
    def step1(self, arg1, arg2):
        print (arg1)
        print (arg2)

Please help me understand how I can properly interpolate the value of subarg2.

[hiroyuki-sato]

Hello, @jaymed

You can't use ${} syntax in _export part in the current implementation. I'll add more detail later.

[chezou]

seems like same issue as https://github.com/treasure-data/digdag/issues/678 or https://github.com/treasure-data/digdag/issues/862 ?

[jaymed]

seems like same issue as #678 or #862 ?

Thanks, @chezou. It's exactly like #678. I'll have to come up with a different way to pass nested parameters to my python function.

I'm also finding that secrets can't be evaluated as py parameters, even if they aren't nested.

+setup:
  py>: tasks.MyWorkflow.step1
  arg1: ${secret:password}

Without the ability to pass in dates from _export data or secrets to py operators, I'm really limited in what I can do with digdag.