RFE: Allow use of wildcard or regex in task names (dynamic task names)

[ssbarnea]

In order enable taskfile to a viable replacement for some more specialized test runners, such tox or nox, we need to allow it to accept task names that are using a pattern (regex or glob), so users would not have to copy/paste task definitions when creating test-matrixes.

A very easy to explain example is that tox accepts tasknames like py, py36, py311 without forcing user to explicitly define each value. The idea is that the variadic part of the name ends-up as being processed as a variable.

In fact its support for this goes even deeper allowing you to use multiple dimensions and have tasks like: py36-func, py36-unit`, all without having to define an endless list of combinations.

While supporting multiple dimensions might be seen maybe as too much for taskfile, maybe we should at least allow user to define a taskname like foo* and ensure that the effective taskname is available inside a variable. That should allow someone to process the name and infer the matrix variables to use for it, or give an error message if the arguments are not valid.

While the glob version (py*) would be easier to implement, the regex would be more flexible as it would allow to validate the task name and even document allowed inputs, like py(\d+)\-((unit|func)). Using capture groups we can also extract the dimensions, if using named capturing groups we could even determine in which variables we could save them.

[theunrepentantgeek]

Using regular expressions would open up a whole raft of complex edge cases, purely because regular expressions are so powerful. I suspect there'd also be some funky encoding issues (esp around " and ') embedding them into YAML files as well.

Allowing a simple * wildcard would be easier to explain, while still quite powerful.

Just to toss out one possible approach ...

• Each * could be captured as a separate value, with the values accessible via a new template function (maybe glob(index))
• Enforcing values could be done by using the existing preconditions support

There would still lots of edge cases to solve, not least of which is what should happen if multiple tasks match.

E.g. what if you had

tasks:
    Start*:
        # ... elided...
    *Stop:
        # ... elided ...
    Limbo:
        deps: [StartDoorStop]

What does the task Limbo depend on?

Possible answers include:

• Limbo depends on Start* with the * resolving to DoorStop
• Limbo depends on *Stop with the * resolving to StartDoor
• Limbo depends on both Start* and *Stop, with the * resolving as above
• Limbo depends on neither, the dependency is ignored
• It's an error; nothing is run.

I think I can make a convincing argument for all five of these choices.

What about when the match is empty?

tasks:
    Disco:
        deps: [Start]

Does the task Disco depend on Start* (with an empty match for the *), or not?

Whichever answers are picked, they need to be documented - and Task needs to give good diagnostics so that people aren't left scratching their heads in disbelief.

[ssbarnea]

I am inclined to advise making "found multiple matches" as well as "no matches" a runtime error, just to avoid confusions like you described.

This could also play well with an option that would allow minor typos in task names based on their name, like guess_task_name(input), where the same rule applies return value only if one and only one is found. Obviously that this interactive auto-correct feature would need to be enabled only on interactive consoles, while still displaying a warning about the replacement being made. We never want to accept typos in code, but on console,... is quite common to get one letter wrong.

In fact * is invalid in yaml if not quoted. I am not worried about double quoting in YAML, the spec is very clear on how it works. Also I really doubt that anyone would be insane to attempt to use single or double quote on a task name, so regex quoting is a not really an issue. Also am I sure that we will never attempt to implement our own globbing or regex engine either. In fact it seems that go has built-in regex support, which makes this very safe to use.

That is why I am biased toward regex, as it does open the door for more complex patterns in the future and capture groups, something globbing will never allow. Obviously that for first implementation we should only support simple regex pattern matching (no capture groups).

[theunrepentantgeek]

I am not worried about double quoting in YAML,

Quoting in YAML is straightforward. Quoting in Regular expressions is also straightforward. Quoting a Regex when it's embedded in YAML? That's a far more complicated beast because the different quoting rules interact. Even if you get the quoting correct, what's in the YAML file wouldn't be the original regex, but something that's harder to read. I don't know about you, but I'm not in favour of anything that makes a Regex harder to read.

Also am I sure that we will never attempt to implement our own globbing or regex engine

Where did I suggest that?

Anyway, globbing is really simple to implement - pass the string to regexp.QuoteMeta(), then do a string replace replacing \* with (.*) and you're ready to go.

[Zebradil]

I really miss this feature of Makefiles:

update-foo::
update-bar::
update-baz::
update-%::
	$(MAKE) -C pkg/$* update

[liblaf]

will this feature be added to the roadmap in the future? it's really awesome!

[hans-d]

Would be great to have, as run: once would be respected as well. Currently using the following code to do this, but lacking the run: once

tasks:
  run_dynamic:
    internal: true
    requires:
      vars: [ TASK_SUFFIX ]
    vars:
      TASKS:
        sh: |
          task --list-all -j \
          | jq '.tasks[].name | select (endswith(":{{.TASK_SUFFIX}}"))' -r
    cmds:
      - for: { var: TASKS }
        # hack, as using the `task: :{{.ITEM}} is not working
        cmd: task {{.ITEM}}