feat: enable root remote taskfiles

[pd93]

This PR enables the ability to pass a remote Taskfile URI to the --taskfile flag. This basically means that your root Taskfile will be a remote one.

When specifying the --taskfile flag, Task usually assumes that the execution directory is the directory that the root Taskfile resides in. Since this isn't possible with a remote Taskfile, the execution directory will default to the current working directory instead when the Task command is executed.

A couple of notes:

1. If your remote Taskfile includes another file, this is currently treated as a local file instead of a remote one, so likely will not work. Fixing this is on the TODO list.
2. You cannot currently use both the --dir and --entrypoint flags, so there is no way to override the directory that is used when calling a remote Taskfile. We might want to consider enabling this later. For now, you would need to cd to the correct directory before executing the command.

[wburningham]

Hi @pd93, I would love to use this feature with the other changes that are in v3.30.1. Could you merge this or rebase off of v3.30.1?

[wburningham]

@pd93 just following up again on this PR. Is there feedback (workflow usage details) or coding support I could provide to help get this work across the line?

[pd93]

This PR has taken a long time to complete 😆 I did not initially realise the complexity of the changes that would be required to complete this work. Over the last couple of months or so I have spent many hours reworking this from the ground up. The functionality described in the PR description required several major changes to some core code and I wanted to make sure I'm not breaking anything for existing users.

I've spent the last couple of days testing. I suspect there will still be some issues, but the main changes are now complete and ready to be reviewed 🚀

@vmaerten @wburningham Since you've both been trying out remote Taskfiles recently, would you mind kicking the tyres on this for me? I'm pretty sure it's ready to go now. This should resolve both remote root Taskfiles and includes in remote Taskfiles.

[pd93]

@brianbraunstein Since this PR also implements the ability to use both --taskfile and --dir flags as previously requested by you in https://github.com/go-task/task/issues/655#issuecomment-1910545019, it would be great if you are also able to give this a go with regard to the new stdin reader.

[vmaerten]

@pd93 For the includes in remote Taskfiles, I've tested it with two differents setup and it works as expected Thanks a lot :fire:

[wburningham]

I got around to testing this. I ran

task --dir . -t https://gist.githubusercontent.com/wburningham/86586099785bcfd4c0289dba09ae2829/raw/b3fa6add37c947b77c435cb07d7257f8f64c6628/taskfile.yaml```

Where the remote taskfile looks like this:

version: 3

tasks: default: cmds: - cmd: echo "CLI_ARGS = '{{.CLI_ARGS}}'" silent: true - cmd: echo "CLI_FORCE = '{{.CLI_FORCE}}'" silent: true - cmd: echo "TASK = '{{.TASK}}'" silent: true - cmd: echo "ROOT_TASKFILE = '{{.ROOT_TASKFILE}}'" silent: true - cmd: echo "ROOT_DIR = '{{.ROOT_DIR}}'" silent: true - cmd: echo "TASKFILE = '{{.TASKFILE}}'" silent: true - cmd: echo "TASKFILE_DIR = '{{.TASKFILE_DIR}}'" silent: true - cmd: echo "USER_WORKING_DIR = '{{.USER_WORKING_DIR}}'" silent: true - cmd: echo "TASK_VERSION = '{{.TASK_VERSION}}'" silent: true

Using the command above, should `TASKFILE_DIR` be the value from `--dir` flag?

[pd93]

Hey @wburningham. Special variables for remote Taskfiles are something I haven't worked on yet. It seems they're pretty broken right now. I will add this to the todo list in #1317 and take a look at these in a future PR.

As long as the ability to access root files remotely and to include other remote files in a remote file is working correctly then I'm happy to merge this as-is.

Using the command above, should TASKFILE_DIR be the value from --dir flag?

To answer your question directly. TASKFILE_DIR is supposed to contain the path to the directory containing the Taskfile that holds the current task. Since a remote file is, by definition, not on the same filesystem, I'd expect it to contain a URL to the remote directory containing the file.

[wburningham]

I think I found a bug related to the special variables. I can see TASKFILE_DIR being set to the remote taskfile when the taskfile is specified with the --taskfile flag. If the taskfile is references with an includes directive in a local taskfile then TASKFILE_DIR should be the local taskfile dir (I think, or at least that is how it works today).

Steps to reproduce:

❯ cat taskfile.yaml 
version: 3

includes:
  remote: https://gist.githubusercontent.com/wburningham/86586099785bcfd4c0289dba09ae2829/raw/b3fa6add37c947b77c435cb07d7257f8f64c6628/taskfile.yaml

tasks:
  default:
    cmds: 
      - task: remote:default

❯ go install github.com/go-task/task/v3/cmd/[email protected]

❯ task --yes
CLI_ARGS = ''
CLI_FORCE = 'false'
TASK = 'remote:default'
ROOT_TASKFILE = '/private/tmp/test/Taskfile.yaml'
ROOT_DIR = '/private/tmp/test'
TASKFILE = ''
TASKFILE_DIR = '/private/tmp/test'
USER_WORKING_DIR = '/private/tmp/test'
task: [remote:default] echo "TASK_VERSION = 'v3.34.1 (h1:yAAxUM54zoaHv+OtDnGgkWSVeiRuaOCn1lPUXPQQA0o=)'"
TASK_VERSION = 'v3.34.1 (h1:yAAxUM54zoaHv+OtDnGgkWSVeiRuaOCn1lPUXPQQA0o=)'

❯ go install github.com/go-task/task/v3/cmd/task@46470038b134034edf37c0926072fc094822384d

❯ task --yes
CLI_ARGS = ''
CLI_FORCE = 'false'
TASK = 'remote:default'
ROOT_TASKFILE = '/private/tmp/test'
ROOT_DIR = '/private/tmp/test'
TASKFILE = 'https://gist.githubusercontent.com/wburningham/86586099785bcfd4c0289dba09ae2829/raw/b3fa6add37c947b77c435cb07d7257f8f64c6628/taskfile.yaml'
TASKFILE_DIR = 'https:/gist.githubusercontent.com/wburningham/86586099785bcfd4c0289dba09ae2829/raw/b3fa6add37c947b77c435cb07d7257f8f64c6628'
USER_WORKING_DIR = '/private/tmp/test'
task: [remote:default] echo "TASK_VERSION = 'v3.35.2-0.20240310214338-46470038b134 (h1:tHuecP/JTRy2ehr+s0Wo+GilpuUqtFLEHb79pCDVias=)'"
TASK_VERSION = 'v3.35.2-0.20240310214338-46470038b134 (h1:tHuecP/JTRy2ehr+s0Wo+GilpuUqtFLEHb79pCDVias=)'

Notice how TASKFILE and TASKFILE_DIR change between the two versions.