Automatically initialize Apptainer

[natefoo]

Ultimately the goal here is to support easier production deployments where Galaxy itself does not have to be installed in the shared FS. Admins can configure the tool_dependency_dir to be on the shared FS and then we can just use Apptainer + a galaxy-job-execution container for running set_meta().

I should probably add a few more tests but wanted to get this out here for comment in case people don't like the direction I've taken.

How to test the changes?

(Select all options that apply)

• [x] ~~I've included~~ I will include appropriate automated tests.
• [x] This is a refactoring of components with existing test coverage.
• [x] Instructions for manual testing are as follows:
  1. Configure some resolvers in container_resolvers where s/singularity/apptainer/
  2. Set auto_init: true
  3. Try setting prefix and/or exec to change where Apptainer is installed.

License

• [x] I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

[natefoo]

I added some unit tests, I am a little iffy on adding to the integration tests since they are a good bit slower and our tests are already slow enough. Since the Apptainer resolvers are just the Singularity resolvers in a trenchcoat we wouldn't be testing anything new. I can add a test of the auto-init functionality though if we want.

[mvdbeek]

This seems to do so much more than just initialize apptainer ? About the core apptainer install idea, this will have the same issues that conda auto init has, namely it breaks and leaves behind lock files in our production setup with multiple processes. It'd seem significantly cleaner if this was just a separate command you can launch.

[natefoo]

It'd seem significantly cleaner if this was just a separate command you can launch.

I just assumed we'd want things done similar to how everything else is done but I'm actually very happy for that to be the case. I'll scrap this and make a simpler version.

[mvdbeek]

There's a lot of cool stuff in here though, maybe you can just drop the auto_init stuff and we could merge this ?

[ahmedhamidawan]

Hey @natefoo 👋 . Any updates on this?

[natefoo]

Without auto_init I think a lot of the other changes aren't worth keeping, @mvdbeek did you have any specific thoughts on what was good? i.e. that's sort of the whole purpose of the apptainer resolver and all the refactoring that was done in support of that.