feat: add FUSE support
There are instances where using a FUSE is useful/essential for the completion of a TES task. For example, for tasks that use crypt4gh-encrypted files, using crypt4ghfs would allow the files to remain encrypted at rest and only be decrypted during read operations, greatly enhancing security.
Currently, using a FUSE in Docker requires the use of --cap-add SYS_ADMIN or --privileged flags, per this discussion. As such, I was able to use cryptgh4fs in a standalone docker container, but not in a TES implementation.
Ideally, a user would specify an image with a FUSE and specify a mount point to mount the FUSE in a TES request, and file operations in all the executors would be done with the FUSE. However, allowing individual executors to run in privileged mode could also be useful.
It is an interesting idea. I will check and prepare a demo of a crypt4gh FUSE TESK server.
I did some checks on my laptop and it works well, but it was a bit complicated to install. I also saw that the standalone crypt4gh command exists with decrypt and encrypt options and it was much easier to install. I wonder if another option to consider for achieving this is to use an innitContainer and an emptyDir using memory instead of storage.
We had similar issues with our storage infrastructure. To solve this we implemented KubeDAL, which is a unified storage / csi driver that enables persistent volumes to just be fuse mounts for a number of storage backends (S3, etc.). This circumvents the capability / permission problem but would need additional work for the crypt4gh use-case.
Thanks a lot for the pointer to KubeDAL (and also OpenDAL in the first place), @St4NNi - it looks fantastic!