[feature request] `LD_PRELOAD`-based interface/variant for fuse-archive

[vadimkantorov]

It appears that fuse-archive is a complete FS impl reading from libarchive-supported archives.

It some cases (where one does not have FUSE kernel module available or root permissions), it is beneficial and workable to "mount" files like uncompressed ZIPs or ISOs etc via LD_PRELOAD-based stdio and file-related libc syscall interception. E.g. it could eliminated an archive extraction or allow simple mmap-based access to uncompressed archive contents (like TAR, ISO, uncompressed ZIP etc). Sure, it has its own limitations, but for many usecases it would be a breeze.

Compiling with such a shim makes a nice embedded virtual read-only FS (e.g. for building a Python distro into a single executable) in a relatively portable way (e.g. can include Emscripten). Something like this is implemented in cosmo libc (in a tightly coupled way) under the name of zipos. But libarchive-based mini virtual FS is a useful primer/scaffolding in general (and especially when cosmo libc is not being used).

[fdegros]

I'm not sure what exactly this feature request is about.

If the feature you have in mind is implemented, what does it change concretely for a fuse-archive user? What would a user be able to do that they cannot currently do? What would it look like? How would they use it?

[vadimkantorov]

I think, this would expand the fuse-archive's usecases, including listing, working with archives and supporting random-access seek without modifying existing user's scripts/code - also for the cases where FUSE kernel module is not available or the user does not have root permissions for mounting.

Of course, you are correct that this idea would change nothing for existing fuse-archive users.

[fdegros]

Sorry, but I still don't understand. Could you please explain what you exactly mean by:

working with archives and supporting random-access seek without modifying existing user's scripts/code"

What does not currently work with fuse-archive? What are the steps to reproduce the issue? What script or code did you have to modify? What was the expected result? What was the observable result?

... for the cases where FUSE kernel module is not available or the user does not have root permissions for mounting

The fuse-archive program implements a filesystem in userspace (aka FUSE). As such, it relies on the FUSE support in the Linux kernel. But it does not require the user calling the fuse-archive program to be the root user. An "ordinary" user can use fuse-archive to mount an archive as a read-only filesystem.

$ mkdir mnt

$ fuse-archive MyArchive.zip mnt

$ findmnt mnt | cat
TARGET                                       SOURCE       FSTYPE            OPTIONS
/usr/local/google/home/fdegros/Downloads/mnt fuse-archive fuse.fuse-archive ro,nosuid,nodev,relatime,user_id=270632,group_id=89939

$ umount mnt

[vadimkantorov]

Sorry, but I still don't understand. Could you please explain what you exactly mean by:

fuse-archive provides a proper FUSE interface (requiring FUSE kernel module being loaded) for working with archives as if they were regular directories. an alternative avenue is using LD_PRELOAD-mechanism for intercepting file-related functions to forward the calls to a custom FS. LD_PRELOAD mechanism of course has its downsides, the upside (when LD_PRELOAD is applicable at all) being that it does not need FUSE kernel module.

I just found a repo which implements an adapter which lets one use LD_PRELOAD for FUSE-based file systems without FUSE kernel module: https://github.com/sholtrop/ldpfuse

An "ordinary" user can use fuse-archive to mount an archive as a read-only filesystem.

Thanks you for clarification!

[fdegros]

I just found a repo which implements an adapter which lets one use LD_PRELOAD for FUSE-based file systems without FUSE kernel module: https://github.com/sholtrop/ldpfuse

Thanks. I see. That's good to know.

[ymartin59]

Here is a project that may fulfill transparent access to archives in user land (without FUSE): AVFS at https://avf.sourceforge.net/ and https://sourceforge.net/projects/avf/ using file access implementations from Midnight Commander.

[vadimkantorov]

Also, for some reason AVF has removed LD_PRELOAD support: https://sourceforge.net/p/avf/git/ci/8989ae447eaea0a700ae25e72898fddfec6e156a/

It's a known problem that very complete LD_PRELOAD-based impls are quite brittle (but impl of basic methods sufficient for simple data processing is quite simple and compact), but I wonder what precisely pushed them to remove this support...

[fdegros]

Thanks for this feature request. I think I understand what it is about.

However, I'm going to close it. At its heart, fuse-archive is a FUSE-based file system and I'm not planning to change that for the time being.

[vadimkantorov]

Got it! It would be curious to see if fuse-archive can be used with ldpfuse on a machine without the fuse kernel module

[vadimkantorov]

An "ordinary" user can use fuse-archive to mount an archive as a read-only filesystem.

Just FYI, if I understood well, on Docker/k8s for now, one would still require CAP_SYS_ADMIN or --prvilieged to use FUSE:

• https://github.com/awslabs/mountpoint-s3/issues/1204#issuecomment-2563726258
• https://github.com/yandex-cloud/geesefs/issues/127#issuecomment-2557852034