nut-scanner: additional multiarch libraries search path

[aquette]

Standard libraries search paths, as currently done for nut-scanner to load libraries using lt_dlopen (search_paths[] + get_libname()) does not address multiarch environment, where one can cross-compile for other architecture. Hence, some libraries may not be found automatically, with the default configure options, and without specifying using --libdir. A workaround, at least for GCC enabled platform, would be to add a MULTIARCH_LIBDIR, mapped in configure checks to 'gcc -print-multiarch' For example, on Debian Jessie and Ubuntu Xenial, this returns 'x86_64-linux-gnu' and the multiarch libdir is '/usr/lib/x86_64-linux-gnu'

[jimklimov]

Work done for #1527 and #1548 can help with this quest.

[gdt]

I don't understand this. libdir has to be right, by default or being set. It is interpreted in terms of the target machine, and any binary that is cross built will only be run in that target environment.

Maybe this is really about "gcc -m32" which is not really cross.

FWIW, on NetBSD 9 amd64, gcc -print-multiarch is empty.

[jimklimov]

Just for more context: the nut-scanner is a very special piece of work architecturally. While NUT drivers support a lot of different media protocols, they do so one (or few) at a time, and run-time installation of NUT can be aggregated into optional sub-packages with compact dependency lists: one set for USB capabilities, another for SNMP, another for NetXML, another for avahi, modbus, ipmi and so on. End-users can install only e.g. nut-drivers-usb package and lack libsnmp locally.

The nut-scanner however has to speak everything possible, while pre-built as a binary once.

To achieve this, ltdl is used to optionally load third-party shared libraries and pointers to needed methods, or to safely say that protocol X will not be tried on this system and to march on with others. And it has a list of locations and expected filenames there to search explicitly, before falling back to ltdl to default something on its own.

So this depends on build-time (partially run-time by envvar) settings quite a bit.

For another piece of context: Some UPS vendors actually package/bundle NUT as their companion software, and this often means delivering all the dependencies pre-built for the basic edition of the platform (say, a build for Solaris 8 GA could cover all until now thanks to legacy compatibility..., clumsily for the services but functional for binaries). NUT tools, init-scripts etc. then often have to know to prefer their co-bundled locations for tested counterparts, instead of system-provided equivalents that may be a dozen releases newer and with different APIs.

Somewhat similar to how NUT for Windows tarballs now co-bundle lots of DLLs to make sure it works. Or how Docker co-bundles whole operating environments...

There are a lot of things that could be done differently - perhaps "correctly" - if the world was not a moving target, technology was not evolving, etc... but it is. And there still are needs for one-off builds that are not part of OS distro deeply intimate integration. And approaches for systems made 20 years ago and still running - so needing NUT in practice - have to be kept in mind...

[gdt]

Thanks for the background. So is the essence of this issue that nut-scanner has to load libs from places that are not in the standard search path, because

1. they are not built for this host, but somehow (??) can be dlopened into a binary for this host?
2. they are someplace else?

Wouldn't the solution for (2) be to pass something to configure which is "list of paths for nut-scanner"? Or to have a $PREFIX/etc/nut/nut-scanner.conf which has a setting?

I don't understand (1).

Or is it (3): because someone might have FooOS/i386 libraries, then even if building for FooOS/x86_64, then do a crossbuild for nut-scanner for i386, so it can dlopen those i386 binaries?

[jimklimov]

I think this issue was more about the (pre-built) contents of search_paths[] as in https://github.com/networkupstools/nut/blob/ad70749f243527e774c3f03a08228430143396e9/common/common.c#L1562 and considering its much smaller contents when the ticket was logged in 2016 and the code was part of nut-scanner codebase, moved in 2017 by https://github.com/networkupstools/nut/commit/e4430c620f8c3323c9231ec54b7e11a5ba9048bb commit (as part of DMF effort which spilled into main codebase to minimize the differences), which at that time covered just this semi-hard-coding:

/* FIXME: would be good to get more from /etc/ld.so.conf[.d] */
char * search_paths[] = {
	LIBDIR,
	"/usr"LIBDIR,
	"/usr/lib64",
	"/lib64",
	"/usr/lib",
	"/lib",
	"/usr/local/lib",
	NULL
};

Currently this array does optionally include locations like "/usr/lib/" AUTOTOOLS_TARGET_SHORT_ALIAS which on one hand may count as addressing much of this ticket, but on another hardcode the /usr/lib part which may not be ubiquitous (e.g. /usr/x86_64-w64-mingw32/lib/ or /mingw64/x86_64-w64-mingw32/lib are paths I commonly see in cross-builds of NUT for Windows).

Note that not all systems (e.g. embedded) do or can follow the LFS recommendations, so wild locations like --libdir=/run/current-system/sw/lib are also a thing (example from NixOS per #805).

[jimklimov]

As for a config file to list the locations - that idea apparently crossed our minds at various points (including the comment above to consult ld.so.conf where available), and does make sense (as a separate ticket) but did not happen yet.

[gdt]

I see. From the pkgsrc viewpoint, it's a bug to load from random places not in $PREFIX. Maybe make a dir in $PREFIX/lib/nut/extensions, use that, and people can symlink in modules they want to be found?

[jimklimov]

Well, for things NUT cares about and delivers itself (libnutclient, co-bundled DLLs on windows, etc.), or vendor bundles with NUT as their UPS companion SW, in that array there is the highest-priority LIBDIR (which by default is ${prefix}/lib but does not have to be as far as NUT is concerned).

I think (hard to remember after 7 years for a ticket proposed not even by myself) that the concern here was about systems peppering files into more than one of (/usr)/lib(BITS?)/(ARCH?) and (/usr)(/ARCH?)/lib and similar locations as normal part of the distro behavior, where several places may contain libraries that you can actually link for the current run-time.

[gdt]

I see. If the default has libs that are the right places according to target norms, that seems fine. I think that means moving the deciding to configure.ac, and subsituting a @DLOPEN_PATH@ or some such in a .c or .h. That would let people customize it easily at build time.

For background, on NetBSD, we don't use /usr/libb64 (which is basically a workaround for /usr/lib having i386 binaries on an x86_64 system). We put binaries for other arches under /emul. Things often incorrectly assume that one wants /usr/lib64 on x86_64.

[jimklimov]

Add a couple of lines, they said. Looked so simple, it did. Ended up a day worth of fun (and true bug fixes caught before releasing new code made after 2.8.0), typical... :)