Exa-Networks
Helper script should not be in /etc/exabgp
ISSUE TYPE
- Feature Idea
OS
CentOS Linux
VERSION
ExaBGP : 3.4.15
Python : 2.7.3 (f66246c46ca30b26a5c73e4cc95dd6235c966b8f, Jul 30 2013, 09:27:06) [PyPy 2.0.2 with GCC 4.4.7 20120313 (Red Hat 4.4.7-3)]
Uname : #1 SMP Wed Jul 12 14:17:22 UTC 2017
ENVIRONMENT
Not applicable
CONFIGURATION
Not applicable
SUMMARY
When exabgp is packaged as a wheel, the helper scripts, such as "watchdog-1.pl", "dynamic-1.pl" and so on are getting installed into etc/exabgp. The "etc" directory is meant to be for configuration, and NOT for scripts. Helper scripts, and in fact, scripts of any kind, belong in "bin" or "sbin".
STEPS TO REPRODUCE
Just install exabgp anywhere from a tarball, or make a wheel of it.
EXPECTED RESULTS
Scripts should be installed into "bin" or "sbin", not "etc/exabgp".
ACTUAL RESULTS
Executable scripts get installed into "etc/exabgp".
IMPORTANCE
It hasn't broken anything, but it's messy. When a package installs executable scripts, I expect them in "bin" or "sbin" directories, not in /etc.
Hello Anand, I think your interpretation of the LSB is may be a bit strict, you would not delete all the /etc/init.d scripts from your distribution - would you not ? :-)
I agree that in production another location would be better but as these scripts are part of the configuration, and provided as examples, I believe that etc is acceptable as it will make it easy for users to find them.
But IMHO in production, the ExaBGP scripts could/should indeed be installed in /usr/lib/exabgp like other program such as postfix but not in bin or sbin folders as they are not to be run on their own.
I had not ever installed exabgp using pip before but looking at it I see the examples and scripts in /usr/local/share/exabgp. processes seems to have files but not etc, so something may be wrong and needing fixing ... sigh !
Therefore the only place I can see things in /etc is the root of the repo which is good as people looking for examples are not asking me half as often as they used to :-)
Does this reflect your experience of the installation via pip too ? Or are you still thinking something still need changing ?
Hi Thomas! You're right about scripts in /etc/init.d of course :) You're also right in that these "helper" scripts ideally belong in /usr/lib/exabgp, because they are not going to run stand-alone, but rather invoked from the exabgp process to assist with certain functions. I think it's good that we both agree that they really should not be in /etc.
I'm looking at the setup.py file of version 3.4.18. In there, the setup() function's "data_files" parameter lists files that are packaged as "data" when you build an egg or a wheel. It is calling a function called "files_definition" which is building a list of files from /etc/exabgp, and asking them to be installed in /etc/exabgp, unless the platform is NetBSD (in which case it uses the share/exabgp prefix). This is why all these files are ending up in /etc/exabgp. So, at least until 3.4.18, things are still bad.
In 3.4.20, things look somewhat better, where the helpers are going into share/exabgp/processes, and the example configs into share/exabgp/etc. However, as is the convention, you should probably be installing examples into share/exabgp/examples. And the helpers should go into lib/exabgp.
Moving on, in HEAD, the setup.py file's setup() function is different yet again. This time, the helpers are all being installed into share/exabgp/processes, but there is no example configuration. I don't know if this intentional or an oversight. In any case, I'm not too bothered if example configuration is not packaged into a wheel.
Let me explain the reason for all my thinking. On our servers, we install exabgp from a pre-built wheel. We install exabgp into a virtualenv, as is common with python applications. This is to keep the exabgp code isolated inside its own environment. Our virtualenv on the filesystem is /opt/exabgp.
Now, we also need configuration. In my opinion, configuration files should be in /etc/exabgp, and this is a directory we should create and maintain using our configuration management tools. However, because the version we are using (3.4.15) creates /opt/exabgp/etc/exabgp with scripts in there, we have been keeping the exabgp config files there too. But I don't like that. I would prefer if exabgp installations, whether from sources or eggs or wheels, drop just the code into the virtualenv, without shipping any config files there, or without expecting to find config files in there. Config files should always be expected at /etc/exabgp.
I know this is a long message and audit, but I hope you understand why I'm doing it, because packaging is important, especially if you want to be as close as possible to FHS as possible.
Finally, I have discovered spelling errors in the setup.py of HEAD, so I'm going to have to submit a PR just now.
Thank you for the long write-up. I am happy to change the behaviour of both 3.4 and master in that aspect. As I am not a pip expert (I can do stuff but I realise that there is many auto-magic things I do not know) I would be happy to take patches to fix stuff on both trees. Looking forward to your PR.
Hi Thomas. There is no magic in pip. When someone wants to install exabgp somewhere, they can download the tarball, unpack it and run "python setup.py install" to install it. However, instead of "install", if you run "python setup.py bdist_wheel", the setup script does the same installation into a temporary directory, and then packages up all the installed files into a compressed archive called a "wheel". This wheel can be installed on one or more systems, even if they don't have python's setuptools. Think of it in the same way as packaging up things into .deb or .rpm packages.
You'll noticed that I didn't mention "eggs" because eggs are an older archive format that is discouraged. Wheels are the modern and new way of doing things. Wheels are also not supposed to contain any distro-specific things like SysV init scripts or systemd unit files. And wheels should also not assume that they will be installed into the root file system of a target. They could get installed into a virtualenv, and all files will be relative to that virtualenv.
@aabdnn I notice this did not move forward .. can you help and get this sorted - otherwise I fear it will have to wait a little bit more as my focus is to fix the last issues of 4.0 to have a stable release.
@aabdnn I would welcome a patch to implement this .. something you can lend us time for ?