How to handle external dependencies (Was: Consider replacing ExternalProject with Hunter)

[jolting]

You can read about it here: https://docs.hunter.sh/en/latest/

It actually uses ExternalProject, but it seems to have support for many libraries built in and the syntax is amazing.

From their example:

hunter_add_package(Boost COMPONENTS system filesystem iostreams)
find_package(Boost CONFIG REQUIRED system filesystem iostreams)

https://github.com/ruslo/hunter

To-do List:

• [x] eigen3
• [x] assimp
• [ ] libtclap
• [ ] boost for python wrappers
• [ ] opencv
• [ ] suitesparse? (test for windows)
• [ ] xmlparser ? (or just remove from mrpt?)

[jlblancoc]

(Moved from another issue ticket) To do:

• [x] Eigen.
• [x] assimp
• [ ] libtclap => don't download sources by default, show an error, and an option to tick (DOWNLOAD_XXX) so the user controls whether to install system pkgs or to download & build from sources.

[jlblancoc]

Done for eigen3.

[jlblancoc]

@jolting Right now, assimp and Eigen are downloaded directly using CMake commands.

libtclap-dev could be autodetected from Debian packages, or if not found, via Hunter:

hunter_add_package(TCLAP)

Any other package that may be interesting to add via Hunter?

[jolting]

Boost for python bindings on windows. OpenCV could be done via Hunter as well SuiteSparse

Some of the ROS stuff is available.

[jlblancoc]

Cool. I updated the first comment with a list.

[lsolanka]

Would this be useful: https://github.com/MRPT/mrpt/compare/master...lsolanka:hunter-master?

I am actually running some of my simple code that uses this version. My main aim is to use mrpt through Hunter though. The idea there is to replace most of the libs with those from Hunter - so it's a slightly larger patch than you are perhaps expecting.

I also have personal version of Octomap and libcvd which is not in official Hunter yet but could be added I suppose.

The diff above also touches a little bit on #602.

[jolting]

Cool!!!! One of the things that I have struggled with when using Hunter is the opinionated nature of the package manager when it comes to mixing system packages with Hunter packages. Basically, you can either use Hunter packages or system packages, but not both. We'll have to make sure this doesn't break the Debian build. I think for Windows and Mac always using Hunter is fine.

[jolting]

Looks like it needs

option(HUNTER_ENABLED "Enable Hunter package manager" OFF)

This for some reason doesn't work when HUNTER is disabled

   FIND_PACKAGE(JPEG CONFIG REQUIRED)

Specifically, the CONFIG REQUIRED doesn't work.

External Package is still required when HUNTER is disabled for CVD because we build that as a static lib. It has never been debianized.

[jlblancoc]

Thanks a lot @lsolanka ! I'll need to look more in detail to your patch, but I like the idea.

The problem we have is trying to make happy everyone: Windows, MacOS (?) and Linux users... we must be sure the final version works by detecting system libraries for the Debian guys to be happy (and I agree with that opinion that system libs should be preferred in general!).

Travis could be a great help because MRPT examples are all compiled using FIND_PACKAGE(MRPT), so that's a good testbed to check if user apps will work.

[jolting]

When HUNTER is on using all HUNTER libs and when HUNTER is off use only system libs and EXTERNAL package.

It will require a light sprinkling of IF(HUNTER_ENABLED). I'm ok with that.

[lsolanka]

Thanks for the feedback, yes, the patch I linked to doesn't support HUNTER_ENABLED=OFF, mainly because some libraries need to be found in CONFIG mode when Hunter is enabled. So as @jolting mentioned there might be some if statements needed and a few reverts to parts of the patch, e.g. for JPEG.

It is possible to mix and match system libraries and Hunter libraries, but it needs much more care and I guess in general it brings more problems than it solves. If you do hunter_add_package(Boost) then every dependency in your project that uses Boost will need to be coming through Hunter. This can be a problem in some cases.

One thing to note is that whatever is pulled through Hunter is linked in statically by default -> something that might increase the size of .so/.dll.

I agree that on Debian it is best to link against system libraries. The biggest benefit would be on platforms where it is hard to use system libraries, such as Windows (but I'm not really a user of this platform so not much experience there...).

[jolting]

Probably worth reading from here: https://github.com/ruslo/hunter/issues/501

[jlblancoc]

Closing due to inactivity... Using git submodules and detecting system libraries is probably already good enough for most use cases at present (!?).