AcademySoftwareFoundation
[FEATURE REQUEST] Add self-building abilities for more dependencies
We recently added (in PR #4242) the ability for OIIO's CMake build system to automatically download and build certain dependencies if they are not found.
(Note: It doesn't happen by default, it requires OpenImageIO_BUILD_MISSING_DEPS to be set to a comma-separated list of deps to self-build (or "all"). Also, OpenImageIO_BUILD_LOCAL_DEPS can be use to force listed dependencies to be built in this way, even if found.)
The original PR #4242 set this up for Imath, OpenEXR, fmt, Robinmap, and OpenColorIO. Subsequent PRs added the self-build ability for Freetype (#4283), libtiff and its libdeflate dependency (#4296), pybind11 (#4297), and improved that of OpenColorIO (#4367).
It would be great to have this ability added for other dependencies! Each can be tackled separately, and each is therefore a fairly self-contained task that doesn't require any prior deep knowledge of the OIIO internals, and could be a good first issue or Dev Days project.
The other high priority dependencies that would be good to add these self-build scripts for are:
- [ ] LibJPEG (maybe unnecessary because we can auto-build jpeg-turbo, which is better?)
- [x] libPNG
- [x] GIFlib
- [x] LibWEBP
- [ ] Ptex
- [ ] OpenJPEG
- [ ] libheif (may also require libde265 to be made)
- [ ] openjph
If you'd like to work on one of these, just comment below 👇 and say which one you will work on so that two people aren't trying to do the same one. Also feel free to ask below if you have any questions about what needs to be done, where to find examples of the ones that already work, or anything else that would help you get started.
I've actually already done libjpeg-turbo -- was necessary for building with cibuildwheel on the manylinux images. I'll submit a PR for that.
Hi @lgritz The GIFLIB project repository at https://gitlab.com/giflib/giflib.git appears to be no longer accessible.
It's one of the foundational libraries of the web. It must live somewhere, even if that particular location is no longer used. I'm very confident that you will be able to find it.
Also, make sure it's not just some kind of temporary glitch. When you run dozens of CI tests a day that download dependencies, you learn pretty quickly that any repo out there (or even a huge service like GitHub) will experience a surprising number of glitches and spurious failures to resolve a repo.
It looks like GIFLIB lives here: https://sourceforge.net/projects/giflib/
There's also a library called GIFLIB-Turbo, supposedly "a faster drop-in replacement for GIFLIB" that may be worth checking out. https://github.com/bitbank2/giflib-turbo Doesn't hurt that giflib-turbo lives on github and uses a cmake-based build system...!
I'm sorry, I didn't read or think hard enough about your message 3 weeks ago, @hisergiorojas. I should have recognized at the time that the Gitlab link was simply incorrect.
That's not where the project lives -- it's on sourceforge. In fact, if you look at our own helper script that our CI uses, OIIO's src/build-scripts/build_gif.bash, you can see that to build giflib from source, it also downloads from sourceforge.
@zachlewis: giflib, whatever its shortcomings may be, is one of the most widely used libraries in the known universe, and I think we can take for granted that it's stable and well maintained. Giflib-turbo is conceptually attractive but has only 37 stars, NO tagged releases ever, and seems to go years between source code edits. I have no knowledge that there's anything wrong with it per se, but given how unimportant GIF performance is to typical OIIO users, I think it's a fine project to bookmark and remember in case anything goes seriously wrong with the real giflib in the future, but it seems like switching to it would introduce a bunch of new risks while not solving any legit problems for us.
I think it's a fine project to bookmark and remember in case anything goes seriously wrong with the real giflib in the future, but it seems like switching to it would introduce a bunch of new risks while not solving any legit problems for us.
Ha -- sorry, I should have been more clear -- I didn't mean to suggest that the GIF plugin suddenly switches to using giflib-turbo instead of giflib in any official capacity...! I meant it more as a point of interest, I guess for the hardcore gifaholics with a need for speed, that they might not have a difficult time tricking custom OIIO builds into using giflib-turbo. Probably not too helpful to bring up in this thread, but... what's done is done.
As discussed in the slack thread, we'd like to flag interest in JPEG-2000 and JPEG-XL support in the PyPi wheels, which would come via the work noted in this issue.
Thank you!
Tagging @davisadam10 for vis.
Poking this for DevDays inclusion. In particular, still high on the hit list are builders for:
- GIF
- OpenJPEG (JPEG-2000)
- JPEG-XL
- openjph
Each of these can be a separate DD task for somebody.
@vangeliq Yes, please do! See the other examples in src/cmake/build_*.cmake for examples of what we did for other dependencies.
The hard part about giflib is that it doesn't have a cmake-based build system, so you can't directly use our build_dependency_with_cmake function. But it's also a really simple library, so it's probably straightforward to write a tiny CMakeLists.txt for it on our side. You probably will need a bit of CMake familiarity to do that.
I can take a crack at one of the jpgs this coming dev days. Maybe JPEG-2000?
@grdanny Sure. JPEG2000 is read by OpenJPEG. We have a bash script, build_OpenJPEG.bash, that we use to pre-build it on the CI machines. So that gives the basic recipe. But what we need is a build_OpenJPEG.cmake that does it from without our build system.
It's the same as the relationship between, say, build_libpng.bash and build_PNG.cmake.