[Bug]: Windows Build: Missing OpenSSL dependencies
What happened?
Hello.
I'm attempting to complete my first build of OpenRV, and when building run into issues with OpenSSL.
I have been following the instructions from https://github.com/AcademySoftwareFoundation/OpenRV/blob/main/docs/build_system/config_windows.md and the only mention of OpenSSL is in the Qt section.
Possibly somewhat related tickets: https://github.com/AcademySoftwareFoundation/OpenRV/issues/581 https://github.com/AcademySoftwareFoundation/OpenRV/issues/682
List all the operating systems versions where this is happening
Windows 11
On what computer hardware is this happening?
Core i9-12900, Nvidia RTX 3080 Ti, 64Gb RAM
Relevant console log output
"C:\OpenRV\_build\cmake\dependencies\dependencies.vcxproj" (default target) (1) ->
"C:\OpenRV\_build\cmake\dependencies\RV_DEPS_FFMPEG-stage-target.vcxproj" (default target) (
11) ->
"C:\OpenRV\_build\cmake\dependencies\RV_DEPS_FFMPEG.vcxproj" (default target) (29) ->
(CustomBuild target) ->
CUSTOMBUILD : error : openssl not found [C:\OpenRV\_build\cmake\dependencies\RV_DEPS_FFMPE
G.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsof
t.CppCommon.targets(254,5): error MSB8066: Custom build for 'C:\OpenRV\_build\CMakeFiles\cfe
1b385b767f99aba28de75c1583622\RV_DEPS_FFMPEG-download.rule;C:\OpenRV\_build\CMakeFiles\cfe1b
385b767f99aba28de75c1583622\RV_DEPS_FFMPEG-update.rule;C:\OpenRV\_build\CMakeFiles\cfe1b385b
767f99aba28de75c1583622\RV_DEPS_FFMPEG-patch.rule;C:\OpenRV\_build\CMakeFiles\cfe1b385b767f9
9aba28de75c1583622\RV_DEPS_FFMPEG-configure.rule;C:\OpenRV\_build\CMakeFiles\cfe1b385b767f99
aba28de75c1583622\RV_DEPS_FFMPEG-build.rule;C:\OpenRV\_build\CMakeFiles\cfe1b385b767f99aba28
de75c1583622\RV_DEPS_FFMPEG-install.rule;C:\OpenRV\_build\CMakeFiles\ddfe0fbca828d26d63401d1
d79fd8a3b\RV_DEPS_FFMPEG-complete.rule;C:\OpenRV\_build\CMakeFiles\dc0b82080efe119cccb8b58ef
9885864\RV_DEPS_FFMPEG.rule;C:\OpenRV\cmake\dependencies\CMakeLists.txt' exited with code 1.
Environment variables
No response
Extra information
No response
Code of Conduct
- [x] I agree to follow this project's Code of Conduct
As I spend more time trying to resolve this I'm now realizing OpenSSL does get built when building the dependencies, so now trying to determine why FFMPEG is complaining about it.
I noticed another warning during my build which could be related:
"C:\OpenRV\_build\cmake\dependencies\dependencies.vcxproj" (default target) (1) ->
"C:\OpenRV\_build\cmake\dependencies\RV_DEPS_OPENSSL-stage-target.vcxproj" (default target)
(15) ->
(CustomBuild target) ->
C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsof
t.CppCommon.targets(237,5): warning MSB8065: Custom build for item "C:\OpenRV\_build\CMakeFi
les\d2708addcadbb8629f109d12d3d452c4\libcrypto-1_1-x64.dll.rule" succeeded, but specified ou
tput "c:\openrv\_build\stage\app\lib\libcrypto-1_1-x64.dll" has not been created. This may c
ause incremental build to work incorrectly. [C:\OpenRV\_build\cmake\dependencies\RV_DEPS_OPE
NSSL-stage-target.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsof
t.CppCommon.targets(237,5): warning MSB8065: Custom build for item "C:\OpenRV\_build\CMakeFi
les\d2708addcadbb8629f109d12d3d452c4\libcrypto-1_1-x64.dll.rule" succeeded, but specified ou
tput "c:\openrv\_build\stage\app\lib\libssl-1_1-x64.dll" has not been created. This may caus
e incremental build to work incorrectly. [C:\OpenRV\_build\cmake\dependencies\RV_DEPS_OPENSS
L-stage-target.vcxproj]
Pulling the thread on this it got me to this cmake file:
https://github.com/AcademySoftwareFoundation/OpenRV/blob/main/cmake/dependencies/openssl.cmake#L241
The OUTPUT specifies 2 DLLs in the LIB folder, but the copies below copy these DLLs in the BIN folder. Adding extra copies (I wasn't sure if modifying them would make things worse) fixed the warning, but I'm still unable to compile.
I managed to make a little bit more progress thanks to https://github.com/AcademySoftwareFoundation/OpenRV/discussions/469
1) get PCRE2 building
I had to edit the block in "C:\OpenRV\_build\RV_DEPS_PCRE2\src\configure.ac"
# Platform specific issues
NO_UNDEFINED=
EXPORT_ALL_SYMBOLS=
case $host_os in
- cygwin* | mingw* )
+ cygwin* | mingw* | msys* )
if test X"$enable_shared" = Xyes; then
NO_UNDEFINED="-no-undefined"
EXPORT_ALL_SYMBOLS="-Wl,--export-all-symbols"
fi
;;
esac
It's not ideal because it's a file that's not part of this project, but I did not find any way to pass it from cmake, everything I tried failed.
If someone know how to set this from a file that is part of OpenRV I'd love to switch to that (to test: It seems liek it added LDFLAGS=-Wl,-no-undefined --no-create --no-recursion in the config command, I had managed to set LDFLAGS=-Wl,-no-undefined via cmake, but that wasn't working, maybe setting the whole thing would)
2) Get PCRE2 building with the right name
Cmake attempts to copy libpcre2-8-0.dll to the stage area, but for some reason it built as msys-pcre2-8-0.dll, same for the libpcre2-posix-3.dll that became msys-pcre2-posix-3.dll
Taking a page out of the OpenSSL cmake, I first make a copy of the DLLs with the right names, then let it do the copy, that seems to work. I wasn't sure if renaming instead of copying would break something else.
pcr2.cmake
IF(RV_TARGET_WINDOWS)
SET(_pcre2_libname
libpcre2-8-0${CMAKE_SHARED_LIBRARY_SUFFIX}
)
SET(_pcre2_libname_posix
libpcre2-posix-3${CMAKE_SHARED_LIBRARY_SUFFIX}
)
+ SET(_pcre2_badlibname
+ msys-pcre2-8-0${CMAKE_SHARED_LIBRARY_SUFFIX}
+ )
+ SET(_pcre2_badlibname_posix
+ msys-pcre2-posix-3${CMAKE_SHARED_LIBRARY_SUFFIX}
+ )
SET(_pcre2_implibname
libpcre2-8.dll.a
)
SET(_pcre2_implibname_posix
libpcre2-posix.dll.a
)
SET(_pcre2_libpath
${_bin_dir}/${_pcre2_libname}
)
SET(_pcre2_libpath_posix
${_bin_dir}/${_pcre2_libname_posix}
)
+ SET(_pcre2_badlibpath
+ ${_bin_dir}/${_pcre2_badlibname}
+ )
+ SET(_pcre2_badlibpath_posix
+ ${_bin_dir}/${_pcre2_badlibname_posix}
+ )
SET(_pcre2_implibpath
${_lib_dir}/${_pcre2_implibname}
)
SET(_pcre2_implibpath_posix
${_lib_dir}/${_pcre2_implibname_posix}
)
ENDIF()
# PCRE is not used for Linux and MacOS (Boost regex is used) in the current code.
ADD_CUSTOM_COMMAND(
TARGET ${_target}
POST_BUILD
+ COMMENT "Fixing bad lib names ${_pcre2_badlibname} and ${_pcre2_badlibname_posix} to ${_pcre2_libname} and ${_pcre2_libname_posix}"
+ COMMAND ${CMAKE_COMMAND} -E copy ${_pcre2_badlibpath} ${_pcre2_libpath}
+ COMMAND ${CMAKE_COMMAND} -E copy ${_pcre2_badlibpath_posix} ${_pcre2_libpath_posix}
COMMENT "Installing ${_target}'s shared library into ${RV_STAGE_BIN_DIR}"
# Copy library files manually since there are tools that are not needed in the bin folder.
COMMAND ${CMAKE_COMMAND} -E copy ${_pcre2_libpath} ${_pcre2_libpath_posix} -t ${RV_STAGE_BIN_DIR}
)
3) Fixing openssl not found
I'm still stuck here.
I have attempted the patch suggested in https://github.com/Neumann-A/vcpkg/blob/eecdbfaec03c1e8dec3225dd17efa2daf779b89d/ports/ffmpeg/0002-detect-openssl.patch with no luck.
I wonder if it's again something related to msys, I get a suspiciously high number of syntax errors in my ffbuild/config.log, Since I started yesterday trying to compile OpenRv that has been the recurrent issue (https://github.com/AcademySoftwareFoundation/OpenRV/issues/712 yesterday..)
I feel dumb, apparently I'm supposed to use "C:\msys64\mingw64.exe" rather than the default "C:\msys64\msys2.exe".
This gets me a lot further and it seems like all the dependencies now build properly, but RV itself doesn't finish (no rv.exe in the final bin).
I'll resume tomorrow.
I'm glad to hear that you've made progress. Using the correct executable for MSYS64 is indeed crucial, as you've discovered.
Perhaps we should put the notice to use mingw64.exe at the top of the windows page in the documentation?
When you get the chance, update back here with the new errors and we'll see what we can do.
Thanks @cedrik-fuoco-adsk, I don't know if necessarily at the top of the page, but maybe make it a little bit more explicit. Now that I know there is a difference between the executables the page already notes this, but the default that opens after install is msys2.exe, so I kept using that one assuming it was the same. I had absolutely seen this:
Maybe this could just add "Do not use msys2.exe or any of the other executables installed by msys", for those like me who are exposed to msys for the very first time.
Regarding the cmpilation:
I restarted from scratch today (reverted my changes, plus rvclean).
This is the current errors I get. I had other higher priorities today so I did not look at it in details, but if these ring a bell I'd happily take a pointer.
"C:\OpenRV\_build\main_executable.vcxproj" (default target) (1) ->
"C:\OpenRV\_build\src\bin\apps\rv\rv.vcxproj" (default target) (3) ->
"C:\OpenRV\_build\cmake\dependencies\RV_DEPS_OCIO.vcxproj" (default target) (24) ->
(CustomBuild target) ->
C:\OpenRV\_build\RV_DEPS_OCIO\build\ext\dist\include\pybind11\detail\common.h(213,10): error C1083: Cannot open include file: 'Python.h': No such file or directory [C:\OpenRV\_build\RV_DEPS_OCIO\build\src\bindings\python\PyOpenColorIO.vcxproj
] [C:\OpenRV\_build\cmake\dependencies\RV_DEPS_OCIO.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(254,5): error MSB8066: Custom build for 'C:\OpenRV\_build\CMakeFiles\f43b5ad58213adfc506b0d32ec913bf4\RV_DEPS_OCIO-download.rule;C:\
OpenRV\_build\CMakeFiles\f43b5ad58213adfc506b0d32ec913bf4\RV_DEPS_OCIO-update.rule;C:\OpenRV\_build\CMakeFiles\f43b5ad58213adfc506b0d32ec913bf4\RV_DEPS_OCIO-patch.rule;C:\OpenRV\_build\CMakeFiles\f43b5ad58213adfc506b0d32ec913bf4\RV_DEPS_OCIO-co
nfigure.rule;C:\OpenRV\_build\CMakeFiles\f43b5ad58213adfc506b0d32ec913bf4\RV_DEPS_OCIO-build.rule;C:\OpenRV\_build\CMakeFiles\f43b5ad58213adfc506b0d32ec913bf4\RV_DEPS_OCIO-install.rule;C:\OpenRV\_build\CMakeFiles\ddfe0fbca828d26d63401d1d79fd8a3
b\RV_DEPS_OCIO-complete.rule;C:\OpenRV\_build\CMakeFiles\dc0b82080efe119cccb8b58ef9885864\RV_DEPS_OCIO.rule;C:\OpenRV\cmake\dependencies\CMakeLists.txt' exited with code 1. [C:\OpenRV\_build\cmake\dependencies\RV_DEPS_OCIO.vcxproj]
"C:\OpenRV\_build\main_executable.vcxproj" (default target) (1) ->
"C:\OpenRV\_build\src\bin\apps\rv\rv.vcxproj" (default target) (3) ->
"C:\OpenRV\_build\installed_image_formats.vcxproj" (default target) (21) ->
"C:\OpenRV\_build\src\bin\imgtools\makeFBIOformats\formats.gto.vcxproj" (default target) (43) ->
CUSTOMBUILD : error : cannot open fb plugin C:/OpenRV/_build/stage/app/PlugIns/ImageFormats/io_oiio.dll: "C:\OpenRV\_build\stage\app\PlugIns\ImageFormats\io_oiio.dll": The specified module could not be found. [C:\OpenRV\_build\src\bin\imgtool
s\makeFBIOformats\formats.gto.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(237,5): error MSB8066: Custom build for 'C:\OpenRV\_build\CMakeFiles\2ba4b800b6ff9c92f39365951f8e5068\formats.gto.rule' exited with
code -1. [C:\OpenRV\_build\src\bin\imgtools\makeFBIOformats\formats.gto.vcxproj]
5042 Warning(s)
4 Error(s)
If not I'll resume next time I have a few hours in front of me.
I finally managed to build RV but I don't quite know how.
I did a full reboot of my machine, re-pulled everything, and was able to build the dependencies right away.
Upon building the main executable, however, I still had the error:
"C:\OpenRV\_build\main_executable.vcxproj" (default target) (1) ->
"C:\OpenRV\_build\src\bin\apps\rv\rv.vcxproj" (default target) (3) ->
"C:\OpenRV\_build\installed_image_formats.vcxproj" (default target) (21) ->
"C:\OpenRV\_build\src\bin\imgtools\makeFBIOformats\formats.gto.vcxproj" (default target) (43) ->
CUSTOMBUILD : error : cannot open fb plugin C:/OpenRV/_build/stage/app/PlugIns/ImageFormats/io_oiio.dll: "C:\OpenRV\_build\stage\app\PlugIns\ImageFormats\io_oiio.dll": The specified module could not be found. [C:\OpenRV\_build\src\bin\imgtool
s\makeFBIOformats\formats.gto.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(237,5): error MSB8066: Custom build for 'C:\OpenRV\_build\CMakeFiles\2ba4b800b6ff9c92f39365951f8e5068\formats.gto.rule' exited with
code -1. [C:\OpenRV\_build\src\bin\imgtools\makeFBIOformats\formats.gto.vcxproj]
However, simply retrying to build the main executable a second time in the same session without cleaning things made it go through.
I'm going to keep this ticket open for a little bit, there were a couple of legit issues above that I might make MRs for.
The switch to MSYS2 MINGW64 with a reinstall also got me through the openssl error. Thanks for the reminder!