OpenRV icon indicating copy to clipboard operation
OpenRV copied to clipboard

[Bug]: Windows Build: Missing OpenSSL dependencies

Open herronelou opened this issue 10 months ago • 8 comments

What happened?

Hello.

I'm attempting to complete my first build of OpenRV, and when building run into issues with OpenSSL.

I have been following the instructions from https://github.com/AcademySoftwareFoundation/OpenRV/blob/main/docs/build_system/config_windows.md and the only mention of OpenSSL is in the Qt section.

Possibly somewhat related tickets: https://github.com/AcademySoftwareFoundation/OpenRV/issues/581 https://github.com/AcademySoftwareFoundation/OpenRV/issues/682

List all the operating systems versions where this is happening

Windows 11

On what computer hardware is this happening?

Core i9-12900, Nvidia RTX 3080 Ti, 64Gb RAM

Relevant console log output

"C:\OpenRV\_build\cmake\dependencies\dependencies.vcxproj" (default target) (1) ->
       "C:\OpenRV\_build\cmake\dependencies\RV_DEPS_FFMPEG-stage-target.vcxproj" (default target) (
       11) ->
       "C:\OpenRV\_build\cmake\dependencies\RV_DEPS_FFMPEG.vcxproj" (default target) (29) ->
       (CustomBuild target) ->
         CUSTOMBUILD : error : openssl not found [C:\OpenRV\_build\cmake\dependencies\RV_DEPS_FFMPE
       G.vcxproj]
         C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsof
       t.CppCommon.targets(254,5): error MSB8066: Custom build for 'C:\OpenRV\_build\CMakeFiles\cfe
       1b385b767f99aba28de75c1583622\RV_DEPS_FFMPEG-download.rule;C:\OpenRV\_build\CMakeFiles\cfe1b
       385b767f99aba28de75c1583622\RV_DEPS_FFMPEG-update.rule;C:\OpenRV\_build\CMakeFiles\cfe1b385b
       767f99aba28de75c1583622\RV_DEPS_FFMPEG-patch.rule;C:\OpenRV\_build\CMakeFiles\cfe1b385b767f9
       9aba28de75c1583622\RV_DEPS_FFMPEG-configure.rule;C:\OpenRV\_build\CMakeFiles\cfe1b385b767f99
       aba28de75c1583622\RV_DEPS_FFMPEG-build.rule;C:\OpenRV\_build\CMakeFiles\cfe1b385b767f99aba28
       de75c1583622\RV_DEPS_FFMPEG-install.rule;C:\OpenRV\_build\CMakeFiles\ddfe0fbca828d26d63401d1
       d79fd8a3b\RV_DEPS_FFMPEG-complete.rule;C:\OpenRV\_build\CMakeFiles\dc0b82080efe119cccb8b58ef
       9885864\RV_DEPS_FFMPEG.rule;C:\OpenRV\cmake\dependencies\CMakeLists.txt' exited with code 1.

Environment variables

No response

Extra information

No response

Code of Conduct

  • [x] I agree to follow this project's Code of Conduct

herronelou avatar Mar 11 '25 18:03 herronelou

As I spend more time trying to resolve this I'm now realizing OpenSSL does get built when building the dependencies, so now trying to determine why FFMPEG is complaining about it.

herronelou avatar Mar 11 '25 23:03 herronelou

I noticed another warning during my build which could be related:

       "C:\OpenRV\_build\cmake\dependencies\dependencies.vcxproj" (default target) (1) ->
       "C:\OpenRV\_build\cmake\dependencies\RV_DEPS_OPENSSL-stage-target.vcxproj" (default target)
       (15) ->
       (CustomBuild target) ->
         C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsof
       t.CppCommon.targets(237,5): warning MSB8065: Custom build for item "C:\OpenRV\_build\CMakeFi
       les\d2708addcadbb8629f109d12d3d452c4\libcrypto-1_1-x64.dll.rule" succeeded, but specified ou
       tput "c:\openrv\_build\stage\app\lib\libcrypto-1_1-x64.dll" has not been created. This may c
       ause incremental build to work incorrectly. [C:\OpenRV\_build\cmake\dependencies\RV_DEPS_OPE
       NSSL-stage-target.vcxproj]
         C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsof
       t.CppCommon.targets(237,5): warning MSB8065: Custom build for item "C:\OpenRV\_build\CMakeFi
       les\d2708addcadbb8629f109d12d3d452c4\libcrypto-1_1-x64.dll.rule" succeeded, but specified ou
       tput "c:\openrv\_build\stage\app\lib\libssl-1_1-x64.dll" has not been created. This may caus
       e incremental build to work incorrectly. [C:\OpenRV\_build\cmake\dependencies\RV_DEPS_OPENSS
       L-stage-target.vcxproj]

Pulling the thread on this it got me to this cmake file:

https://github.com/AcademySoftwareFoundation/OpenRV/blob/main/cmake/dependencies/openssl.cmake#L241

The OUTPUT specifies 2 DLLs in the LIB folder, but the copies below copy these DLLs in the BIN folder. Adding extra copies (I wasn't sure if modifying them would make things worse) fixed the warning, but I'm still unable to compile.

herronelou avatar Mar 12 '25 03:03 herronelou

I managed to make a little bit more progress thanks to https://github.com/AcademySoftwareFoundation/OpenRV/discussions/469

1) get PCRE2 building

I had to edit the block in "C:\OpenRV\_build\RV_DEPS_PCRE2\src\configure.ac"

  # Platform specific issues
  NO_UNDEFINED=
  EXPORT_ALL_SYMBOLS=
  case $host_os in
-  cygwin* | mingw* )
+  cygwin* | mingw* | msys* )
      if test X"$enable_shared" = Xyes; then
        NO_UNDEFINED="-no-undefined"
        EXPORT_ALL_SYMBOLS="-Wl,--export-all-symbols"
      fi
      ;;
  esac

It's not ideal because it's a file that's not part of this project, but I did not find any way to pass it from cmake, everything I tried failed. If someone know how to set this from a file that is part of OpenRV I'd love to switch to that (to test: It seems liek it added LDFLAGS=-Wl,-no-undefined --no-create --no-recursion in the config command, I had managed to set LDFLAGS=-Wl,-no-undefined via cmake, but that wasn't working, maybe setting the whole thing would)

2) Get PCRE2 building with the right name

Cmake attempts to copy libpcre2-8-0.dll to the stage area, but for some reason it built as msys-pcre2-8-0.dll, same for the libpcre2-posix-3.dll that became msys-pcre2-posix-3.dll

Taking a page out of the OpenSSL cmake, I first make a copy of the DLLs with the right names, then let it do the copy, that seems to work. I wasn't sure if renaming instead of copying would break something else.

pcr2.cmake

IF(RV_TARGET_WINDOWS)
    SET(_pcre2_libname
        libpcre2-8-0${CMAKE_SHARED_LIBRARY_SUFFIX}
    )
    SET(_pcre2_libname_posix
        libpcre2-posix-3${CMAKE_SHARED_LIBRARY_SUFFIX}
    )

+   SET(_pcre2_badlibname
+       msys-pcre2-8-0${CMAKE_SHARED_LIBRARY_SUFFIX}
+    )
+   SET(_pcre2_badlibname_posix
+       msys-pcre2-posix-3${CMAKE_SHARED_LIBRARY_SUFFIX}
+   )

    SET(_pcre2_implibname
        libpcre2-8.dll.a
    )
    SET(_pcre2_implibname_posix
        libpcre2-posix.dll.a
    )

    SET(_pcre2_libpath
        ${_bin_dir}/${_pcre2_libname}
    )
    SET(_pcre2_libpath_posix
        ${_bin_dir}/${_pcre2_libname_posix}
    )

+   SET(_pcre2_badlibpath
+       ${_bin_dir}/${_pcre2_badlibname}
+   )
+   SET(_pcre2_badlibpath_posix
+       ${_bin_dir}/${_pcre2_badlibname_posix}
+   )

    SET(_pcre2_implibpath
        ${_lib_dir}/${_pcre2_implibname}
    )
    SET(_pcre2_implibpath_posix
        ${_lib_dir}/${_pcre2_implibname_posix}
    )

ENDIF()
# PCRE is not used for Linux and MacOS (Boost regex is used) in the current code.
ADD_CUSTOM_COMMAND(
    TARGET ${_target}
    POST_BUILD
+   COMMENT "Fixing bad lib names ${_pcre2_badlibname} and ${_pcre2_badlibname_posix} to ${_pcre2_libname} and ${_pcre2_libname_posix}"
+   COMMAND ${CMAKE_COMMAND} -E copy ${_pcre2_badlibpath} ${_pcre2_libpath}
+   COMMAND ${CMAKE_COMMAND} -E copy ${_pcre2_badlibpath_posix} ${_pcre2_libpath_posix}
    COMMENT "Installing ${_target}'s shared library into ${RV_STAGE_BIN_DIR}"
    # Copy library files manually since there are tools that are not needed in the bin folder.
    COMMAND ${CMAKE_COMMAND} -E copy ${_pcre2_libpath} ${_pcre2_libpath_posix} -t ${RV_STAGE_BIN_DIR}
)

3) Fixing openssl not found

I'm still stuck here.

I have attempted the patch suggested in https://github.com/Neumann-A/vcpkg/blob/eecdbfaec03c1e8dec3225dd17efa2daf779b89d/ports/ffmpeg/0002-detect-openssl.patch with no luck.

I wonder if it's again something related to msys, I get a suspiciously high number of syntax errors in my ffbuild/config.log, Since I started yesterday trying to compile OpenRv that has been the recurrent issue (https://github.com/AcademySoftwareFoundation/OpenRV/issues/712 yesterday..)

herronelou avatar Mar 12 '25 08:03 herronelou

I feel dumb, apparently I'm supposed to use "C:\msys64\mingw64.exe" rather than the default "C:\msys64\msys2.exe". This gets me a lot further and it seems like all the dependencies now build properly, but RV itself doesn't finish (no rv.exe in the final bin). I'll resume tomorrow.

herronelou avatar Mar 12 '25 09:03 herronelou

I'm glad to hear that you've made progress. Using the correct executable for MSYS64 is indeed crucial, as you've discovered.

Perhaps we should put the notice to use mingw64.exe at the top of the windows page in the documentation?

When you get the chance, update back here with the new errors and we'll see what we can do.

cedrik-fuoco-adsk avatar Mar 12 '25 12:03 cedrik-fuoco-adsk

Thanks @cedrik-fuoco-adsk, I don't know if necessarily at the top of the page, but maybe make it a little bit more explicit. Now that I know there is a difference between the executables the page already notes this, but the default that opens after install is msys2.exe, so I kept using that one assuming it was the same. I had absolutely seen this:

Image

Maybe this could just add "Do not use msys2.exe or any of the other executables installed by msys", for those like me who are exposed to msys for the very first time.

Regarding the cmpilation:

I restarted from scratch today (reverted my changes, plus rvclean).

This is the current errors I get. I had other higher priorities today so I did not look at it in details, but if these ring a bell I'd happily take a pointer.

       "C:\OpenRV\_build\main_executable.vcxproj" (default target) (1) ->
       "C:\OpenRV\_build\src\bin\apps\rv\rv.vcxproj" (default target) (3) ->
       "C:\OpenRV\_build\cmake\dependencies\RV_DEPS_OCIO.vcxproj" (default target) (24) ->
       (CustomBuild target) ->
         C:\OpenRV\_build\RV_DEPS_OCIO\build\ext\dist\include\pybind11\detail\common.h(213,10): error C1083: Cannot open include file: 'Python.h': No such file or directory [C:\OpenRV\_build\RV_DEPS_OCIO\build\src\bindings\python\PyOpenColorIO.vcxproj
       ] [C:\OpenRV\_build\cmake\dependencies\RV_DEPS_OCIO.vcxproj]
         C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(254,5): error MSB8066: Custom build for 'C:\OpenRV\_build\CMakeFiles\f43b5ad58213adfc506b0d32ec913bf4\RV_DEPS_OCIO-download.rule;C:\
       OpenRV\_build\CMakeFiles\f43b5ad58213adfc506b0d32ec913bf4\RV_DEPS_OCIO-update.rule;C:\OpenRV\_build\CMakeFiles\f43b5ad58213adfc506b0d32ec913bf4\RV_DEPS_OCIO-patch.rule;C:\OpenRV\_build\CMakeFiles\f43b5ad58213adfc506b0d32ec913bf4\RV_DEPS_OCIO-co
       nfigure.rule;C:\OpenRV\_build\CMakeFiles\f43b5ad58213adfc506b0d32ec913bf4\RV_DEPS_OCIO-build.rule;C:\OpenRV\_build\CMakeFiles\f43b5ad58213adfc506b0d32ec913bf4\RV_DEPS_OCIO-install.rule;C:\OpenRV\_build\CMakeFiles\ddfe0fbca828d26d63401d1d79fd8a3
       b\RV_DEPS_OCIO-complete.rule;C:\OpenRV\_build\CMakeFiles\dc0b82080efe119cccb8b58ef9885864\RV_DEPS_OCIO.rule;C:\OpenRV\cmake\dependencies\CMakeLists.txt' exited with code 1. [C:\OpenRV\_build\cmake\dependencies\RV_DEPS_OCIO.vcxproj]


       "C:\OpenRV\_build\main_executable.vcxproj" (default target) (1) ->
       "C:\OpenRV\_build\src\bin\apps\rv\rv.vcxproj" (default target) (3) ->
       "C:\OpenRV\_build\installed_image_formats.vcxproj" (default target) (21) ->
       "C:\OpenRV\_build\src\bin\imgtools\makeFBIOformats\formats.gto.vcxproj" (default target) (43) ->
         CUSTOMBUILD : error : cannot open fb plugin C:/OpenRV/_build/stage/app/PlugIns/ImageFormats/io_oiio.dll: "C:\OpenRV\_build\stage\app\PlugIns\ImageFormats\io_oiio.dll": The specified module could not be found. [C:\OpenRV\_build\src\bin\imgtool
       s\makeFBIOformats\formats.gto.vcxproj]
         C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(237,5): error MSB8066: Custom build for 'C:\OpenRV\_build\CMakeFiles\2ba4b800b6ff9c92f39365951f8e5068\formats.gto.rule' exited with
       code -1. [C:\OpenRV\_build\src\bin\imgtools\makeFBIOformats\formats.gto.vcxproj]

    5042 Warning(s)
    4 Error(s)

If not I'll resume next time I have a few hours in front of me.

herronelou avatar Mar 12 '25 23:03 herronelou

I finally managed to build RV but I don't quite know how.

I did a full reboot of my machine, re-pulled everything, and was able to build the dependencies right away.

Upon building the main executable, however, I still had the error:

"C:\OpenRV\_build\main_executable.vcxproj" (default target) (1) ->
       "C:\OpenRV\_build\src\bin\apps\rv\rv.vcxproj" (default target) (3) ->
       "C:\OpenRV\_build\installed_image_formats.vcxproj" (default target) (21) ->
       "C:\OpenRV\_build\src\bin\imgtools\makeFBIOformats\formats.gto.vcxproj" (default target) (43) ->
         CUSTOMBUILD : error : cannot open fb plugin C:/OpenRV/_build/stage/app/PlugIns/ImageFormats/io_oiio.dll: "C:\OpenRV\_build\stage\app\PlugIns\ImageFormats\io_oiio.dll": The specified module could not be found. [C:\OpenRV\_build\src\bin\imgtool
       s\makeFBIOformats\formats.gto.vcxproj]
         C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(237,5): error MSB8066: Custom build for 'C:\OpenRV\_build\CMakeFiles\2ba4b800b6ff9c92f39365951f8e5068\formats.gto.rule' exited with
       code -1. [C:\OpenRV\_build\src\bin\imgtools\makeFBIOformats\formats.gto.vcxproj]

However, simply retrying to build the main executable a second time in the same session without cleaning things made it go through.

I'm going to keep this ticket open for a little bit, there were a couple of legit issues above that I might make MRs for.

herronelou avatar Mar 15 '25 06:03 herronelou

The switch to MSYS2 MINGW64 with a reinstall also got me through the openssl error. Thanks for the reminder!

wywarren avatar Apr 14 '25 03:04 wywarren