MSVC RelWithDebInfo configuration uses mixture of debug and release libraries

[DomClark]

Bug Summary

When building with MSVC using the RelWithDebInfo configuration, a mixture of debug and release libraries is used.

Steps to reproduce

Build the master branch with MSVC using RelWithDebInfo configuration, or download an MSVC build from CI, since that's the configuration we use there.

Expected behavior

All libraries used should belong to the same configuration (preferably Release).

Actual behavior

LMMS uses the debug CRT, and debug Qt libraries, but release versions of the libraries from vcpkg.

Affected LMMS versions

MSVC RelWithDebInfo builds of master branch.

Further information

This leads to crashes when, e.g., trying to load samples. LMMS and Qt use the debug CRT to open files, but pass these file handles to libsndfile, which is using the release CRT. This causes a crash when the release CRT is asked to read a file using a handle it doesn't recognise.

This could be addressed by setting the CMAKE_MAP_IMPORTED_CONFIG_RELWITHDEBINFO variable to RelWithDebInfo Release, to instruct CMake to use Release libraries for the RelWithDebInfo configuration if RelWithDebInfo versions don't exist, instead of Debug libraries. This won't work for SDL2 however - we have a custom FindSDL2.cmake module, which takes precedence over SDL2's own SDL2Config.cmake script, and doesn't know how to find vcpkg's SDL2 debug build.

[DomClark]

Actually tried building RelWithDebInfo locally, and it's backwards on my machine. I get the release version of the CRT and Qt, but debug libraries from vcpkg. (SDL2 is still release for the aforementioned reason.) I'm not sure why - perhaps newer versions of Qt and vcpkg? Either way, the inconsistency is a problem.

[tresf]

I get the release version of the CRT and Qt, but debug libraries from vcpkg.

CRT seems to be an odd duck... I searched and found solutions involving forced flags and generator expressions. Almost smells like a CMake bug. https://stackoverflow.com/questions/8335499/cmake-visual-studio-debug-release-config https://cmake.org/pipermail/cmake/2019-June/069633.html

Qt is even more of a mystery. From what I'm reading, FindQt doesn't seem to handle this. I see some talking about the debug paths being different, reminds me a bit of https://github.com/tresf/lmms/commit/a7f8251f4b11ee051c47ca85ccc94b6bfe78c8c1#diff-0cfe54b7b7513b8854b14b3fb8411b78R44 but anyway, the tutorials all seem to have some Qt-specific debug flag. The GUI shows a setting called "Build variant" but some articles reference something called configure.bat. Even CLion has some oddball steps for copying debug libraries: https://www.jetbrains.com/help/clion/qt-tutorial.html#troubleshooting

I'm sure when it's resolved we'll find it's easier than all of that, but I wanted to share my research as I don't think we're alone.

[tresf]

Found a great write-up on this.... quoting:

Multi-configuration IDE generators: Visual Studio, Xcode

CMAKE_BUILD_TYPE on generate step is ignored

So apparently it's 100% dependant on the --config Release flag? I think I'm getting this right... https://stackoverflow.com/a/24470998/3196753. It has 200 upvotes and I can't find it documented much elsewhere.

[DomClark]

GitHub Actions MSVC RelWithDebInfo builds now use the correct library versions (only release versions), so this bug should no longer affect MSVC builds downloaded from GitHub. I still don't know why the AppVeyor builds used debug versions of Qt, but that's no longer relevant.

On my machine, the problem persists as described in my earlier comment - I get release versions of Qt, but debug versions of vcpkg libraries. This is because I use a multi-config CMake generator (specifically Ninja Multi-Config, but others should be affected too), but the vcpkg library search order is specific to the value of CMAKE_BUILD_TYPE at configure time:

# https://github.com/microsoft/vcpkg/blob/9b22b40c6c61bf0da2d46346dd44a11e90972cc9/scripts/buildsystems/vcpkg.cmake#L396-L402
set(vcpkg_paths
    "${_VCPKG_INSTALLED_DIR}/${VCPKG_TARGET_TRIPLET}${suffix}"
    "${_VCPKG_INSTALLED_DIR}/${VCPKG_TARGET_TRIPLET}/debug${suffix}"
)
if(NOT DEFINED CMAKE_BUILD_TYPE OR CMAKE_BUILD_TYPE MATCHES "^[Dd][Ee][Bb][Uu][Gg]$")
    list(REVERSE vcpkg_paths) # Debug build: Put Debug paths before Release paths.
endif()

This is incompatible with a multi-config generator, leading to the same set of libraries being chosen for all configurations, and so will offer incorrect libraries either for debug builds, or for RelWithDebInfo and release builds.