LLVM error in ::dlopen running on Mac, when CMake project loads another CMake project

[Ravbug]

Describe the bug Crash trying to load dynamic library on Mac, if the library is loaded as part of another library using CMake.

To Reproduce Steps to reproduce the behavior:

1. Input Shader:

// Example.hlsl

cbuffer Settings : register(b0) {
	float4x4 wvpMatrix;
};

struct InputVS
{
	float3 position : POSITION;
	float3 color : COLOR;
};

struct OutputVS
{
	float4 position : SV_Position;
	float3 color : COLOR;
};

// Vertex shader main function
OutputVS VS(InputVS inp)
{
	OutputVS outp;
	outp.position = mul(wvpMatrix,float4(inp.position, 1));
	outp.color = inp.color;
	return outp;
}

// Pixel shader main function
float4 PS(OutputVS inp) : SV_Target
{
	return float4(inp.color, 1);
};

2. Full parameters
   • ShaderSrc is C string of the input shader
   • Options left at defaults
   • TargetDesc::language = ShadingLanguage::Msl_macOS
   • TargetDesc::version = "1.0"
   • TargetDesc::asModule = true
3. File structure:

bug-repro
|
+ -- CMakeLists.txt
+ -- shader-transpiler
   |
   + -- CMakeLists.txt
   + -- main.cpp
   + -- ShaderConductor

Create a CMakeLists.txt in a folder named shader-transpiler with the following contents:

cmake_minimum_required(VERSION 3.17)
PROJECT(shader-transpiler)

add_subdirectory("${CMAKE_CURRENT_SOURCE_DIR}/ShaderConductor")

add_executable("${PROJECT_NAME}" "main.cpp")
target_include_directories("${PROJECT_NAME}" PRIVATE 
    "${ST_DEPS_DIR}/ShaderConductor/External/cxxopts/include"
    )

target_link_libraries("${PROJECT_NAME}" PRIVATE "ShaderConductor")

Example main.cpp:

#include <ShaderConductor/ShaderConductor.hpp>
#include <fstream>
#include <streambuf>
#include <cassert>
#include <iostream>

using namespace std;
using namespace ShaderConductor;

int main(){
	
	//read in source file
	auto name = "Example.hlsl";
	
	ifstream in(name);
	assert(in.good());
	string shader_src((std::istreambuf_iterator<char>(in)),
					std::istreambuf_iterator<char>());
	
	//describe the input
	Compiler::SourceDesc source;
	source.source = shader_src.c_str();
	source.entryPoint = "VS";
	//source.fileName = name;
	source.defines = nullptr;
	source.numDefines = 0;
	source.stage = ShaderStage::VertexShader;
	
	//default options
	Compiler::Options options;
	
	//describe the output
	Compiler::TargetDesc target;
	target.language = ShadingLanguage::Msl_macOS;
	target.version = "2.0";
	target.asModule = true;
	
	auto result = Compiler::Compile(source, options, target);
	
	ofstream out("Example_DS.metal", ios::out | (!result.isText ? ios::binary : 0 ));
	if(result.hasError){
		char* errormsg = (char*)result.errorWarningMsg->Data();
		cerr << "Compile Failed: " << errormsg << endl;
		return -1;
	}
	else{
		char* generatedsrc = (char*)result.target->Data();
		auto size = result.target->Size();
		
		for(int i = 0; i < size; i++){
			auto byte = (char)*(generatedsrc + i);
			out << byte;
		}
		
		cout << "Wrote result" << endl;
	}
	
	return 0;
}

4. Create a second CMakeLists.txt, one directory up, with the following contents:

project(bug_demo)
add_subdirectory("shader-transpiler")

5. Within the directory of the upper-level CMakeLists.txt, run mkdir build && cd build && cmake -G "Xcode" ..
6. Place Example.hlsl in the build directory so the program can open it.
7. Open the generated Xcode project, select the shader-transpiler target, and attempt to run it. The program will crash with the following error:

LLVM ERROR: inconsistency in registered CommandLine options
LLVM ERROR: IO failure on output stream.
Program ended with exit code: 1

The faulty line is: ShaderConductor.cpp:145

#ifdef _WIN32
            m_dxcompilerDll = ::LoadLibraryA(dllName);
#else
            m_dxcompilerDll = ::dlopen(dllName, RTLD_LAZY);    // <------ HERE
#endif

            if (m_dxcompilerDll != nullptr)
            {

Expected behavior Compiler::Compile should succeed.

Additional context The error does not occur if ShaderConductor is loaded from the top-level CMakeLists.txt. In the context of my project, this is not possible for me.

[gongminmin]

Can RPATH or build time link to dxc help?