codon icon indicating copy to clipboard operation
codon copied to clipboard
verified publisher icon exaloop

Generated LLVM code can't be used by lli

Open pramit-j2-sl opened this issue 1 year ago • 3 comments

Installed codon (0.16.3) with /bin/bash -c "$(curl -fsSL https://exaloop.io/install.sh)" Created a fib.py with

    def fib(n):
        a, b = 0, 1
        while a < n:
            print(a, end=' ')
            a, b = b, a+b
        print()
    fib(1000)

generated fib.ll file with codon build -release -llvm fib.py

Run the generated ll file with lli (18.1.4) gives the following core-dump:

    lli: /data/learn-compiler/test1/llvm/llvm/lib/CodeGen/MachineFunction.cpp:245: void llvm::MachineFunction::init(): Assertion `Target.isCompatibleDataLayout(getDataLayout()) && "Can't create a MachineFunction using a Module with a " "Target-incompatible DataLayout attached\n"' failed.
    PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
    Stack dump:
    0.      Program arguments: /data/learn-compiler/codon-eg/../test1/llvm/build//bin/lli fib.ll
    1.      Running pass 'Function Pass Manager' on module 'fib.ll'.
    2.      Running pass 'X86 DAG->DAG Instruction Selection' on function '@main'
     #0 0x00005576193f82e0 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/data/learn-compiler/codon-eg/../test1/llvm/build//bin/lli+0x148b2e0)
     #1 0x00005576193f56ef llvm::sys::RunSignalHandlers() (/data/learn-compiler/codon-eg/../test1/llvm/build//bin/lli+0x14886ef)
     #2 0x00005576193f5845 SignalHandler(int) Signals.cpp:0:0
     #3 0x00007fe2573a2520 (/lib/x86_64-linux-gnu/libc.so.6+0x42520)
     #4 0x00007fe2573f69fc pthread_kill (/lib/x86_64-linux-gnu/libc.so.6+0x969fc)
     #5 0x00007fe2573a2476 gsignal (/lib/x86_64-linux-gnu/libc.so.6+0x42476)
     #6 0x00007fe2573887f3 abort (/lib/x86_64-linux-gnu/libc.so.6+0x287f3)
     #7 0x00007fe25738871b (/lib/x86_64-linux-gnu/libc.so.6+0x2871b)
     #8 0x00007fe257399e96 (/lib/x86_64-linux-gnu/libc.so.6+0x39e96)
     #9 0x000055761859ef2c llvm::MachineFunction::init() (/data/learn-compiler/codon-eg/../test1/llvm/build//bin/lli+0x631f2c)
    #10 0x00005576185c9776 llvm::MachineModuleInfo::getOrCreateMachineFunction(llvm::Function&) (/data/learn-compiler/codon-eg/../test1/llvm/build//bin/lli+0x65c776)
    #11 0x00005576185a0df1 llvm::MachineFunctionPass::runOnFunction(llvm::Function&) (.part.0) MachineFunctionPass.cpp:0:0
    #12 0x0000557618bb47ce llvm::FPPassManager::runOnFunction(llvm::Function&) (/data/learn-compiler/codon-eg/../test1/llvm/build//bin/lli+0xc477ce)
    #13 0x0000557618bb4a19 llvm::FPPassManager::runOnModule(llvm::Module&) (/data/learn-compiler/codon-eg/../test1/llvm/build//bin/lli+0xc47a19)
    #14 0x0000557618bb5365 llvm::legacy::PassManagerImpl::run(llvm::Module&) (/data/learn-compiler/codon-eg/../test1/llvm/build//bin/lli+0xc48365)
    #15 0x0000557618e215d6 llvm::orc::SimpleCompiler::operator()(llvm::Module&) (/data/learn-compiler/codon-eg/../test1/llvm/build//bin/lli+0xeb45d6)
    #16 0x0000557618e8c17d llvm::orc::IRCompileLayer::emit(std::unique_ptr<llvm::orc::MaterializationResponsibility, std::default_delete<llvm::orc::MaterializationResponsibility>>, llvm::orc::ThreadSafeModule) (/data/learn-compiler/codon-eg/../test1/llvm/build//bin/lli+0xf1f17d)
    #17 0x0000557618f2dfd9 llvm::orc::IRTransformLayer::emit(std::unique_ptr<llvm::orc::MaterializationResponsibility, std::default_delete<llvm::orc::MaterializationResponsibility>>, llvm::orc::ThreadSafeModule) (/data/learn-compiler/codon-eg/../test1/llvm/build//bin/lli+0xfc0fd9)
    #18 0x0000557618f2dfd9 llvm::orc::IRTransformLayer::emit(std::unique_ptr<llvm::orc::MaterializationResponsibility, std::default_delete<llvm::orc::MaterializationResponsibility>>, llvm::orc::ThreadSafeModule) (/data/learn-compiler/codon-eg/../test1/llvm/build//bin/lli+0xfc0fd9)
    #19 0x0000557618e991a6 llvm::orc::BasicIRLayerMaterializationUnit::materialize(std::unique_ptr<llvm::orc::MaterializationResponsibility, std::default_delete<llvm::orc::MaterializationResponsibility>>) (/data/learn-compiler/codon-eg/../test1/llvm/build//bin/lli+0xf2c1a6)
    #20 0x0000557618e4708c llvm::orc::MaterializationTask::run() (/data/learn-compiler/codon-eg/../test1/llvm/build//bin/lli+0xeda08c)
    #21 0x0000557618e2200c void llvm::detail::UniqueFunctionBase<void, std::unique_ptr<llvm::orc::Task, std::default_delete<llvm::orc::Task>>>::CallImpl<void (*)(std::unique_ptr<llvm::orc::Task, std::default_delete<llvm::orc::Task>>)>(void*, std::unique_ptr<llvm::orc::Task, std::default_delete<llvm::orc::Task>>&) (/data/learn-compiler/codon-eg/../test1/llvm/build//bin/lli+0xeb500c)
    #22 0x0000557618e25f53 llvm::orc::ExecutionSession::dispatchTask(std::unique_ptr<llvm::orc::Task, std::default_delete<llvm::orc::Task>>) (/data/learn-compiler/codon-eg/../test1/llvm/build//bin/lli+0xeb8f53)
    #23 0x0000557618e47d39 llvm::orc::ExecutionSession::dispatchOutstandingMUs() (/data/learn-compiler/codon-eg/../test1/llvm/build//bin/lli+0xedad39)
    #24 0x0000557618e502a4 llvm::orc::ExecutionSession::OL_completeLookup(std::unique_ptr<llvm::orc::InProgressLookupState, std::default_delete<llvm::orc::InProgressLookupState>>, std::shared_ptr<llvm::orc::AsynchronousSymbolQuery>, std::function<void (llvm::DenseMap<llvm::orc::JITDylib*, llvm::DenseSet<llvm::orc::SymbolStringPtr, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void>>, llvm::DenseMapInfo<llvm::orc::JITDylib*, void>, llvm::detail::DenseMapPair<llvm::orc::JITDylib*, llvm::DenseSet<llvm::orc::SymbolStringPtr, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void>>>> const&)>) (/data/learn-compiler/codon-eg/../test1/llvm/build//bin/lli+0xee32a4)
    #25 0x0000557618e51174 llvm::orc::InProgressFullLookupState::complete(std::unique_ptr<llvm::orc::InProgressLookupState, std::default_delete<llvm::orc::InProgressLookupState>>) (/data/learn-compiler/codon-eg/../test1/llvm/build//bin/lli+0xee4174)
    #26 0x0000557618e34ddf llvm::orc::ExecutionSession::OL_applyQueryPhase1(std::unique_ptr<llvm::orc::InProgressLookupState, std::default_delete<llvm::orc::InProgressLookupState>>, llvm::Error) (/data/learn-compiler/codon-eg/../test1/llvm/build//bin/lli+0xec7ddf)
    #27 0x0000557618e481e3 llvm::orc::ExecutionSession::lookup(llvm::orc::LookupKind, std::vector<std::pair<llvm::orc::JITDylib*, llvm::orc::JITDylibLookupFlags>, std::allocator<std::pair<llvm::orc::JITDylib*, llvm::orc::JITDylibLookupFlags>>> const&, llvm::orc::SymbolLookupSet, llvm::orc::SymbolState, llvm::unique_function<void (llvm::Expected<llvm::DenseMap<llvm::orc::SymbolStringPtr, llvm::orc::ExecutorSymbolDef, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void>, llvm::detail::DenseMapPair<llvm::orc::SymbolStringPtr, llvm::orc::ExecutorSymbolDef>>>)>, std::function<void (llvm::DenseMap<llvm::orc::JITDylib*, llvm::DenseSet<llvm::orc::SymbolStringPtr, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void>>, llvm::DenseMapInfo<llvm::orc::JITDylib*, void>, llvm::detail::DenseMapPair<llvm::orc::JITDylib*, llvm::DenseSet<llvm::orc::SymbolStringPtr, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void>>>> const&)>) (/data/learn-compiler/codon-eg/../test1/llvm/build//bin/lli+0xedb1e3)
    #28 0x0000557618e484ec llvm::orc::ExecutionSession::lookup(std::vector<std::pair<llvm::orc::JITDylib*, llvm::orc::JITDylibLookupFlags>, std::allocator<std::pair<llvm::orc::JITDylib*, llvm::orc::JITDylibLookupFlags>>> const&, llvm::orc::SymbolLookupSet, llvm::orc::LookupKind, llvm::orc::SymbolState, std::function<void (llvm::DenseMap<llvm::orc::JITDylib*, llvm::DenseSet<llvm::orc::SymbolStringPtr, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void>>, llvm::DenseMapInfo<llvm::orc::JITDylib*, void>, llvm::detail::DenseMapPair<llvm::orc::JITDylib*, llvm::DenseSet<llvm::orc::SymbolStringPtr, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void>>>> const&)>) (/data/learn-compiler/codon-eg/../test1/llvm/build//bin/lli+0xedb4ec)
    #29 0x0000557618e489ac llvm::orc::ExecutionSession::lookup(std::vector<std::pair<llvm::orc::JITDylib*, llvm::orc::JITDylibLookupFlags>, std::allocator<std::pair<llvm::orc::JITDylib*, llvm::orc::JITDylibLookupFlags>>> const&, llvm::orc::SymbolStringPtr, llvm::orc::SymbolState) (/data/learn-compiler/codon-eg/../test1/llvm/build//bin/lli+0xedb9ac)
    #30 0x0000557618ea29fa llvm::orc::LLJIT::lookupLinkerMangled(llvm::orc::JITDylib&, llvm::orc::SymbolStringPtr) (/data/learn-compiler/codon-eg/../test1/llvm/build//bin/lli+0xf359fa)
    #31 0x0000557618432e1c llvm::orc::LLJIT::lookup(llvm::orc::JITDylib&, llvm::StringRef) (/data/learn-compiler/codon-eg/../test1/llvm/build//bin/lli+0x4c5e1c)
    #32 0x00005576184404a5 runOrcJIT(char const*) (/data/learn-compiler/codon-eg/../test1/llvm/build//bin/lli+0x4d34a5)
    #33 0x00005576183ced2a main (/data/learn-compiler/codon-eg/../test1/llvm/build//bin/lli+0x461d2a)
    #34 0x00007fe257389d90 (/lib/x86_64-linux-gnu/libc.so.6+0x29d90)
    #35 0x00007fe257389e40 __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x29e40)
    #36 0x000055761842cc55 _start (/data/learn-compiler/codon-eg/../test1/llvm/build//bin/lli+0x4bfc55)
    ./codon-eg/demo.sh: line 11: 198835 Aborted                 (core dumped) ${LLVM_BUILD_DIR}/bin/lli fib.ll

Please help me to understand if I'm doing something wrong.

pramit-j2-sl avatar May 09 '24 10:05 pramit-j2-sl

Same here

codon build -release -llvm fib.py

lli fib.ll

JIT session error: Symbols not found: [ seq_print_full, seq_init, seq_str_int, seq_stdout, seq_personality ] lli: Failed to materialize symbols: { (main, { main, DW.ref.seq_personality }) }

Elfsong avatar Jun 22 '24 17:06 Elfsong

@arshajii Could you provide an example to run the fib.ll using lli? Thank you!

Elfsong avatar Jun 23 '24 04:06 Elfsong

@pramit-j2-sl Hi I took some time managed to figure out how to make it work:

  1. Assume we have a python snippet like that:
# File Name: foo.py
def foo():
    print("Hello World!")
foo()
  1. Convert the code to LLVM IR using Codon:
codon build -release -llvm foo.py

3. Compile Codon runtime (you need to install and setup CMake in advance):

# Clone Codon Repo
git clone https://github.com/exaloop/codon.git

# Create a build dir
mkdir build
cd build

# Compile the Codon runtime
cmake ..
make
  1. Execute the generated IR using ‘lli’ with Codon runtime libcodonrt.so):
lli -load ./codon/build/libcodonrt.so ./foo.ll

Elfsong avatar Jul 01 '24 08:07 Elfsong

Thank you for the solution!---and yes, all Codon code right now links to libcodonrt.

inumanag avatar Sep 23 '24 05:09 inumanag