GPUCompiler.jl GCN: fix_alloca_addrspace doesn't handle nested pointers

julia-debug: /home/tim/Julia/src/julia/deps/srccache/llvm-julia-13.0.1-0/llvm/lib/IR/Value.cpp:493: void llvm::Value::doRAUW(llvm::Value*, llvm::Value::ReplaceMetadataUses): Assertion `New->getType() == getType() && "replaceAllUses of value with new value of different type!"' failed.

#0  0x00007ffff7e1c34c in __pthread_kill_implementation () from /usr/lib/libc.so.6
#1  0x00007ffff7dcf4b8 in raise () from /usr/lib/libc.so.6
#2  0x00007ffff7db9534 in abort () from /usr/lib/libc.so.6
#3  0x00007ffff7db945c in __assert_fail_base.cold () from /usr/lib/libc.so.6
#4  0x00007ffff7dc8116 in __assert_fail () from /usr/lib/libc.so.6
#5  0x00007fffee7b2a09 in llvm::Value::doRAUW (this=0x5555577e2400, New=0x555556ae39d0, ReplaceMetaUses=llvm::Value::ReplaceMetadataUses::Yes)
    at /home/tim/Julia/src/julia/deps/srccache/llvm-julia-13.0.1-0/llvm/lib/IR/Value.cpp:493
#6  0x00007fffee7b2b62 in llvm::Value::replaceAllUsesWith (this=0x5555577e2400, New=0x555556ae39d0) at /home/tim/Julia/src/julia/deps/srccache/llvm-julia-13.0.1-0/llvm/lib/IR/Value.cpp:521
#7  0x00007fffef69d1e7 in llvm::BitcodeReaderValueList::assignValue (this=0x5555580eb728, V=0x555556ae39d0, Idx=67)
    at /home/tim/Julia/src/julia/deps/srccache/llvm-julia-13.0.1-0/llvm/lib/Bitcode/Reader/ValueList.cpp:88
#8  0x00007fffef6432ec in (anonymous namespace)::BitcodeReader::parseFunctionBody (this=0x5555580eb4e0, F=0x555555eab788)
    at /home/tim/Julia/src/julia/deps/srccache/llvm-julia-13.0.1-0/llvm/lib/Bitcode/Reader/BitcodeReader.cpp:5415
#9  0x00007fffef643b4d in (anonymous namespace)::BitcodeReader::materialize (this=0x5555580eb4e0, GV=0x555555eab788)
    at /home/tim/Julia/src/julia/deps/srccache/llvm-julia-13.0.1-0/llvm/lib/Bitcode/Reader/BitcodeReader.cpp:5499
#10 0x00007fffef64456a in (anonymous namespace)::BitcodeReader::materializeModule (this=0x5555580eb4e0)
    at /home/tim/Julia/src/julia/deps/srccache/llvm-julia-13.0.1-0/llvm/lib/Bitcode/Reader/BitcodeReader.cpp:5598
#11 0x00007fffee72a4cb in llvm::Module::materializeAll (this=0x555557547880) at /home/tim/Julia/src/julia/deps/srccache/llvm-julia-13.0.1-0/llvm/lib/IR/Module.cpp:453
#12 0x00007fffef64b90e in llvm::BitcodeModule::getModuleImpl(llvm::LLVMContext&, bool, bool, bool, llvm::function_ref<llvm::Optional<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > (llvm::StringRef)>) (this=0x7fffffff1de0, Context=..., MaterializeAll=true, ShouldLazyLoadMetadata=false, IsImporting=false, DataLayoutCallback=...)
    at /home/tim/Julia/src/julia/deps/srccache/llvm-julia-13.0.1-0/llvm/lib/Bitcode/Reader/BitcodeReader.cpp:6795
#13 0x00007fffef64cdeb in llvm::BitcodeModule::parseModule(llvm::LLVMContext&, llvm::function_ref<llvm::Optional<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > (llvm::StringRef)>) (this=0x7fffffff1de0, Context=..., DataLayoutCallback=...) at /home/tim/Julia/src/julia/deps/srccache/llvm-julia-13.0.1-0/llvm/lib/Bitcode/Reader/BitcodeReader.cpp:6977
#14 0x00007fffef64cebd in llvm::parseBitcodeFile(llvm::MemoryBufferRef, llvm::LLVMContext&, llvm::function_ref<llvm::Optional<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > (llvm::StringRef)>) (Buffer=..., Context=..., DataLayoutCallback=...) at /home/tim/Julia/src/julia/deps/srccache/llvm-julia-13.0.1-0/llvm/lib/Bitcode/Reader/BitcodeReader.cpp:6989
#15 0x00007fffef623b3b in LLVMParseBitcodeInContext2 (ContextRef=0x55555559c1d0, MemBuf=0x555556ae3880, OutModule=0x7fffffff1f10)
    at /home/tim/Julia/src/julia/deps/srccache/llvm-julia-13.0.1-0/llvm/lib/Bitcode/Reader/BitReader.cpp:65
#16 0x00007fff703385a2 in LLVMParseBitcodeInContext2 () at /home/tim/Julia/depot/packages/LLVM/tVv0H/lib/13/libLLVM_h.jl:5863
#17 julia_#parse#80_701 (ctx=..., membuf=...) at /home/tim/Julia/depot/packages/LLVM/tVv0H/src/bitcode.jl:6
#18 0x00007fff7035cbdc in parse##kw () at /home/tim/Julia/depot/packages/LLVM/tVv0H/src/bitcode.jl:4
#19 #parse#81 () at /home/tim/Julia/depot/packages/LLVM/tVv0H/src/bitcode.jl:12
#20 parse##kw () at /home/tim/Julia/depot/packages/LLVM/tVv0H/src/bitcode.jl:12
#21 #97 () at /home/tim/Julia/pkg/GPUCompiler/src/rtlib.jl:168
#22 julia_#open#377_1037 (kwargs=..., f=..., args...=...) at io.jl:384
#23 0x00007fff7035d352 in open () at io.jl:382
#24 julia_#95_1029 () at /home/tim/Julia/pkg/GPUCompiler/src/rtlib.jl:167
#25 0x00007fff7035ea3f in julia_lock_1026 (f=..., l=...) at lock.jl:185
#26 0x00007fff7035eb32 in jfptr_lock_1027 ()
#27 0x00007ffff736296a in _jl_invoke (F=0x7fffdcddcdc0 <jl_system_image_data+4803584>, args=0x7fffffff2e28, nargs=2, mfunc=0x7ffd8b6e83d0, world=32392) at /home/tim/Julia/src/julia/src/gf.c:2339
#28 0x00007ffff736340c in ijl_apply_generic (F=0x7fffdcddcdc0 <jl_system_image_data+4803584>, args=0x7fffffff2e28, nargs=2) at /home/tim/Julia/src/julia/src/gf.c:2540
#29 0x00007fff70335d7e in julia_#load_runtime#94_741 (ctx=..., job=0x0) at /home/tim/Julia/pkg/GPUCompiler/src/utils.jl:64
#30 0x00007fff7033c290 in load_runtime##kw () at /home/tim/Julia/pkg/GPUCompiler/src/utils.jl:62

#5  0x00007fffee7b2a09 in llvm::Value::doRAUW (this=0x5555577e2400, New=0x555556ae39d0, ReplaceMetaUses=llvm::Value::ReplaceMetadataUses::Yes)
    at /home/tim/Julia/src/julia/deps/srccache/llvm-julia-13.0.1-0/llvm/lib/IR/Value.cpp:493
493	  assert(New->getType() == getType() &&


(gdb) call getType()->dump()
{} addrspace(10)**
(gdb) call New->getType()->dump()
{} addrspace(10)* addrspace(5)*

(gdb) up

(gdb) call F->dump()
; Materializable
define i64 @ijl_unbox_uint64({} addrspace(10)* nonnull readonly %0) local_unnamed_addr #0 {
  %2 = alloca {} addrspace(10)*, align 8, addrspace(5)
}

Apr 08 '22 13:04 maleadt

Ahh, looks like the alloca rewriting pass needs to account for nested pointers. Thanks for debugging and reporting this @maleadt !

Apr 08 '22 13:04 jpsamaroo

Updated information: our rewrite pass is doing the right thing, since we only know that the outermost pointer type is wrong because of how alloca chooses its outermost pointer AS.

The real issue is that parsing bitcode (for the runtime module) causes a RAUW, and RAUW will replace alloca types based on the embedded DL. While we set the right DL before the module is serialized, we haven't yet fixed up the addrspaces, because that happens as part of optimization, which we skip for the runtime module.

Apr 14 '22 15:04 jpsamaroo

I guess we should then be consistent with the datalayout (whatever that means) when serializing the module, and changing it back afterwards if that's required? Or ensure the address spaces are fixed up before serializing the module?

Apr 17 '22 13:04 maleadt

Agreed, and I've got some changes locally to set the triple/DL to the native target for the runtime module, which generally are working (although this is all one big hack and I hate it).

Apr 18 '22 20:04 jpsamaroo