omnitrace
omnitrace copied to clipboard
Segfault instrumenting Cray MPI w/ v1.2.0
To reproduce on Crusher:
source sw/omnitrace-devel/share/omnitrace/setup-env.sh
module load craype-accel-amd-gfx90a
module load PrgEnv-cray
module load rocm
omnitrace -o $(basename /opt/cray/pe/lib64/libmpi_cray.so.12) -v 3 -- /opt/cray/pe/lib64/libmpi_cray.so.12
...
<output in attached log>
Looking at the core file shows:
(gdb) bt
#0 0x00007fffed4ef26f in Dyninst::Relocation::Instrumenter::handleCondDirExits(Dyninst::Relocation::RelocBlock*, Dyninst::Relocation::RelocGraph*, instPoint*) () from /autofs/nccs-svm1_home1/nicurtis/sw/omnitrace-devel/bin/../lib/omnitrace/libdyninstAPI.so.11.0
#1 0x00007fffed4f0015 in Dyninst::Relocation::Instrumenter::funcExitInstrumentation(Dyninst::Relocation::RelocBlock*, Dyninst::Relocation::RelocGraph*) () from /autofs/nccs-svm1_home1/nicurtis/sw/omnitrace-devel/bin/../lib/omnitrace/libdyninstAPI.so.11.0
#2 0x00007fffed4f020b in Dyninst::Relocation::Instrumenter::process(Dyninst::Relocation::RelocBlock*, Dyninst::Relocation::RelocGraph*) () from /autofs/nccs-svm1_home1/nicurtis/sw/omnitrace-devel/bin/../lib/omnitrace/libdyninstAPI.so.11.0
#3 0x00007fffed4ed280 in Dyninst::Relocation::Transformer::processGraph(Dyninst::Relocation::RelocGraph*) () from /autofs/nccs-svm1_home1/nicurtis/sw/omnitrace-devel/bin/../lib/omnitrace/libdyninstAPI.so.11.0
#4 0x00007fffed4d8c32 in Dyninst::Relocation::CodeMover::transform(Dyninst::Relocation::Transformer&) () from /autofs/nccs-svm1_home1/nicurtis/sw/omnitrace-devel/bin/../lib/omnitrace/libdyninstAPI.so.11.0
#5 0x00007fffed45cb4b in AddressSpace::transform(boost::shared_ptr<Dyninst::Relocation::CodeMover>) () from /autofs/nccs-svm1_home1/nicurtis/sw/omnitrace-devel/bin/../lib/omnitrace/libdyninstAPI.so.11.0
#6 0x00007fffed45dcf3 in AddressSpace::relocateInt(std::_Rb_tree_const_iterator<func_instance*>, std::_Rb_tree_const_iterator<func_instance*>, unsigned long) () from /autofs/nccs-svm1_home1/nicurtis/sw/omnitrace-devel/bin/../lib/omnitrace/libdyninstAPI.so.11.0
#7 0x00007fffed461fce in AddressSpace::relocate() () from /autofs/nccs-svm1_home1/nicurtis/sw/omnitrace-devel/bin/../lib/omnitrace/libdyninstAPI.so.11.0
#8 0x00007fffed506e1a in Dyninst::PatchAPI::DynInstrumenter::run() () from /autofs/nccs-svm1_home1/nicurtis/sw/omnitrace-devel/bin/../lib/omnitrace/libdyninstAPI.so.11.0
#9 0x00007fffed14f831 in Dyninst::PatchAPI::Patcher::run() () from /autofs/nccs-svm1_home1/nicurtis/sw/omnitrace-devel/bin/../lib/omnitrace/libpatchAPI.so.11.0
#10 0x00007fffed14f010 in Dyninst::PatchAPI::Command::commit() () from /autofs/nccs-svm1_home1/nicurtis/sw/omnitrace-devel/bin/../lib/omnitrace/libpatchAPI.so.11.0
#11 0x00007fffed45e97c in AddressSpace::patch(AddressSpace*) () from /autofs/nccs-svm1_home1/nicurtis/sw/omnitrace-devel/bin/../lib/omnitrace/libdyninstAPI.so.11.0
#12 0x00007fffed429c7e in BPatch_binaryEdit::writeFile(char const*) () from /autofs/nccs-svm1_home1/nicurtis/sw/omnitrace-devel/bin/../lib/omnitrace/libdyninstAPI.so.11.0
#13 0x000000000042670f in ?? ()
#14 0x00007fffe86ec2bd in __libc_start_main () from /lib64/libc.so.6
#15 0x00000000004299ea in ?? ()
https://gist.github.com/arghdos/1132b39c116cc7dbe46a89b71b831274
Yeah looks like another internal dyninst bug. I'll look into this when I'm working on #91
This should be fixed by the Dyninst updates in #164