Pup Buffer Test failure on ofi-smp

[nitbhat]

http://charm.cs.illinois.edu/autobuild/old.2020_11_04__01_07/ofi-linux-x86_64-smp.txt

../../../../bin/testrun  +p4 ./pup_buffer +balancer RotateLB +noCMAForZC +setcpuaffinity +CmiSleepOnIdle 

Running on 4 processors:  ./pup_buffer +balancer RotateLB +noCMAForZC +setcpuaffinity +CmiSleepOnIdle 
charmrun>  /usr/bin/setarch x86_64 -R  mpirun -np 4  ./pup_buffer +balancer RotateLB +noCMAForZC +setcpuaffinity +CmiSleepOnIdle 
Charm++>ofi> provider: psm2
Charm++>ofi> control progress: 2
Charm++>ofi> data progress: 2
Charm++>ofi> maximum inject message size: 64
Charm++>ofi> eager maximum message size: 65536 (maximum header size: 40)
Charm++>ofi> cq entries count: 8
Charm++>ofi> use inject: 1
Charm++>ofi> maximum rma size: 4294963200
Charm++>ofi> mr mode: 0x1
Charm++>ofi> use memory pool: 0
Charm++>ofi> use request cache: 0
Charm++>ofi> number of pre-allocated recvs: 8
Charm++>ofi> exchanging addresses over OFI
Charm++> Running in SMP mode: 4 processes, 1 worker threads (PEs) + 1 comm threads per process, 4 PEs total
Charm++> The comm. thread both sends and receives messages
Converse/Charm++ Commit ID: 590e80c
Charm++ built with internal error checking enabled.
Do not use for performance benchmarking (build without --enable-error-checking to do so).
Isomalloc> Synchronized global address space.
CharmLB> Load balancer assumes all CPUs are same.
Charm++> cpu affinity enabled. 
Charm++> Running on 1 hosts (2 sockets x 2 cores x 1 PUs = 4-way SMP)
Charm++> cpu topology info is gathered in 0.076 seconds.
[0] TreeLB in LEGACY MODE support
[0] TreeLB: Using PE_Root tree with: Rotate 
Iteration 1 completed
Iteration 2 completed
Iteration 3 completed
Iteration 4 completed
Iteration 5 completed
[5] Stack Traceback:
  [5:0] pup_buffer 0x5d7e8f CmiAbortHelper(char const*, char const*, char const*, int, int)
  [5:1] pup_buffer 0x5d7fad 
  [5:2] pup_buffer 0x5a6544 
  [5:3] pup_buffer 0x5d74bc 
  [5:4] pup_buffer 0x5ddec7 LrtsAdvanceCommunication(int)
  [5:5] pup_buffer 0x5e069e 
  [5:6] pup_buffer 0x5e0c95 
  [5:7] pup_buffer 0x5e1515 
  [5:8] libpthread.so.0 0x7fbc1e0bbdd5 
  [5:9] libc.so.6 0x7fbc1cf6a02d clone
------------- Processor 5 Exiting: Called CmiAbort ------------
Reason: zcPupGetCompleted: object not found


===================================================================================
=   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
=   RANK 0 PID 32195 RUNNING AT r003
=   KILLED BY SIGNAL: 9 (Killed)
===================================================================================

===================================================================================
=   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
=   RANK 2 PID 32197 RUNNING AT r003
=   KILLED BY SIGNAL: 9 (Killed)
===================================================================================

===================================================================================
=   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
=   RANK 3 PID 32198 RUNNING AT r003
=   KILLED BY SIGNAL: 9 (Killed)
===================================================================================

real	0m0.988s
user	0m0.027s
sys	0m0.029s
make[4]: *** [test] Error 255
make[4]: Leaving directory `/pylon5/ac7k4vp/nbhat4/autobuild/ofi-smp/charm/ofi-linux-x86_64-smp/tests/charm++/zerocopy/pup_buffer'
make[3]: *** [test-pup_buffer] Error 2
make[3]: Target `test' not remade because of errors.
make[3]: Leaving directory `/pylon5/ac7k4vp/nbhat4/autobuild/ofi-smp/charm/ofi-linux-x86_64-smp/tests/charm++/zerocopy'
make[2]: *** [test-zerocopy] Error 2