glog
glog copied to clipboard
google
Getting `(unknown)` in stacktrace?
Can someone take a quick look at the report here:
https://stackoverflow.com/q/53844443/1131467
Thanks, Andrew.
When I run it from fresh sources I still have the issue:
$ cat > t.cc
#include <glog/logging.h>
void h() { LOG(FATAL) << "test"; }
void g() { h(); }
void f() { g(); }
int main(int argc, char** argv) {
google::InitGoogleLogging(argv[0]);
f();
}
^D
$ ./a.out
F0102 23:54:15.603329 10358 t.cc:2] test
*** Check failure stack trace: ***
@ 0x55c930b28d2c google::LogMessage::Fail()
@ 0x55c930b28c74 google::LogMessage::SendToLog()
@ 0x55c930b285b5 google::LogMessage::Flush()
@ 0x55c930b2be66 google::LogMessageFatal::~LogMessageFatal()
@ 0x55c930b25820 google::thread_msg_data
@ 0x55c930b25835 google::thread_msg_data
@ 0x55c930b25841 google::thread_msg_data
@ 0x55c930b25867 google::thread_msg_data
@ 0x7ff9fbf2eb97 __libc_start_main
@ 0x55c930b256ea google::thread_msg_data
@ (nil) (unknown)
Aborted (core dumped)
@tomazos : could you check if the code works with glog version 0.3.4? If yes, then the breaking commit is a93a4511ec5f21b44a1b21b4310c7a9c30c213a5 and the failing condition is in the line 359: https://github.com/google/glog/blob/a6a166db069520dbbd653c97c2e5b12e08a8bb26/src/symbolize.cc#L359.
So here is what happens in src/symbolize.cc after a93a4511ec5f21b44a1b21b4310c7a9c30c213a5:
Each line in the stack trace is being prepared in the function SymbolizeAndDemangle().
It calculates variables start_address / base_address and calls the function GetSymbolFromObjectFile(), which tries to look up symbol corresponding to a given frame instruction pointer (pc).
In v0.3.4 the call to GetSymbolFromObjectFile() passed value of start_address, a start address of a memory region, which can never be zero. GetSymbolFromObjectFile() did not check the value of start_address and just used it for lookups in two symbol tables.
In v0.3.5 the call to GetSymbolFromObjectFile() started passing value of base_address. That value can become zero if the frame program counter was found within the very first memory region and the executable image has been mapped with zero offset. That seems to be a common case.
Another change in GetSymbolFromObjectFile() added a code for symbol offset adjustment and a new check: if symbol_offset equals zero after all adjustments then the function returns a failure value meaning "symbol not found". Since the symbol_offset variable was then calculated from base_address, that condition became quite possible.
@sergiud : it is actually not a question, but rather a bug in v0.3.5 when using under Ubuntu 18.04. See Damian's comment, the stack trace does not find symbols defined in the executable itself. Can maybe @pcc help with triaging?
Can you show the contents of /proc/self/maps while your program is running and the output of readelf -aW a.out?
Here you are, I started and paused a.out under gdb.
If you'd like to reproduce then it was on AWS Ubuntu Server 18.04, AMI ID ami-00035f41c82244dab. The source code and build instructions are the same as at issue author's URL.
ubuntu@ec2-ubuntu18:~$ cat /proc/8003/maps
555555554000-555555555000 r-xp 00000000 ca:01 264709 /home/ubuntu/a.out
555555754000-555555755000 r--p 00000000 ca:01 264709 /home/ubuntu/a.out
555555755000-555555756000 rw-p 00001000 ca:01 264709 /home/ubuntu/a.out
555555756000-555555777000 rw-p 00000000 00:00 0 [heap]
7ffff63eb000-7ffff63ee000 r-xp 00000000 ca:01 2082 /lib/x86_64-linux-gnu/libdl-2.27.so
7ffff63ee000-7ffff65ed000 ---p 00003000 ca:01 2082 /lib/x86_64-linux-gnu/libdl-2.27.so
7ffff65ed000-7ffff65ee000 r--p 00002000 ca:01 2082 /lib/x86_64-linux-gnu/libdl-2.27.so
7ffff65ee000-7ffff65ef000 rw-p 00003000 ca:01 2082 /lib/x86_64-linux-gnu/libdl-2.27.so
7ffff65ef000-7ffff6613000 r-xp 00000000 ca:01 2094 /lib/x86_64-linux-gnu/liblzma.so.5.2.2
7ffff6613000-7ffff6813000 ---p 00024000 ca:01 2094 /lib/x86_64-linux-gnu/liblzma.so.5.2.2
7ffff6813000-7ffff6814000 r--p 00024000 ca:01 2094 /lib/x86_64-linux-gnu/liblzma.so.5.2.2
7ffff6814000-7ffff6815000 rw-p 00025000 ca:01 2094 /lib/x86_64-linux-gnu/liblzma.so.5.2.2
7ffff6815000-7ffff682c000 r-xp 00000000 ca:01 25722 /lib/x86_64-linux-gnu/libgcc_s.so.1
7ffff682c000-7ffff6a2b000 ---p 00017000 ca:01 25722 /lib/x86_64-linux-gnu/libgcc_s.so.1
7ffff6a2b000-7ffff6a2c000 r--p 00016000 ca:01 25722 /lib/x86_64-linux-gnu/libgcc_s.so.1
7ffff6a2c000-7ffff6a2d000 rw-p 00017000 ca:01 25722 /lib/x86_64-linux-gnu/libgcc_s.so.1
7ffff6a2d000-7ffff6bca000 r-xp 00000000 ca:01 2083 /lib/x86_64-linux-gnu/libm-2.27.so
7ffff6bca000-7ffff6dc9000 ---p 0019d000 ca:01 2083 /lib/x86_64-linux-gnu/libm-2.27.so
7ffff6dc9000-7ffff6dca000 r--p 0019c000 ca:01 2083 /lib/x86_64-linux-gnu/libm-2.27.so
7ffff6dca000-7ffff6dcb000 rw-p 0019d000 ca:01 2083 /lib/x86_64-linux-gnu/libm-2.27.so
7ffff6dcb000-7ffff6de5000 r-xp 00000000 ca:01 2180 /lib/x86_64-linux-gnu/libpthread-2.27.so
7ffff6de5000-7ffff6fe4000 ---p 0001a000 ca:01 2180 /lib/x86_64-linux-gnu/libpthread-2.27.so
7ffff6fe4000-7ffff6fe5000 r--p 00019000 ca:01 2180 /lib/x86_64-linux-gnu/libpthread-2.27.so
7ffff6fe5000-7ffff6fe6000 rw-p 0001a000 ca:01 2180 /lib/x86_64-linux-gnu/libpthread-2.27.so
7ffff6fe6000-7ffff6fea000 rw-p 00000000 00:00 0
7ffff6fea000-7ffff6ff6000 r-xp 00000000 ca:01 6318 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1
7ffff6ff6000-7ffff71f5000 ---p 0000c000 ca:01 6318 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1
7ffff71f5000-7ffff71f6000 r--p 0000b000 ca:01 6318 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1
7ffff71f6000-7ffff71f7000 rw-p 0000c000 ca:01 6318 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1
7ffff71f7000-7ffff7205000 rw-p 00000000 00:00 0
7ffff7205000-7ffff7229000 r-xp 00000000 ca:01 55002 /usr/lib/x86_64-linux-gnu/libgflags.so.2.2.1
7ffff7229000-7ffff7428000 ---p 00024000 ca:01 55002 /usr/lib/x86_64-linux-gnu/libgflags.so.2.2.1
7ffff7428000-7ffff7429000 r--p 00023000 ca:01 55002 /usr/lib/x86_64-linux-gnu/libgflags.so.2.2.1
7ffff7429000-7ffff742a000 rw-p 00024000 ca:01 55002 /usr/lib/x86_64-linux-gnu/libgflags.so.2.2.1
7ffff742a000-7ffff7611000 r-xp 00000000 ca:01 2079 /lib/x86_64-linux-gnu/libc-2.27.so
7ffff7611000-7ffff7811000 ---p 001e7000 ca:01 2079 /lib/x86_64-linux-gnu/libc-2.27.so
7ffff7811000-7ffff7815000 r--p 001e7000 ca:01 2079 /lib/x86_64-linux-gnu/libc-2.27.so
7ffff7815000-7ffff7817000 rw-p 001eb000 ca:01 2079 /lib/x86_64-linux-gnu/libc-2.27.so
7ffff7817000-7ffff781b000 rw-p 00000000 00:00 0
7ffff781b000-7ffff7994000 r-xp 00000000 ca:01 55065 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25
7ffff7994000-7ffff7b94000 ---p 00179000 ca:01 55065 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25
7ffff7b94000-7ffff7b9e000 r--p 00179000 ca:01 55065 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25
7ffff7b9e000-7ffff7ba0000 rw-p 00183000 ca:01 55065 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25
7ffff7ba0000-7ffff7ba4000 rw-p 00000000 00:00 0
7ffff7ba4000-7ffff7bc3000 r-xp 00000000 ca:01 28141 /usr/lib/x86_64-linux-gnu/libglog.so.0.0.0
7ffff7bc3000-7ffff7dc3000 ---p 0001f000 ca:01 28141 /usr/lib/x86_64-linux-gnu/libglog.so.0.0.0
7ffff7dc3000-7ffff7dc4000 r--p 0001f000 ca:01 28141 /usr/lib/x86_64-linux-gnu/libglog.so.0.0.0
7ffff7dc4000-7ffff7dc5000 rw-p 00020000 ca:01 28141 /usr/lib/x86_64-linux-gnu/libglog.so.0.0.0
7ffff7dc5000-7ffff7dd5000 rw-p 00000000 00:00 0
7ffff7dd5000-7ffff7dfc000 r-xp 00000000 ca:01 2075 /lib/x86_64-linux-gnu/ld-2.27.so
7ffff7fe7000-7ffff7ff0000 rw-p 00000000 00:00 0
7ffff7ff7000-7ffff7ffa000 r--p 00000000 00:00 0 [vvar]
7ffff7ffa000-7ffff7ffc000 r-xp 00000000 00:00 0 [vdso]
7ffff7ffc000-7ffff7ffd000 r--p 00027000 ca:01 2075 /lib/x86_64-linux-gnu/ld-2.27.so
7ffff7ffd000-7ffff7ffe000 rw-p 00028000 ca:01 2075 /lib/x86_64-linux-gnu/ld-2.27.so
7ffff7ffe000-7ffff7fff000 rw-p 00000000 00:00 0
7ffffffde000-7ffffffff000 rw-p 00000000 00:00 0 [stack]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
ubuntu@ec2-ubuntu18:~$ readelf -aW a.out
ELF Header:
Magic: 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00
Class: ELF64
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: UNIX - System V
ABI Version: 0
Type: DYN (Shared object file)
Machine: Advanced Micro Devices X86-64
Version: 0x1
Entry point address: 0x860
Start of program headers: 64 (bytes into file)
Start of section headers: 7440 (bytes into file)
Flags: 0x0
Size of this header: 64 (bytes)
Size of program headers: 56 (bytes)
Number of program headers: 9
Size of section headers: 64 (bytes)
Number of section headers: 30
Section header string table index: 29
Section Headers:
[Nr] Name Type Address Off Size ES Flg Lk Inf Al
[ 0] NULL 0000000000000000 000000 000000 00 0 0 0
[ 1] .interp PROGBITS 0000000000000238 000238 00001c 00 A 0 0 1
[ 2] .note.ABI-tag NOTE 0000000000000254 000254 000020 00 A 0 0 4
[ 3] .note.gnu.build-id NOTE 0000000000000274 000274 000024 00 A 0 0 4
[ 4] .gnu.hash GNU_HASH 0000000000000298 000298 000038 00 A 5 0 8
[ 5] .dynsym DYNSYM 00000000000002d0 0002d0 000198 18 A 6 1 8
[ 6] .dynstr STRTAB 0000000000000468 000468 0001a6 00 A 0 0 1
[ 7] .gnu.version VERSYM 000000000000060e 00060e 000022 02 A 5 0 2
[ 8] .gnu.version_r VERNEED 0000000000000630 000630 000050 00 A 6 2 8
[ 9] .rela.dyn RELA 0000000000000680 000680 0000d8 18 A 5 0 8
[10] .rela.plt RELA 0000000000000758 000758 000078 18 AI 5 23 8
[11] .init PROGBITS 00000000000007d0 0007d0 000017 00 AX 0 0 4
[12] .plt PROGBITS 00000000000007f0 0007f0 000060 10 AX 0 0 16
[13] .plt.got PROGBITS 0000000000000850 000850 000008 08 AX 0 0 8
[14] .text PROGBITS 0000000000000860 000860 000222 00 AX 0 0 16
[15] .fini PROGBITS 0000000000000a84 000a84 000009 00 AX 0 0 4
[16] .rodata PROGBITS 0000000000000a90 000a90 000036 00 A 0 0 4
[17] .eh_frame_hdr PROGBITS 0000000000000ac8 000ac8 000054 00 A 0 0 4
[18] .eh_frame PROGBITS 0000000000000b20 000b20 000188 00 A 0 0 8
[19] .gcc_except_table PROGBITS 0000000000000ca8 000ca8 00000c 00 A 0 0 1
[20] .init_array INIT_ARRAY 0000000000200d78 000d78 000008 08 WA 0 0 8
[21] .fini_array FINI_ARRAY 0000000000200d80 000d80 000008 08 WA 0 0 8
[22] .dynamic DYNAMIC 0000000000200d88 000d88 000210 10 WA 6 0 8
[23] .got PROGBITS 0000000000200f98 000f98 000068 08 WA 0 0 8
[24] .data PROGBITS 0000000000201000 001000 000018 00 WA 0 0 8
[25] .bss NOBITS 0000000000201018 001018 000008 00 WA 0 0 1
[26] .comment PROGBITS 0000000000000000 001018 00002a 01 MS 0 0 1
[27] .symtab SYMTAB 0000000000000000 001048 0007c8 18 28 54 8
[28] .strtab STRTAB 0000000000000000 001810 0003f0 00 0 0 1
[29] .shstrtab STRTAB 0000000000000000 001c00 000110 00 0 0 1
Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
L (link order), O (extra OS processing required), G (group), T (TLS),
C (compressed), x (unknown), o (OS specific), E (exclude),
l (large), p (processor specific)
There are no section groups in this file.
Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
PHDR 0x000040 0x0000000000000040 0x0000000000000040 0x0001f8 0x0001f8 R 0x8
INTERP 0x000238 0x0000000000000238 0x0000000000000238 0x00001c 0x00001c R 0x1
[Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
LOAD 0x000000 0x0000000000000000 0x0000000000000000 0x000cb4 0x000cb4 R E 0x200000
LOAD 0x000d78 0x0000000000200d78 0x0000000000200d78 0x0002a0 0x0002a8 RW 0x200000
DYNAMIC 0x000d88 0x0000000000200d88 0x0000000000200d88 0x000210 0x000210 RW 0x8
NOTE 0x000254 0x0000000000000254 0x0000000000000254 0x000044 0x000044 R 0x4
GNU_EH_FRAME 0x000ac8 0x0000000000000ac8 0x0000000000000ac8 0x000054 0x000054 R 0x4
GNU_STACK 0x000000 0x0000000000000000 0x0000000000000000 0x000000 0x000000 RW 0x10
GNU_RELRO 0x000d78 0x0000000000200d78 0x0000000000200d78 0x000288 0x000288 R 0x1
Section to Segment mapping:
Segment Sections...
00
01 .interp
02 .interp .note.ABI-tag .note.gnu.build-id .gnu.hash .dynsym .dynstr .gnu.version .gnu.version_r .rela.dyn .rela.plt .init .plt .plt.got .text .fini .rodata .eh_frame_hdr .eh_frame .gcc_except_table
03 .init_array .fini_array .dynamic .got .data .bss
04 .dynamic
05 .note.ABI-tag .note.gnu.build-id
06 .eh_frame_hdr
07
08 .init_array .fini_array .dynamic .got
Dynamic section at offset 0xd88 contains 29 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libglog.so.0]
0x0000000000000001 (NEEDED) Shared library: [libstdc++.so.6]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x000000000000000c (INIT) 0x7d0
0x000000000000000d (FINI) 0xa84
0x0000000000000019 (INIT_ARRAY) 0x200d78
0x000000000000001b (INIT_ARRAYSZ) 8 (bytes)
0x000000000000001a (FINI_ARRAY) 0x200d80
0x000000000000001c (FINI_ARRAYSZ) 8 (bytes)
0x000000006ffffef5 (GNU_HASH) 0x298
0x0000000000000005 (STRTAB) 0x468
0x0000000000000006 (SYMTAB) 0x2d0
0x000000000000000a (STRSZ) 422 (bytes)
0x000000000000000b (SYMENT) 24 (bytes)
0x0000000000000015 (DEBUG) 0x0
0x0000000000000003 (PLTGOT) 0x200f98
0x0000000000000002 (PLTRELSZ) 120 (bytes)
0x0000000000000014 (PLTREL) RELA
0x0000000000000017 (JMPREL) 0x758
0x0000000000000007 (RELA) 0x680
0x0000000000000008 (RELASZ) 216 (bytes)
0x0000000000000009 (RELAENT) 24 (bytes)
0x000000000000001e (FLAGS) BIND_NOW
0x000000006ffffffb (FLAGS_1) Flags: NOW PIE
0x000000006ffffffe (VERNEED) 0x630
0x000000006fffffff (VERNEEDNUM) 2
0x000000006ffffff0 (VERSYM) 0x60e
0x000000006ffffff9 (RELACOUNT) 3
0x0000000000000000 (NULL) 0x0
Relocation section '.rela.dyn' at offset 0x680 contains 9 entries:
Offset Info Type Symbol's Value Symbol's Name + Addend
0000000000200d78 0000000000000008 R_X86_64_RELATIVE 960
0000000000200d80 0000000000000008 R_X86_64_RELATIVE 920
0000000000201008 0000000000000008 R_X86_64_RELATIVE 201008
0000000000200fd8 0000000100000006 R_X86_64_GLOB_DAT 0000000000000000 __cxa_finalize@GLIBC_2.2.5 + 0
0000000000200fe0 0000000700000006 R_X86_64_GLOB_DAT 0000000000000000 _ITM_deregisterTMCloneTable + 0
0000000000200fe8 0000000800000006 R_X86_64_GLOB_DAT 0000000000000000 __libc_start_main@GLIBC_2.2.5 + 0
0000000000200ff0 0000000900000006 R_X86_64_GLOB_DAT 0000000000000000 __gmon_start__ + 0
0000000000200ff8 0000000a00000006 R_X86_64_GLOB_DAT 0000000000000000 _ITM_registerTMCloneTable + 0
0000000000201010 0000000500000001 R_X86_64_64 0000000000000000 __gxx_personality_v0@CXXABI_1.3 + 0
Relocation section '.rela.plt' at offset 0x758 contains 5 entries:
Offset Info Type Symbol's Value Symbol's Name + Addend
0000000000200fb0 0000000200000007 R_X86_64_JUMP_SLOT 0000000000000000 _ZStlsISt11char_traitsIcEERSt13basic_ostreamIcT_ES5_PKc@GLIBCXX_3.4 + 0
0000000000200fb8 0000000300000007 R_X86_64_JUMP_SLOT 0000000000000000 _ZN6google10LogMessage6streamEv + 0
0000000000200fc0 0000000400000007 R_X86_64_JUMP_SLOT 0000000000000000 _ZN6google17InitGoogleLoggingEPKc + 0
0000000000200fc8 0000000600000007 R_X86_64_JUMP_SLOT 0000000000000000 _ZN6google15LogMessageFatalD1Ev + 0
0000000000200fd0 0000000b00000007 R_X86_64_JUMP_SLOT 0000000000000000 _ZN6google15LogMessageFatalC1EPKci + 0
The decoding of unwind sections for machine type Advanced Micro Devices X86-64 is not currently supported.
Symbol table '.dynsym' contains 17 entries:
Num: Value Size Type Bind Vis Ndx Name
0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
1: 0000000000000000 0 FUNC WEAK DEFAULT UND __cxa_finalize@GLIBC_2.2.5 (2)
2: 0000000000000000 0 FUNC GLOBAL DEFAULT UND _ZStlsISt11char_traitsIcEERSt13basic_ostreamIcT_ES5_PKc@GLIBCXX_3.4 (3)
3: 0000000000000000 0 FUNC GLOBAL DEFAULT UND _ZN6google10LogMessage6streamEv
4: 0000000000000000 0 FUNC GLOBAL DEFAULT UND _ZN6google17InitGoogleLoggingEPKc
5: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __gxx_personality_v0@CXXABI_1.3 (4)
6: 0000000000000000 0 FUNC GLOBAL DEFAULT UND _ZN6google15LogMessageFatalD1Ev
7: 0000000000000000 0 NOTYPE WEAK DEFAULT UND _ITM_deregisterTMCloneTable
8: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __libc_start_main@GLIBC_2.2.5 (2)
9: 0000000000000000 0 NOTYPE WEAK DEFAULT UND __gmon_start__
10: 0000000000000000 0 NOTYPE WEAK DEFAULT UND _ITM_registerTMCloneTable
11: 0000000000000000 0 FUNC GLOBAL DEFAULT UND _ZN6google15LogMessageFatalC1EPKci
12: 0000000000201018 0 NOTYPE GLOBAL DEFAULT 24 _edata
13: 0000000000201020 0 NOTYPE GLOBAL DEFAULT 25 _end
14: 00000000000007d0 0 FUNC GLOBAL DEFAULT 11 _init
15: 0000000000201018 0 NOTYPE GLOBAL DEFAULT 25 __bss_start
16: 0000000000000a84 0 FUNC GLOBAL DEFAULT 15 _fini
Symbol table '.symtab' contains 83 entries:
Num: Value Size Type Bind Vis Ndx Name
0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
1: 0000000000000238 0 SECTION LOCAL DEFAULT 1
2: 0000000000000254 0 SECTION LOCAL DEFAULT 2
3: 0000000000000274 0 SECTION LOCAL DEFAULT 3
4: 0000000000000298 0 SECTION LOCAL DEFAULT 4
5: 00000000000002d0 0 SECTION LOCAL DEFAULT 5
6: 0000000000000468 0 SECTION LOCAL DEFAULT 6
7: 000000000000060e 0 SECTION LOCAL DEFAULT 7
8: 0000000000000630 0 SECTION LOCAL DEFAULT 8
9: 0000000000000680 0 SECTION LOCAL DEFAULT 9
10: 0000000000000758 0 SECTION LOCAL DEFAULT 10
11: 00000000000007d0 0 SECTION LOCAL DEFAULT 11
12: 00000000000007f0 0 SECTION LOCAL DEFAULT 12
13: 0000000000000850 0 SECTION LOCAL DEFAULT 13
14: 0000000000000860 0 SECTION LOCAL DEFAULT 14
15: 0000000000000a84 0 SECTION LOCAL DEFAULT 15
16: 0000000000000a90 0 SECTION LOCAL DEFAULT 16
17: 0000000000000ac8 0 SECTION LOCAL DEFAULT 17
18: 0000000000000b20 0 SECTION LOCAL DEFAULT 18
19: 0000000000000ca8 0 SECTION LOCAL DEFAULT 19
20: 0000000000200d78 0 SECTION LOCAL DEFAULT 20
21: 0000000000200d80 0 SECTION LOCAL DEFAULT 21
22: 0000000000200d88 0 SECTION LOCAL DEFAULT 22
23: 0000000000200f98 0 SECTION LOCAL DEFAULT 23
24: 0000000000201000 0 SECTION LOCAL DEFAULT 24
25: 0000000000201018 0 SECTION LOCAL DEFAULT 25
26: 0000000000000000 0 SECTION LOCAL DEFAULT 26
27: 0000000000000000 0 FILE LOCAL DEFAULT ABS crtstuff.c
28: 0000000000000890 0 FUNC LOCAL DEFAULT 14 deregister_tm_clones
29: 00000000000008d0 0 FUNC LOCAL DEFAULT 14 register_tm_clones
30: 0000000000000920 0 FUNC LOCAL DEFAULT 14 __do_global_dtors_aux
31: 0000000000201018 1 OBJECT LOCAL DEFAULT 25 completed.7696
32: 0000000000200d80 0 OBJECT LOCAL DEFAULT 21 __do_global_dtors_aux_fini_array_entry
33: 0000000000000960 0 FUNC LOCAL DEFAULT 14 frame_dummy
34: 0000000000200d78 0 OBJECT LOCAL DEFAULT 20 __frame_dummy_init_array_entry
35: 0000000000000000 0 FILE LOCAL DEFAULT ABS t.cc
36: 0000000000000a94 1 OBJECT LOCAL DEFAULT 16 _ZStL19piecewise_construct
37: 0000000000000a98 4 OBJECT LOCAL DEFAULT 16 _ZN6googleL9GLOG_INFOE
38: 0000000000000a9c 4 OBJECT LOCAL DEFAULT 16 _ZN6googleL12GLOG_WARNINGE
39: 0000000000000aa0 4 OBJECT LOCAL DEFAULT 16 _ZN6googleL10GLOG_ERRORE
40: 0000000000000aa4 4 OBJECT LOCAL DEFAULT 16 _ZN6googleL10GLOG_FATALE
41: 0000000000000aa8 4 OBJECT LOCAL DEFAULT 16 _ZN6googleL14NUM_SEVERITIESE
42: 0000000000000aac 4 OBJECT LOCAL DEFAULT 16 _ZN6googleL4INFOE
43: 0000000000000ab0 4 OBJECT LOCAL DEFAULT 16 _ZN6googleL7WARNINGE
44: 0000000000000ab4 4 OBJECT LOCAL DEFAULT 16 _ZN6googleL5ERRORE
45: 0000000000000ab8 4 OBJECT LOCAL DEFAULT 16 _ZN6googleL5FATALE
46: 0000000000000000 0 FILE LOCAL DEFAULT ABS crtstuff.c
47: 0000000000000ca4 0 OBJECT LOCAL DEFAULT 18 __FRAME_END__
48: 0000000000000000 0 FILE LOCAL DEFAULT ABS
49: 0000000000000ac8 0 NOTYPE LOCAL DEFAULT 17 __GNU_EH_FRAME_HDR
50: 0000000000200d88 0 OBJECT LOCAL DEFAULT 22 _DYNAMIC
51: 0000000000200d80 0 NOTYPE LOCAL DEFAULT 20 __init_array_end
52: 0000000000200d78 0 NOTYPE LOCAL DEFAULT 20 __init_array_start
53: 0000000000200f98 0 OBJECT LOCAL DEFAULT 23 _GLOBAL_OFFSET_TABLE_
54: 0000000000201018 0 NOTYPE GLOBAL DEFAULT 24 _edata
55: 0000000000201000 0 NOTYPE WEAK DEFAULT 24 data_start
56: 0000000000000a90 4 OBJECT GLOBAL DEFAULT 16 _IO_stdin_used
57: 0000000000000000 0 FUNC WEAK DEFAULT UND __cxa_finalize@@GLIBC_2.2.5
58: 00000000000009e4 42 FUNC GLOBAL DEFAULT 14 main
59: 000000000000096a 98 FUNC GLOBAL DEFAULT 14 _Z1hv
60: 0000000000201008 0 OBJECT GLOBAL HIDDEN 24 __dso_handle
61: 0000000000201010 8 OBJECT WEAK HIDDEN 24 DW.ref.__gxx_personality_v0
62: 0000000000000a84 0 FUNC GLOBAL DEFAULT 15 _fini
63: 0000000000000860 43 FUNC GLOBAL DEFAULT 14 _start
64: 0000000000000000 0 FUNC GLOBAL DEFAULT UND _ZStlsISt11char_traitsIcEERSt13basic_ostreamIcT_ES5_PKc@@GLIBCXX_3.4
65: 00000000000009d8 12 FUNC GLOBAL DEFAULT 14 _Z1fv
66: 00000000000007d0 0 FUNC GLOBAL DEFAULT 11 _init
67: 0000000000201018 0 OBJECT GLOBAL HIDDEN 24 __TMC_END__
68: 0000000000000000 0 FUNC GLOBAL DEFAULT UND _ZN6google10LogMessage6streamEv
69: 0000000000000000 0 FUNC GLOBAL DEFAULT UND _ZN6google17InitGoogleLoggingEPKc
70: 0000000000201000 0 NOTYPE GLOBAL DEFAULT 24 __data_start
71: 0000000000201020 0 NOTYPE GLOBAL DEFAULT 25 _end
72: 0000000000201018 0 NOTYPE GLOBAL DEFAULT 25 __bss_start
73: 0000000000000a10 101 FUNC GLOBAL DEFAULT 14 __libc_csu_init
74: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __gxx_personality_v0@@CXXABI_1.3
75: 0000000000000000 0 FUNC GLOBAL DEFAULT UND _ZN6google15LogMessageFatalD1Ev
76: 0000000000000000 0 NOTYPE WEAK DEFAULT UND _ITM_deregisterTMCloneTable
77: 00000000000009cc 12 FUNC GLOBAL DEFAULT 14 _Z1gv
78: 0000000000000a80 2 FUNC GLOBAL DEFAULT 14 __libc_csu_fini
79: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __libc_start_main@@GLIBC_2.2.5
80: 0000000000000000 0 NOTYPE WEAK DEFAULT UND __gmon_start__
81: 0000000000000000 0 NOTYPE WEAK DEFAULT UND _ITM_registerTMCloneTable
82: 0000000000000000 0 FUNC GLOBAL DEFAULT UND _ZN6google15LogMessageFatalC1EPKci
Histogram for `.gnu.hash' bucket list length (total of 3 buckets):
Length Number % of total Coverage
0 0 ( 0.0%)
1 1 ( 33.3%) 20.0%
2 2 ( 66.7%) 100.0%
Version symbols section '.gnu.version' contains 17 entries:
Addr: 000000000000060e Offset: 0x00060e Link: 5 (.dynsym)
000: 0 (*local*) 2 (GLIBC_2.2.5) 3 (GLIBCXX_3.4) 0 (*local*)
004: 0 (*local*) 4 (CXXABI_1.3) 0 (*local*) 0 (*local*)
008: 2 (GLIBC_2.2.5) 0 (*local*) 0 (*local*) 0 (*local*)
00c: 1 (*global*) 1 (*global*) 1 (*global*) 1 (*global*)
010: 1 (*global*)
Version needs section '.gnu.version_r' contains 2 entries:
Addr: 0x0000000000000630 Offset: 0x000630 Link: 6 (.dynstr)
000000: Version: 1 File: libstdc++.so.6 Cnt: 2
0x0010: Name: CXXABI_1.3 Flags: none Version: 4
0x0020: Name: GLIBCXX_3.4 Flags: none Version: 3
0x0030: Version: 1 File: libc.so.6 Cnt: 1
0x0040: Name: GLIBC_2.2.5 Flags: none Version: 2
Displaying notes found in: .note.ABI-tag
Owner Data size Description
GNU 0x00000010 NT_GNU_ABI_TAG (ABI version tag) OS: Linux, ABI: 3.2.0
Displaying notes found in: .note.gnu.build-id
Owner Data size Description
GNU 0x00000014 NT_GNU_BUILD_ID (unique build ID bitstring) Build ID: df8a82fc90c6db628fea16cb987f8e8d7ce8e7b7
ubuntu@ec2-ubuntu18:~$ uname -a
Linux ec2-ubuntu18 4.15.0-1021-aws #21-Ubuntu SMP Tue Aug 28 10:23:07 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
ubuntu@ec2-ubuntu18:~$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.1 LTS"
@pcc: what was the reference system on which the code works as designed?
TL;DR: this appears to be a bug that was already fixed by commit c4d37a78cc8aeeb68ae09d37152114fac24c0cea.
I was able to reproduce the problem in an Ubuntu 18.04 VM, but only with Ubuntu's libgoogle-glog-dev package which is version 0.3.5. If I build and install glog from trunk, I cannot reproduce (either on current Arch Linux or Ubuntu). If I revert the commit linked above and rebuild, I can reproduce again.
But @damian123 mentioned that they were able to reproduce with "fresh sources". @damian123 can you confirm that you were indeed using a glog with the commit included? If so can you please provide more details on your setup?
@ukai is it worth considering making another release so that distro package users don't run into this issue?
Ah, I can reproduce @damian123's output with trunk, but only if I compile the library with cmake (the clue was the presence of the symbol google::thread_msg_data, which doesn't appear at all in the autotools build, but does by default in the cmake build).
This seems to be a separate issue unrelated to my changes (if I revert both c4d37a7 and a93a451 on trunk I see the same output); you should be able to work around it either by using the autotools build or by passing -DWITH_TLS=OFF to cmake.
This can still be reproduced in v0.5.0-rc1 with cmake. The workaround of using -DWITH_TLS=OFF mentioned in https://github.com/google/glog/issues/409#issuecomment-455836857 works. Is that going to be the long-term workaround now that this bug has been closed?
Since we dropped autoconf support, let me reopen the issue.
It seems, the use of TLS exposed a problem in the symbol resolution (related to #630). However, the problem is triggered only when compiling a static library. The suggested workaround implemented in current master (after rc1) is to build a shared library instead.
I'm closing the issue because as of aaefca72b30608acf2ba78b41890fff487a2631b and GCC 13.2.1 I can no longer observe the problem even if TLS is enabled:
F20240108 01:21:25.716890 140642250296896 t.cc:2] test
*** Check failure stack trace: ***
@ 0x7fe9d3c4562a google::LogMessage::Fail()
@ 0x7fe9d3c4ee9d google::LogMessage::SendToLog()
@ 0x7fe9d3c44be2 google::LogMessage::Flush()
@ 0x7fe9d3c483d9 google::LogMessageFatal::~LogMessageFatal()
@ 0x564e58bce29c h()
@ 0x564e58bce2c5 g()
@ 0x564e58bce2d1 f()
@ 0x564e58bce2f7 main
@ 0x7fe9d3645cd0 (unknown)
@ 0x7fe9d3645d8a __libc_start_main
@ 0x564e58bce155 _start
Aborted (core dumped)