glusterfs icon indicating copy to clipboard operation
glusterfs copied to clipboard

volume brick is crash , and can not start

Open WelldoneIT opened this issue 3 years ago • 6 comments

volume brick is down and left core dump file,the log do not show any useful information after 'pending frames'.

The exact command to reproduce the issue: no special operations were taken.

**- Is there any crash ? Provide the backtrace and coredump yes, show like this

Reading symbols from /usr/sbin/glusterfsd...Reading symbols from /usr/lib/debug/usr/sbin/glusterfsd.debug...done. done. Missing separate debuginfo for Try: yum --enablerepo='debug' install /usr/lib/debug/.build-id/16/3c2dc43405427478788bad0afd537a7acf7a13 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `/usr/sbin/glusterfsd -s nasgddged0007 --volfile-id dgjdvol01cl14.nasgddged0007.'. Program terminated with signal 11, Segmentation fault. #0 0x00007fb03858a12a in __GI__IO_vfscanf () from /lib64/libc.so.6 Missing separate debuginfos, use: debuginfo-install glibc-2.17-260.el7.x86_64 keyutils-libs-1.5.8-3.el7.x86_64 krb5-libs-1.15.1-34.el7.x86_64 libacl-2.2.51-14.el7.x86_64 libaio-0.3.109-13.el7.x86_64 libattr-2.4.46-13.el7.x86_64 libcom_err-1.42.9-13.el7.x86_64 libgcc-4.8.5-36.el7.x86_64 libselinux-2.5-14.1.el7.x86_64 libuuid-2.23.2-59.el7.x86_64 openssl-libs-1.0.2k-8.el7.x86_64 pcre-8.32-17.el7.x86_64 sqlite-3.7.17-8.el7.x86_64 zlib-1.2.7-18.el7.x86_64 (gdb) bt #0 0x00007fb03858a12a in __GI__IO_vfscanf () from /lib64/libc.so.6 #1 0x00007fb0385a2357 in vsscanf () from /lib64/libc.so.6 #2 0x00007fb03859c2c7 in sscanf () from /lib64/libc.so.6 #3 0x00007fb0385e63e8 in __tzset_parse_tz () from /lib64/libc.so.6 #4 0x00007fb0385e87db in __tzfile_compute () from /lib64/libc.so.6 #5 0x00007fb0385e7177 in __tz_convert () from /lib64/libc.so.6 #6 0x00007fb038628a3a in __vsyslog_chk () from /lib64/libc.so.6 #7 0x00007fb038628ff2 in __syslog_chk () from /lib64/libc.so.6 #8 0x00007fb039f2a5cb in syslog (__fmt=0x7fb039fd0ce8 "%s\n", __pri=1) at /usr/include/bits/syslog.h:31 #9 _gf_msg_plain_internal (level=level@entry=GF_LOG_ALERT, msg=msg@entry=0x7fb039fccb01 "pending frames:") at logging.c:1027 #10 0x00007fb039f2a827 in _gf_msg_plain_nomem (level=level@entry=GF_LOG_ALERT, msg=msg@entry=0x7fb039fccb01 "pending frames:") at logging.c:1116 #11 0x00007fb039f34b76 in gf_print_trace (signum=11, ctx=0x5633f01c5010) at com

Additional info:

- The operating system / glusterfs version:

Note: Please hide any confidential data which you don't want to share in public like IP address, file name, hostname or any other configuration

WelldoneIT avatar Jul 18 '22 12:07 WelldoneIT

please give me some suggestion, thank you very much!

WelldoneIT avatar Jul 18 '22 13:07 WelldoneIT

There are also some additions for core dump: #11 0x00007fb039f34b76 in gf_print_trace (signum=11, ctx=0x5633f01c5010) at common-utils.c:892 #12 #13 0x00007fb03856c303 in ____strtoull_l_internal () from /lib64/libc.so.6 #14 0x00007fb03968b4bf in uuid_parse () from /lib64/libuuid.so.1 #15 0x00007fb02c044adf in gf_uuid_parse (uuid=, in=) at ../../../../libglusterfs/src/compat-uuid.h:51 #16 posix_make_ancestryfromgfid (this=, path=, pathsize=, head=, type=, gfid=, handle_size=14144332474911870584, priv_base_path=0x8d5503316f678d94 <Address 0x8d5503316f678d94 out of bounds>, itable=0xc844a44990d7fb7a, parent=0xb3717e3b44366d8c, xdata=0xc44ac3723024ce78, op_errno=0x8d5503316f678d94) at posix-handle.c:187 #17 0xb3717e3b44366d8c in ?? ()

WelldoneIT avatar Jul 18 '22 13:07 WelldoneIT

What is the gluster version you are using ?

mohit84 avatar Jul 18 '22 13:07 mohit84

the gluster version is 3.12

WelldoneIT avatar Jul 19 '22 02:07 WelldoneIT

why core dump show ‘Address 0x8d5503316f678d94 out of bounds’ ?

WelldoneIT avatar Jul 19 '22 02:07 WelldoneIT

The gdb is showing "Address out of bound " error because of buffer overflow, you are using a very old version the currently maintained version is release-9/10 so please upgrade the latest version to avoid the issue.

mohit84 avatar Jul 19 '22 02:07 mohit84

Thank you for your contributions. Noticed that this issue is not having any activity in last ~6 months! We are marking this issue as stale because it has not had recent activity. It will be closed in 2 weeks if no one responds with a comment here.

stale[bot] avatar Mar 19 '23 22:03 stale[bot]