KeyDB icon indicating copy to clipboard operation
KeyDB copied to clipboard

[CRASH]

Open RamZy666 opened this issue 4 years ago • 3 comments

Crash report

Paste the complete crash log between the quotes below. Please include a few lines from the log preceding the crash report to provide some context.

=== KEYDB BUG REPORT START: Cut & paste starting from here ===
106476:106483:M 03 Jun 2021 20:50:57.010 # KeyDB 6.0.16 crashed by signal: 11
106476:106483:M 03 Jun 2021 20:50:57.010 # Crashed running the instruction at: 0x45deac
106476:106483:M 03 Jun 2021 20:50:57.010 # Accessing address: 0xffffffffffffffff
106476:106483:M 03 Jun 2021 20:50:57.010 # Failed assertion: <no assertion failed> (<no file>:0)

------ STACK TRACE ------
EIP:
/usr/bin/keydb-server *:6379(processInputBuffer(client*, int)+0x8c) [0x45deac]

Backtrace:
/usr/bin/keydb-server *:6379(logStackTrace(ucontext_t*)+0x29) [0x49d969]
/usr/bin/keydb-server *:6379(sigsegvHandler(int, siginfo_t*, void*)+0xae) [0x49e00e]
/lib64/libpthread.so.0(+0x12b20) [0x7f2d20e21b20]
/usr/bin/keydb-server *:6379(processInputBuffer(client*, int)+0x8c) [0x45deac]
/usr/bin/keydb-server *:6379(processClients()+0x9c) [0x46098c]
/usr/bin/keydb-server *:6379(beforeSleep(aeEventLoop*)+0x75) [0x445fd5]
/usr/bin/keydb-server *:6379(aeMain+0x6b) [0x442dfb]
/usr/bin/keydb-server *:6379(workerThreadMain(void*)+0x72) [0x448ba2]
/lib64/libpthread.so.0(+0x814a) [0x7f2d20e1714a]
/lib64/libc.so.6(clone+0x43) [0x7f2d20b48f23]

------ INFO OUTPUT ------
# Server
redis_version:6.0.16
redis_git_sha1:63c47a09
redis_git_dirty:0
redis_build_id:5663fdc39edfab11
redis_mode:standalone
os:Linux 4.18.0-240.10.1.el8_3.x86_64 x86_64
arch_bits:64
multiplexing_api:epoll
atomicvar_api:atomic-builtin
gcc_version:8.3.1
process_id:106476
run_id:7b78e59d5e5b654bfe0e0e5a2c31539c9210bc0d
tcp_port:6379
uptime_in_seconds:1254024
uptime_in_days:14
hz:10
configured_hz:10
lru_clock:12155117
executable:/usr/bin/keydb-server
config_file:/etc/keydb/keydb.conf

# Clients
connected_clients:22
client_recent_max_input_buffer:77
client_recent_max_output_buffer:0
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0
current_client_thread:1
thread_0_clients:20
thread_1_clients:2

# Memory
used_memory:10801792
used_memory_human:10.30M
used_memory_rss:28983296
used_memory_rss_human:27.64M
used_memory_peak:11931400
used_memory_peak_human:11.38M
used_memory_peak_perc:90.53%
used_memory_overhead:10526841
used_memory_startup:9087192
used_memory_dataset:274951
used_memory_dataset_perc:16.04%
allocator_allocated:11501080
allocator_active:12718080
allocator_resident:15327232
total_system_memory:8103337984
total_system_memory_human:7.55G
used_memory_lua:37888
used_memory_lua_human:37.00K
used_memory_scripts:0
used_memory_scripts_human:0B
number_of_cached_scripts:0
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.11
allocator_frag_bytes:1217000
allocator_rss_ratio:1.21
allocator_rss_bytes:2609152
rss_overhead_ratio:1.89
rss_overhead_bytes:13656064
mem_fragmentation_ratio:2.67
mem_fragmentation_bytes:18121064
mem_not_counted_for_evict:0
mem_replication_backlog:1048576
mem_clients_slaves:0
mem_clients_normal:378169
mem_aof_buffer:0
mem_allocator:jemalloc-5.1.0
active_defrag_running:0
lazyfree_pending_objects:0

# Persistence
loading:0
rdb_changes_since_last_save:21
rdb_bgsave_in_progress:0
rdb_last_save_time:1622767663
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:11
rdb_current_bgsave_time_sec:-1
rdb_last_cow_size:3280896
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0

# Stats
total_connections_received:422295
total_commands_processed:13998242
instantaneous_ops_per_sec:1
total_net_input_bytes:1473692811
total_net_output_bytes:2094802126
instantaneous_input_kbps:0.09
instantaneous_output_kbps:0.03
rejected_connections:0
sync_full:17
sync_partial_ok:0
sync_partial_err:0
expired_keys:16
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:0
evicted_keys:0
keyspace_hits:5256336
keyspace_misses:4422707
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:805
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0

# Replication
role:master
connected_slaves:0
master_replid:e54210eff3d7dfc277edc743e8926bba3f20c416
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:1275863469
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1274814894
repl_backlog_histlen:1048576

# CPU
used_cpu_sys:1056.182023
used_cpu_user:989.938753
used_cpu_sys_children:14.456021
used_cpu_user_children:4.245582
server_threads:2
long_lock_waits:5398

# Modules

# Commandstats
cmdstat_mset:calls=181,usec=693,usec_per_call=3.83
cmdstat_auth:calls=441629,usec=3792000,usec_per_call=8.59
cmdstat_replconf:calls=1245937,usec=97641109,usec_per_call=78.37
cmdstat_setex:calls=16,usec=134,usec_per_call=8.38
cmdstat_exists:calls=6222864,usec=8705106,usec_per_call=1.40
cmdstat_get:calls=3063043,usec=37296284,usec_per_call=12.18
cmdstat_del:calls=3586,usec=18151,usec_per_call=5.06
cmdstat_psync:calls=17,usec=26364980,usec_per_call=1550881.12
cmdstat_select:calls=400886,usec=247768,usec_per_call=0.62
cmdstat_set:calls=2620083,usec=13355930,usec_per_call=5.10

# Cluster
cluster_enabled:0

# Keyspace
db0:keys=220,expires=0,avg_ttl=0

------ CLIENT LIST OUTPUT ------
id=290259 addr=127.0.0.1:58542 fd=107 name= age=382954 idle=121230 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=set user=default
id=290260 addr=127.0.0.1:58544 fd=108 name= age=382954 idle=131 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=set user=default
id=243332 addr=127.0.0.1:37432 fd=97 name= age=523612 idle=124 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=get user=default
id=267225 addr=127.0.0.1:42928 fd=105 name= age=451838 idle=314576 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=set user=default
id=267226 addr=127.0.0.1:42930 fd=104 name= age=451838 idle=121230 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=set user=default
id=243335 addr=127.0.0.1:37802 fd=98 name= age=523605 idle=4528 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=exists user=default
id=243322 addr=127.0.0.1:37266 fd=89 name= age=523615 idle=123 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=get user=default
id=243323 addr=127.0.0.1:37332 fd=90 name= age=523615 idle=131 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=set user=default
id=313255 addr=127.0.0.1:44952 fd=109 name= age=314576 idle=131 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=73 obl=0 oll=0 omem=0 events=r cmd=set user=default
id=243480 addr=127.0.0.1:41568 fd=101 name= age=523177 idle=121230 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=set user=default
id=243324 addr=127.0.0.1:37348 fd=91 name= age=523615 idle=2939 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=exists user=default
id=243340 addr=127.0.0.1:38094 fd=99 name= age=523593 idle=131 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=set user=default
id=243325 addr=127.0.0.1:37350 fd=92 name= age=523615 idle=126 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=set user=default
id=422299 addr=180.76.38.92:56238 fd=110 name= age=42 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=14 qbuf-free=32754 obl=0 oll=0 omem=0 events=r cmd=NULL user=default
id=243326 addr=127.0.0.1:37352 fd=93 name= age=523615 idle=131 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=set user=default
id=243341 addr=127.0.0.1:38096 fd=100 name= age=523593 idle=13367 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=exists user=default
id=245007 addr=127.0.0.1:42588 fd=102 name= age=518597 idle=129 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=set user=default
id=245008 addr=127.0.0.1:42590 fd=103 name= age=518597 idle=131 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=set user=default
id=243331 addr=127.0.0.1:37430 fd=96 name= age=523612 idle=127 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=get user=default
id=400123 addr=127.0.0.1:46062 fd=94 name= age=61296 idle=128 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=get user=default
id=400124 addr=127.0.0.1:46060 fd=88 name= age=61296 idle=128 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=get user=default
id=275247 addr=127.0.0.1:54300 fd=106 name= age=427860 idle=354780 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=setex user=default

------ REGISTERS ------
106476:106483:M 03 Jun 2021 20:51:14.092 # 
RAX:00000000007fceb0 RBX:00007f2d05e305c0
RCX:0000000000000000 RDX:00000000007fced8
RDI:00007f2d1d628000 RSI:0000000000000000
RBP:fffffffffffffbe0 RSP:00007f2d07ffd020
R8 :00000000007fc980 R9 :0000000000010000
R10:0000000000000000 R11:0000000000000246
R12:000000000000000f R13:fffffffffffffbe0
R14:0000000000000000 R15:00007f2d07ffd1c0
RIP:000000000045deac EFL:0000000000010246
CSGSFS:002b000000000033
106476:106483:M 03 Jun 2021 20:51:14.092 # (00007f2d07ffd02f) -> 00007ffe5d9cf53e
106476:106483:M 03 Jun 2021 20:51:14.092 # (00007f2d07ffd02e) -> 0000000000000001
106476:106483:M 03 Jun 2021 20:51:14.092 # (00007f2d07ffd02d) -> 00007f2d1d628000
106476:106483:M 03 Jun 2021 20:51:14.092 # (00007f2d07ffd02c) -> 00007ffe5d9cf53f
106476:106483:M 03 Jun 2021 20:51:14.092 # (00007f2d07ffd02b) -> 0000000000445fd5
106476:106483:M 03 Jun 2021 20:51:14.092 # (00007f2d07ffd02a) -> 00007ffe5d9cf53f
106476:106483:M 03 Jun 2021 20:51:14.092 # (00007f2d07ffd029) -> 00007ffe5d9cf53e
106476:106483:M 03 Jun 2021 20:51:14.092 # (00007f2d07ffd028) -> 00007f2d1d628000
106476:106483:M 03 Jun 2021 20:51:14.092 # (00007f2d07ffd027) -> 0000000000000001
106476:106483:M 03 Jun 2021 20:51:14.092 # (00007f2d07ffd026) -> 0000000000000003
106476:106483:M 03 Jun 2021 20:51:14.092 # (00007f2d07ffd025) -> 0000000000000001
106476:106483:M 03 Jun 2021 20:51:14.092 # (00007f2d07ffd024) -> 00007f2d05e3485c
106476:106483:M 03 Jun 2021 20:51:14.092 # (00007f2d07ffd023) -> 000000000046098c
106476:106483:M 03 Jun 2021 20:51:14.092 # (00007f2d07ffd022) -> 00007f2d0c20f110
106476:106483:M 03 Jun 2021 20:51:14.092 # (00007f2d07ffd021) -> 00007f2d05e305c0
106476:106483:M 03 Jun 2021 20:51:14.092 # (00007f2d07ffd020) -> 00007f2d0c20f108

------ MODULES INFO OUTPUT ------

------ FAST MEMORY TEST ------
106476:106483:M 03 Jun 2021 20:51:14.092 # Bio thread for job type #0 terminated
106476:106483:M 03 Jun 2021 20:51:14.092 # Bio thread for job type #1 terminated
106476:106483:M 03 Jun 2021 20:51:14.092 # Bio thread for job type #2 terminated
*** Preparing to test memory region 7fd000 (2277376 bytes)
*** Preparing to test memory region 25f5000 (270336 bytes)
*** Preparing to test memory region 2637000 (135168 bytes)
*** Preparing to test memory region 7f2cf8000000 (135168 bytes)
*** Preparing to test memory region 7f2d00000000 (135168 bytes)
*** Preparing to test memory region 7f2d055ff000 (10485760 bytes)
*** Preparing to test memory region 7f2d06000000 (8388608 bytes)
*** Preparing to test memory region 7f2d06800000 (2097152 bytes)
*** Preparing to test memory region 7f2d06c00000 (8388608 bytes)
*** Preparing to test memory region 7f2d07400000 (2097152 bytes)
*** Preparing to test memory region 7f2d07800000 (8388608 bytes)
*** Preparing to test memory region 7f2d08000000 (135168 bytes)
*** Preparing to test memory region 7f2d0c04e000 (2097152 bytes)
*** Preparing to test memory region 7f2d0c24f000 (8388608 bytes)
*** Preparing to test memory region 7f2d0ca50000 (8388608 bytes)
*** Preparing to test memory region 7f2d0d251000 (8388608 bytes)
*** Preparing to test memory region 7f2d0da52000 (8388608 bytes)
*** Preparing to test memory region 7f2d0e665000 (24576 bytes)
*** Preparing to test memory region 7f2d0e66c000 (8388608 bytes)
*** Preparing to test memory region 7f2d0f94b000 (9437184 bytes)
*** Preparing to test memory region 7f2d1d200000 (8388608 bytes)
*** Preparing to test memory region 7f2d1df8f000 (36864 bytes)
*** Preparing to test memory region 7f2d1e1c0000 (8192 bytes)
*** Preparing to test memory region 7f2d1e815000 (8192 bytes)
*** Preparing to test memory region 7f2d1ea1a000 (4096 bytes)
*** Preparing to test memory region 7f2d1f1c3000 (4096 bytes)
*** Preparing to test memory region 7f2d1f3cf000 (4096 bytes)
*** Preparing to test memory region 7f2d20397000 (4096 bytes)
*** Preparing to test memory region 7f2d20824000 (4096 bytes)
*** Preparing to test memory region 7f2d20e0b000 (16384 bytes)
*** Preparing to test memory region 7f2d2102b000 (16384 bytes)
*** Preparing to test memory region 7f2d2195b000 (12288 bytes)
*** Preparing to test memory region 7f2d21e40000 (16384 bytes)
*** Preparing to test memory region 7f2d220d7000 (4096 bytes)
*** Preparing to test memory region 7f2d222df000 (4096 bytes)
*** Preparing to test memory region 7f2d228f2000 (8192 bytes)
*** Preparing to test memory region 7f2d22d8b000 (90112 bytes)
*** Preparing to test memory region 7f2d22dac000 (4096 bytes)
.O.O.O.O.O.O.O.O.O.O.733379:733379:C 03 Jun 2021 20:51:20.434 # oO0OoO0OoO0Oo KeyDB is starting oO0OoO0OoO0Oo
733379:733379:C 03 Jun 2021 20:51:20.434 # KeyDB version=6.0.16, bits=64, commit=63c47a09, modified=0, pid=733379, just started
733379:733379:C 03 Jun 2021 20:51:20.434 # Configuration loaded
                                                                     
                                                                     
                                        KeyDB 6.0.16 (63c47a09/0) 64 bit
                                                                     
                                        Running in standalone mode
                                        Port: 6379
                                        PID: 733386
                                                                     
                     Join the KeyDB community! https://community.keydb.dev/

                                                                   

Aditional information

  1. OS distribution and version CentOS 8.3
  2. Steps to reproduce (if any)

RamZy666 avatar Jun 28 '21 00:06 RamZy666

I wasn't able to reproduce this with the information here. Could you answer a couple questions for me?

  1. Please post the config file you used to run the server.
  2. What commands (if any) were being run at the time of the crash?
  3. What kind of traffic does the database usually see? What commands are usually run and how often?

christianEQ avatar Jun 30 '21 20:06 christianEQ

We just ran into this a few different times. Each time was after a promotion to master. I had 4 instances of this during a deploy yesterday:

1:11:M 08 Nov 2022 18:09:15.312 * MASTER MODE enabled (user request from 'id=29 addr=245.134.111.221:54348 fd=0 name= age=11 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=34 qbuf-free=32734 obl=0 oll=0 omem=0 events=r cmd=slaveof user=default')


=== KEYDB BUG REPORT START: Cut & paste starting from here ===
1:11:M 08 Nov 2022 18:09:15.312 # KeyDB 6.0.16 crashed by signal: 11
1:11:M 08 Nov 2022 18:09:15.312 # Crashed running the instruction at: 0x564e09a69868
1:11:M 08 Nov 2022 18:09:15.312 # Accessing address: 0xffffffffffffffff
1:11:M 08 Nov 2022 18:09:15.312 # Failed assertion: <no assertion failed> (<no file>:0)

------ STACK TRACE ------
EIP:
/usr/local/bin/keydb-server *:6379(processInputBuffer(client*, int)+0x98) [0x564e09a69868]

Backtrace:
/usr/local/bin/keydb-server *:6379(logStackTrace(ucontext_t*)+0x6b) [0x564e09aadd4b]
/usr/local/bin/keydb-server *:6379(sigsegvHandler(int, siginfo_t*, void*)+0xc9) [0x564e09aae4e9]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x12980) [0x7f42bbda6980]
/usr/local/bin/keydb-server *:6379(processInputBuffer(client*, int)+0x98) [0x564e09a69868]
/usr/local/bin/keydb-server *:6379(processClients()+0xac) [0x564e09a6c79c]
/usr/local/bin/keydb-server *:6379(beforeSleep(aeEventLoop*)+0x8d) [0x564e09a501bd]
/usr/local/bin/keydb-server *:6379(aeMain+0x81) [0x564e09a4c7c1]
/usr/local/bin/keydb-server *:6379(workerThreadMain(void*)+0x74) [0x564e09a53114]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7f42bbd9b6db]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7f42bbac461f]

------ INFO OUTPUT ------
# Server
redis_version:6.0.16
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:95bd57fa39baf58d
redis_mode:standalone
os:Linux 5.10.109+ x86_64
arch_bits:64
multiplexing_api:epoll
atomicvar_api:atomic-builtin
gcc_version:7.5.0
process_id:1
run_id:c8aa1e94dd77ba130a6ae9b088142cc441f09ff1
tcp_port:6379
uptime_in_seconds:29
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:6986571
executable:/usr/local/bin/keydb-server
config_file:/etc/redis/redis.conf

# Clients
connected_clients:1
client_recent_max_input_buffer:726
client_recent_max_output_buffer:0
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0
current_client_thread:0
thread_0_clients:1
thread_1_clients:0
thread_2_clients:0
thread_3_clients:0

# Memory
used_memory:1061474176
used_memory_human:1012.30M
used_memory_rss:1082314752
used_memory_rss_human:1.01G
used_memory_peak:1063040200
used_memory_peak_human:1013.79M
used_memory_peak_perc:99.85%
used_memory_overhead:140527498
used_memory_startup:84305048
used_memory_dataset:920946678
used_memory_dataset_perc:94.25%
allocator_allocated:1061950408
allocator_active:1114333184
allocator_resident:1130569728
total_system_memory:109720866816
total_system_memory_human:102.19G
used_memory_lua:61440
used_memory_lua_human:60.00K
used_memory_scripts:5512
used_memory_scripts_human:5.38K
number_of_cached_scripts:7
maxmemory:4194304000
maxmemory_human:3.91G
maxmemory_policy:allkeys-lru
allocator_frag_ratio:1.05
allocator_frag_bytes:52382776
allocator_rss_ratio:1.01
allocator_rss_bytes:16236544
rss_overhead_ratio:0.96
rss_overhead_bytes:-48254976
mem_fragmentation_ratio:1.02
mem_fragmentation_bytes:20830760
mem_not_counted_for_evict:0
mem_replication_backlog:1048576
mem_clients_slaves:0
mem_clients_normal:17186
mem_aof_buffer:0
mem_allocator:jemalloc-5.1.0
active_defrag_running:0
lazyfree_pending_objects:0

# Persistence
loading:0
rdb_changes_since_last_save:16565
rdb_bgsave_in_progress:0
rdb_last_save_time:1667930926
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
rdb_last_cow_size:0
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0

# Stats
total_connections_received:28
total_commands_processed:20959
instantaneous_ops_per_sec:9169
total_net_input_bytes:1539234
total_net_output_bytes:16748
instantaneous_input_kbps:614.55
instantaneous_output_kbps:3.41
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:0
evicted_keys:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0

# Replication
role:master
connected_slaves:0
master_replid:8d88fe1b7e0772f9cac3f16bc79dda5429a14beb
master_replid2:6f3b92dfefcea10d2cef42fd11450cca9d586430
master_repl_offset:779767880667
second_repl_offset:779767880668
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:779766832092
repl_backlog_histlen:1048576

# CPU
used_cpu_sys:1.580116
used_cpu_user:9.740707
used_cpu_sys_children:0.000000
used_cpu_user_children:0.000000
server_threads:4
long_lock_waits:5

# Modules

# Commandstats
cmdstat_sadd:calls=15,usec=40,usec_per_call=2.67
cmdstat_lpush:calls=2,usec=10,usec_per_call=5.00
cmdstat_setex:calls=8564,usec=23287,usec_per_call=2.72
cmdstat_multi:calls=4294,usec=628,usec_per_call=0.15
cmdstat_replconf:calls=57,usec=152,usec_per_call=2.67
cmdstat_hmset:calls=15,usec=139,usec_per_call=9.27
cmdstat_del:calls=152,usec=1011,usec_per_call=6.65
cmdstat_zadd:calls=624,usec=3831,usec_per_call=6.14
cmdstat_client:calls=2,usec=79,usec_per_call=39.50
cmdstat_incr:calls=13,usec=35,usec_per_call=2.69
cmdstat_incrby:calls=7,usec=30,usec_per_call=4.29
cmdstat_setnx:calls=8,usec=21,usec_per_call=2.62
cmdstat_expire:calls=32,usec=58,usec_per_call=1.81
cmdstat_ping:calls=20,usec=17,usec_per_call=0.85
cmdstat_hset:calls=21,usec=239,usec_per_call=11.38
cmdstat_slowlog:calls=1,usec=3,usec_per_call=3.00
cmdstat_hdel:calls=1,usec=2,usec_per_call=2.00
cmdstat_info:calls=15,usec=612,usec_per_call=40.80
cmdstat_set:calls=2788,usec=5044,usec_per_call=1.81
cmdstat_slaveof:calls=2,usec=255,usec_per_call=127.50
cmdstat_zincrby:calls=25,usec=447,usec_per_call=17.88
cmdstat_exec:calls=4294,usec=29477,usec_per_call=6.86
cmdstat_zrem:calls=2,usec=21,usec_per_call=10.50
cmdstat_select:calls=1,usec=1,usec_per_call=1.00
cmdstat_ltrim:calls=2,usec=3,usec_per_call=1.50
cmdstat_config:calls=2,usec=43,usec_per_call=21.50

# Cluster
cluster_enabled:0

# Keyspace
db0:keys=815912,expires=806652,avg_ttl=47278025

------ CLIENT LIST OUTPUT ------
id=29 addr=245.134.111.221:54348 fd=0 name= age=11 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=5 oll=0 omem=0 events=r cmd=slaveof user=default

------ REGISTERS ------
1:11:M 08 Nov 2022 18:09:15.313 #
RAX:0000564e09e168c0 RBX:00007f424e778440
RCX:0000000000000000 RDX:0000564e09e168e8
RDI:0000000000000000 RSI:00007f42b5e28100
RBP:fffffffffffffbe8 RSP:00007f42ae4790e0
R8 :0000564e09baafd0 R9 :0000000000001000
R10:0000000000000001 R11:0000000000000003
R12:000000000000000f R13:0000564e09bad308
R14:0000000000000000 R15:00007fff6faea980
RIP:0000564e09a69868 EFL:0000000000010246
CSGSFS:002b000000000033
1:11:M 08 Nov 2022 18:09:15.313 # (00007f42ae4790ef) -> 0000564e09a501bd
1:11:M 08 Nov 2022 18:09:15.313 # (00007f42ae4790ee) -> 0000000000000000
1:11:M 08 Nov 2022 18:09:15.313 # (00007f42ae4790ed) -> 00007f42ae4792c0
1:11:M 08 Nov 2022 18:09:15.313 # (00007f42ae4790ec) -> 00007f42b5e28100
1:11:M 08 Nov 2022 18:09:15.313 # (00007f42ae4790eb) -> 0000000000000000
1:11:M 08 Nov 2022 18:09:15.313 # (00007f42ae4790ea) -> 0000564e09e165e0
1:11:M 08 Nov 2022 18:09:15.313 # (00007f42ae4790e9) -> f1e5bd6e78039700
1:11:M 08 Nov 2022 18:09:15.313 # (00007f42ae4790e8) -> 0000000000000000
1:11:M 08 Nov 2022 18:09:15.313 # (00007f42ae4790e7) -> 0000564e09a4c401
1:11:M 08 Nov 2022 18:09:15.313 # (00007f42ae4790e6) -> 00007f424e77c6dc
1:11:M 08 Nov 2022 18:09:15.313 # (00007f42ae4790e5) -> 0000564e09a6c79c
1:11:M 08 Nov 2022 18:09:15.313 # (00007f42ae4790e4) -> fffffffffffffbe8
1:11:M 08 Nov 2022 18:09:15.313 # (00007f42ae4790e3) -> 00007f428ea63ea0
1:11:M 08 Nov 2022 18:09:15.313 # (00007f42ae4790e2) -> 00007f424e778440
1:11:M 08 Nov 2022 18:09:15.313 # (00007f42ae4790e1) -> 00007f428ea63ea0
1:11:M 08 Nov 2022 18:09:15.313 # (00007f42ae4790e0) -> 0000000000000000

------ MODULES INFO OUTPUT ------

------ FAST MEMORY TEST ------
1:11:M 08 Nov 2022 18:09:15.313 # Bio thread for job type #0 terminated
1:11:M 08 Nov 2022 18:09:15.313 # Bio thread for job type #1 terminated
1:11:M 08 Nov 2022 18:09:15.313 # Bio thread for job type #2 terminated
*** Preparing to test memory region 564e09e17000 (2277376 bytes)
*** Preparing to test memory region 564e0b5e2000 (323584 bytes)
*** Preparing to test memory region 7f424be00000 (817889280 bytes)
*** Preparing to test memory region 7f427cbff000 (381681664 bytes)
*** Preparing to test memory region 7f4293800000 (8388608 bytes)
*** Preparing to test memory region 7f4294000000 (135168 bytes)
*** Preparing to test memory region 7f4298000000 (135168 bytes)
*** Preparing to test memory region 7f429c000000 (135168 bytes)
*** Preparing to test memory region 7f42a0000000 (135168 bytes)
*** Preparing to test memory region 7f42a4000000 (135168 bytes)
*** Preparing to test memory region 7f42a8200000 (6291456 bytes)
*** Preparing to test memory region 7f42a8c00000 (8388608 bytes)
*** Preparing to test memory region 7f42a9400000 (4194304 bytes)
*** Preparing to test memory region 7f42a9a00000 (8388608 bytes)
*** Preparing to test memory region 7f42aa200000 (4194304 bytes)
*** Preparing to test memory region 7f42aa800000 (8388608 bytes)
*** Preparing to test memory region 7f42ab000000 (4194304 bytes)
*** Preparing to test memory region 7f42ab7f9000 (24576 bytes)
*** Preparing to test memory region 7f42ab7ff000 (2097152 bytes)
*** Preparing to test memory region 7f42aba00000 (8388608 bytes)
*** Preparing to test memory region 7f42ac200000 (2097152 bytes)
*** Preparing to test memory region 7f42ac479000 (8388608 bytes)
*** Preparing to test memory region 7f42acc7a000 (8388608 bytes)
*** Preparing to test memory region 7f42ad47b000 (8388608 bytes)
*** Preparing to test memory region 7f42adc7c000 (8388608 bytes)
*** Preparing to test memory region 7f42ae47d000 (8388608 bytes)
*** Preparing to test memory region 7f42aec7e000 (8388608 bytes)
*** Preparing to test memory region 7f42af47f000 (8388608 bytes)
*** Preparing to test memory region 7f42afc80000 (8388608 bytes)
*** Preparing to test memory region 7f42b0480000 (98041856 bytes)
*** Preparing to test memory region 7f42b652e000 (188416 bytes)
*** Preparing to test memory region 7f42b6864000 (4096 bytes)
*** Preparing to test memory region 7f42b6aae000 (4096 bytes)
*** Preparing to test memory region 7f42b753a000 (4096 bytes)
*** Preparing to test memory region 7f42b7a69000 (4096 bytes)
*** Preparing to test memory region 7f42b83b8000 (4096 bytes)
*** Preparing to test memory region 7f42b8a2d000 (8192 bytes)
*** Preparing to test memory region 7f42b906f000 (4096 bytes)
*** Preparing to test memory region 7f42b9d98000 (4096 bytes)
*** Preparing to test memory region 7f42ba792000 (8192 bytes)
*** Preparing to test memory region 7f42baea7000 (12288 bytes)
*** Preparing to test memory region 7f42bbd90000 (16384 bytes)
*** Preparing to test memory region 7f42bbfaf000 (16384 bytes)
*** Preparing to test memory region 7f42bc8ee000 (16384 bytes)
*** Preparing to test memory region 7f42bd391000 (114688 bytes)
*** Preparing to test memory region 7f42bd3b2000 (4096 bytes)
.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.%
➜  ~```

inf-rno avatar Nov 09 '22 21:11 inf-rno

We updated to 6.2.2 and havent seen the issue

inf-rno avatar Nov 14 '22 15:11 inf-rno

Closing as fixed with 6.2.2, please re-open if you run into it again.

msotheeswaran-sc avatar Jan 17 '23 08:01 msotheeswaran-sc