KeyDB icon indicating copy to clipboard operation
KeyDB copied to clipboard
verified publisher icon Snapchat

[CRASH] Active replicas crashed during RDB network transfer

Open reneklacan opened this issue 3 years ago • 3 comments

Crash report

https://gist.github.com/reneklacan/f3fcd8aee08605f0f808abd1b72c63d8

Aditional information

OS distribution and version: Linux 5.4.0-1073-gcp #78~18.04.1-Ubuntu SMP Wed Apr 27 09:05:35 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux KeyDB version: 6.3.1

  1. We have 2 node active-replica setup
  2. Upgraded first server from 6.2 to 6.3 and let it full resync from other node on 6.2
  3. Stopped keydb on second server and upgraded to 6.3
  4. Started keydb on second server
  5. After first keydb did dump, it started sending it to second keydb
  6. Both instances crashed around 0.5% progress during sending dump

reneklacan avatar Jun 08 '22 09:06 reneklacan

I have some issue with 6.3.1

R4scal avatar Jun 11 '22 09:06 R4scal

@reneklacan Do you use docker/kubernetes or keydb is installed in a server/standalone? I'm use k8s and I saw this bug in v6.3.0 but after update to v6.3.1, this bug is disappeared for me.

I'm interested how your bug is appeared and how or when it will fixed.

Thanks.

agosalvez avatar Jun 16 '22 13:06 agosalvez

@agosalvez standalone (GCP VM solely running only KeyDB and nothing else)

reneklacan avatar Jun 16 '22 15:06 reneklacan

Could any of you share your config, data size/usage, and load on servers(including what commands) during crash to help me reproduce the issue?

msotheeswaran-sc avatar Feb 02 '23 20:02 msotheeswaran-sc

@msotheeswaran it has been more than 6 months now so I don't have any additional info anymore, after numerous issues we abandoned KeyDB

reneklacan avatar Feb 03 '23 18:02 reneklacan

Closing as I am unable to reproduce and it seems no one else is currently facing this issue.

msotheeswaran-sc avatar Feb 27 '23 21:02 msotheeswaran-sc