[BUG]redis cluster with nodeport cluster nodes info not correct when hscale replica

Open ahjing99 opened this issue 1 year ago • 1 comments

kbcli version Kubernetes: v1.28.7-gke.1026000 KubeBlocks: 0.9.0-beta.10 kbcli: 0.9.0-beta.1

Create cluster , kubectl apply -f cluster-shard.yaml

apiVersion: apps.kubeblocks.io/v1alpha1
kind: Cluster
metadata:
  name: redis-cluster
  namespace: default
spec:
  # Specifies the cluster termination policy.
  # - DoNotTerminate will block delete operation.
  # - Halt will delete workload resources such as statefulset, deployment workloads but keep PVCs.
  # - Delete is based on Halt and deletes PVCs.
  # - WipeOut is based on Delete and wipe out all volume snapshots and snapshot data from backup storage location.
  terminationPolicy: Delete
  # List of ShardingSpecs used to define the components that make up a cluster.
  # ComponentSpecs and ShardingSpecs cannot both be empty at the same time.
  shardingSpecs:
      # Specifies the identifier for the sharding configuration.
      # This identifier is included as part of the Service DNS name and must comply with IANA Service Naming rules.
    - name: shard
      # Specifies the number of components, all of which will have identical specifications and definitions.
      # The number of replicas for each component should be defined by template.replicas.
      # The logical relationship between these components should be maintained by the components themselves
      # The number of shards should be no less than 3
      shards: 3
      # The blueprint for the components. Generates a set of components (also referred to as shards) based on this template.
      template:
        # Specifies the name of the cluster's component.
        name: redis
        # References the name of the ComponentDefinition.
        componentDef: redis-cluster
        # The cluster-level configuration is used as the default configuration of all components;
        # if the affinity and tolerations exists in a component, the component-level configuration
        # will take effect and cover the default cluster-level configuration
        affinity:
          # Specifies the anti-affinity level of pods within a component.
          # - Preferred
          # - Required
          podAntiAffinity: Preferred
          # Represents the key of node labels.
          topologyKeys:
            - kubernetes.io/hostname
          # Defines how pods are distributed across nodes.
          tenancy: SharedNode
        # Attached to tolerate any taint that matches the triple `key,value,effect` using the matching operator `operator`.
        tolerations:
          - key: kb-data
            operator: Equal
            value: "true"
            effect: NoSchedule
        # enable monitor of the component, default value is false if not claimed
        monitor: false
        # Indicates which log file takes effect
        enabledLogs:
          - running
        # if not specified, will use the default service account created along with cluster
        serviceAccountName: kb-redis-cluster
        # Specifies the number of component replicas.
        replicas: 1
        # Defines the strategy for switchover and failover when workloadType is Replication.
        switchPolicy:
          # Type specifies the type of switch policy to be applied.
          type: Noop
        # Specifies the resources requests and limits of the workload.
        resources:
          limits:
            cpu: "1"
            memory: "1.1Gi"
          requests:
            cpu: "1"
            memory: "1.1Gi"
        # Specifies to use the NodePort, need to delete the serviceVersion and services specs if you want to use container network
        serviceVersion: 7.0.6
        services:
          - name: redis-advertised
            podService: true
            #serverType
            #  - NodePort
            #  - LoadBalancer
            serviceType: NodePort
        # Provides information for statefulset.spec.volumeClaimTemplates.
        volumeClaimTemplates:
            # Refers to `clusterDefinition.spec.componentDefs.containers.volumeMounts.name`.
          - name: data
            spec:
              # Contains the desired access modes the volume should have.
              accessModes:
                - ReadWriteOnce
              # Represents the minimum resources the volume should have.
              resources:
                requests:
                  storage: 20Gi

Edit the yaml ,change replicas from 1 to 2, then kubectl apply -f cluster-shard.yaml

k get cluster
NAME            CLUSTER-DEFINITION   VERSION   TERMINATION-POLICY   STATUS    AGE
redis-cluster                                  Delete               Running   21m

k get pod
NAME                        READY   STATUS    RESTARTS        AGE
redis-cluster-shard-49r-0   3/3     Running   2 (9m33s ago)   12m
redis-cluster-shard-49r-1   3/3     Running   0               11m
redis-cluster-shard-sjh-0   3/3     Running   0               12m
redis-cluster-shard-sjh-1   3/3     Running   0               11m
redis-cluster-shard-wpv-0   3/3     Running   0               12m
redis-cluster-shard-wpv-1   3/3     Running   0               11m

2 new created pod cluster nodes info is not correct

# redis-cli -h 10.128.0.30 -p 31022 -a O3605v7HsS
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
10.128.0.30:31022> cluster nodes
f5cce503559018a68ffb8f29e2a1ad83f74e9690 10.128.0.30:31843@31771,redis-cluster-shard-sjh-1.redis-cluster-shard-sjh-headless.default.svc master - 0 1713521093000 0 connected
88780f6583930a9b4a6f74f1d07d1187af3e56a9 10.128.0.30:30924@31891,redis-cluster-shard-49r-0.redis-cluster-shard-49r-headless.default.svc master - 0 1713521094486 7 connected 6827-10922 15019-16383
9315a33085454e972d2dd489fffa31900d6a9f47 10.128.0.24:30628@31979,redis-cluster-shard-49r-1.redis-cluster-shard-49r-headless.default.svc master - 0 1713521093000 7 connected
8126a4ea2434ab9326e221a38b3e16e14d0dad86 10.128.0.30:31022@31002,redis-cluster-shard-wpv-1.redis-cluster-shard-wpv-headless.default.svc myself,slave 42357755c4a03bb37f759348dfdd132738b6a478 0 1713521091000 6 connected
bf35cb9cdaa340a18f19fa2fc420370b1827ad07 10.128.0.28:30301@32062,redis-cluster-shard-sjh-0.redis-cluster-shard-sjh-headless.default.svc master - 0 1713521092576 5 connected 1365-5460 12288-13653
42357755c4a03bb37f759348dfdd132738b6a478 10.128.0.28:32745@32317,redis-cluster-shard-wpv-0.redis-cluster-shard-wpv-headless.default.svc master - 0 1713521093481 6 connected 0-1364 5461-6826 10923-12287 13654-15018

# redis-cli -h 10.128.0.24 -p 30628 -a O3605v7HsS
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
10.128.0.24:30628> cluster nodes
9315a33085454e972d2dd489fffa31900d6a9f47 10.128.0.24:30628@31979,redis-cluster-shard-49r-1.redis-cluster-shard-49r-headless.default.svc myself,master - 0 1713520872839 0 connected
10.128.0.24:30628> exit
# redis-cli -h 10.128.0.30 -p 31843 -a O3605v7HsS
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
10.128.0.30:31843>  cluster nodes
f5cce503559018a68ffb8f29e2a1ad83f74e9690 10.128.0.24:31843@31771,redis-cluster-shard-sjh-1.redis-cluster-shard-sjh-headless.default.svc myself,master - 0 1713520894969 0 connected

Logs

 k logs redis-cluster-shard-sjh-1
Defaulted container "redis-cluster" out of: redis-cluster, metrics, lorry, init-lorry (init)
+ parse_redis_cluster_advertised_svc_if_exist redis-cluster-shard-sjh-1
+ local pod_name=redis-cluster-shard-sjh-1
+ [[ -z redis-cluster-shard-sjh-redis-advertised-0:30301,redis-cluster-shard-sjh-redis-advertised-1:31843 ]]
+ [[ -z redis-cluster-shard-sjh-redis-advertised-0:32062,redis-cluster-shard-sjh-redis-advertised-1:31771 ]]
+ local port
++ parse_advertised_port redis-cluster-shard-sjh-1 redis-cluster-shard-sjh-redis-advertised-0:30301,redis-cluster-shard-sjh-redis-advertised-1:31843
++ local pod_name=redis-cluster-shard-sjh-1
++ local advertised_ports=redis-cluster-shard-sjh-redis-advertised-0:30301,redis-cluster-shard-sjh-redis-advertised-1:31843
++ local pod_name_ordinal
++ local found=false
+++ extract_ordinal_from_object_name redis-cluster-shard-sjh-1
+++ local object_name=redis-cluster-shard-sjh-1
+++ local ordinal=1
+++ echo 1
++ pod_name_ordinal=1
++ IFS=,
++ read -ra ports_array
++ for entry in "${ports_array[@]}"
++ IFS=:
++ read -ra parts
++ local svc_name=redis-cluster-shard-sjh-redis-advertised-0
++ local port=30301
++ local svc_name_ordinal
+++ extract_ordinal_from_object_name redis-cluster-shard-sjh-redis-advertised-0
+++ local object_name=redis-cluster-shard-sjh-redis-advertised-0
+++ local ordinal=0
+++ echo 0
++ svc_name_ordinal=0
++ [[ 0 == \1 ]]
++ for entry in "${ports_array[@]}"
++ IFS=:
++ read -ra parts
++ local svc_name=redis-cluster-shard-sjh-redis-advertised-1
++ local port=31843
++ local svc_name_ordinal
+++ extract_ordinal_from_object_name redis-cluster-shard-sjh-redis-advertised-1
+++ local object_name=redis-cluster-shard-sjh-redis-advertised-1
+++ local ordinal=1
+++ echo 1
++ svc_name_ordinal=1
++ [[ 1 == \1 ]]
++ echo 31843
++ found=true
++ return 0
+ port=31843
+ [[ 0 -ne 0 ]]
+ [[ -z 31843 ]]
+ redis_advertised_svc_port_value=31843
+ redis_advertised_svc_host_value=10.128.0.24
+ [[ -n redis-cluster-shard-sjh-redis-advertised-0:32062,redis-cluster-shard-sjh-redis-advertised-1:31771 ]]
++ parse_advertised_port redis-cluster-shard-sjh-1 redis-cluster-shard-sjh-redis-advertised-0:32062,redis-cluster-shard-sjh-redis-advertised-1:31771
++ local pod_name=redis-cluster-shard-sjh-1
++ local advertised_ports=redis-cluster-shard-sjh-redis-advertised-0:32062,redis-cluster-shard-sjh-redis-advertised-1:31771
++ local pod_name_ordinal
++ local found=false
+++ extract_ordinal_from_object_name redis-cluster-shard-sjh-1
+++ local object_name=redis-cluster-shard-sjh-1
+++ local ordinal=1
+++ echo 1
++ pod_name_ordinal=1
++ IFS=,
++ read -ra ports_array
++ for entry in "${ports_array[@]}"
++ IFS=:
++ read -ra parts
++ local svc_name=redis-cluster-shard-sjh-redis-advertised-0
++ local port=32062
++ local svc_name_ordinal
+++ extract_ordinal_from_object_name redis-cluster-shard-sjh-redis-advertised-0
+++ local object_name=redis-cluster-shard-sjh-redis-advertised-0
+++ local ordinal=0
+++ echo 0
++ svc_name_ordinal=0
++ [[ 0 == \1 ]]
++ for entry in "${ports_array[@]}"
++ IFS=:
++ read -ra parts
++ local svc_name=redis-cluster-shard-sjh-redis-advertised-1
++ local port=31771
++ local svc_name_ordinal
+++ extract_ordinal_from_object_name redis-cluster-shard-sjh-redis-advertised-1
+++ local object_name=redis-cluster-shard-sjh-redis-advertised-1
+++ local ordinal=1
+++ echo 1
++ svc_name_ordinal=1
++ [[ 1 == \1 ]]
++ echo 31771
++ found=true
++ return 0
+ port=31771
+ [[ 0 -ne 0 ]]
+ [[ -z 31771 ]]
+ redis_advertised_svc_bus_port_value=31771
+ build_redis_conf
+ load_redis_template_conf
+ echo 'include /etc/conf/redis.conf'
+ build_redis_cluster_service_port
+ service_port=6379
+ cluster_bus_port=16379
+ '[' -n 6379 ']'
+ service_port=6379
+ '[' -n 16379 ']'
+ cluster_bus_port=16379
+ echo 'port 6379'
+ echo 'cluster-port 16379'
+ build_announce_ip_and_port
+ '[' -n 10.128.0.24 ']'
+ '[' -n 31843 ']'
+ echo 'redis use advertised svc 10.128.0.24:31843 to announce'
+ echo 'replica-announce-port 31843'
+ echo 'replica-announce-ip 10.128.0.24'
+ build_cluster_announce_info
redis use advertised svc 10.128.0.24:31843 to announce
redis cluster use advertised svc 10.128.0.24:31843@31771 to announce
+ kb_pod_fqdn=redis-cluster-shard-sjh-1.redis-cluster-shard-sjh-headless.default.svc
+ '[' -n 10.128.0.24 ']'
+ '[' -n 31843 ']'
+ '[' -n 31771 ']'
+ echo 'redis cluster use advertised svc 10.128.0.24:31843@31771 to announce'
+ echo 'cluster-announce-ip 10.128.0.24'
+ echo 'cluster-announce-port 31843'
+ echo 'cluster-announce-bus-port 31771'
+ echo 'cluster-announce-hostname redis-cluster-shard-sjh-1.redis-cluster-shard-sjh-headless.default.svc'
+ echo 'cluster-preferred-endpoint-type ip'
+ rebuild_redis_acl_file
+ '[' -f /data/users.acl ']'
+ sed -i '/user default on/d' /data/users.acl
+ sed -i '/user kbreplicator on/d' /data/users.acl
+ sed -i '/user  on/d' /data/users.acl
+ build_redis_default_accounts
+ '[' -n O3605v7HsS ']'
+ echo 'masteruser kbreplicator'
+ echo 'masterauth O3605v7HsS'
+ echo 'user kbreplicator on +psync +replconf +ping >O3605v7HsS'
+ '[' '!' -z O3605v7HsS ']'
+ echo 'protected-mode yes'
+ echo 'user default on >O3605v7HsS ~* &* +@all '
+ echo 'aclfile /data/users.acl'
+ start_redis_server
+ exec redis-server /etc/redis/redis.conf --loadmodule /opt/redis-stack/lib/redisearch.so --loadmodule /opt/redis-stack/lib/redisgraph.so --loadmodule /opt/redis-stack/lib/redistimeseries.so --loadmodule /opt/redis-stack/lib/rejson.so --loadmodule /opt/redis-stack/lib/redisbloom.so
+ scale_redis_cluster_replica
+ '[' -n O3605v7HsS ']'
+ retry redis-cli -h 127.0.0.1 -p 6379 -a O3605v7HsS ping
+ local max_attempts=20
+ local attempt=1
+ redis-cli -h 127.0.0.1 -p 6379 -a O3605v7HsS ping
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
Could not connect to Redis at 127.0.0.1:6379: Connection refused
+ '[' 1 -eq 20 ']'
+ echo 'Command '\''redis-cli -h 127.0.0.1 -p 6379 -a O3605v7HsS ping'\'' failed. Attempt 1 of 20. Retrying in 5 seconds...'
+ attempt=2
+ sleep 3
Command 'redis-cli -h 127.0.0.1 -p 6379 -a O3605v7HsS ping' failed. Attempt 1 of 20. Retrying in 5 seconds...
+ redis-cli -h 127.0.0.1 -p 6379 -a O3605v7HsS ping
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
PONG
+ '[' 2 -eq 20 ']'
+ current_pod_name=redis-cluster-shard-sjh-1
+ current_pod_fqdn=redis-cluster-shard-sjh-1.redis-cluster-shard-sjh-headless.default.svc
+ '[' -n redis-cluster-shard-sjh-0 ']'
+ target_node_fqdn=redis-cluster-shard-sjh-0.redis-cluster-shard-sjh-headless.default.svc
+ get_current_comp_nodes_for_scale_out_replica redis-cluster-shard-sjh-0.redis-cluster-shard-sjh-headless.default.svc 6379
+ local cluster_node=redis-cluster-shard-sjh-0.redis-cluster-shard-sjh-headless.default.svc
+ local cluster_node_port=6379
+ '[' -z O3605v7HsS ']'
++ redis-cli -h redis-cluster-shard-sjh-0.redis-cluster-shard-sjh-headless.default.svc -p 6379 -a O3605v7HsS cluster nodes
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
+ cluster_nodes_info='42357755c4a03bb37f759348dfdd132738b6a478 10.128.0.28:32745@32317,redis-cluster-shard-wpv-0.redis-cluster-shard-wpv-headless.default.svc master - 0 1713520896949 6 connected 0-1364 5461-6826 10923-12287 13654-15018
bf35cb9cdaa340a18f19fa2fc420370b1827ad07 10.128.0.28:30301@32062,redis-cluster-shard-sjh-0.redis-cluster-shard-sjh-headless.default.svc myself,master - 0 1713520897000 5 connected 1365-5460 12288-13653
f5cce503559018a68ffb8f29e2a1ad83f74e9690 10.128.0.30:31843@31771,redis-cluster-shard-sjh-1.redis-cluster-shard-sjh-headless.default.svc slave,fail bf35cb9cdaa340a18f19fa2fc420370b1827ad07 1713520867793 1713520865783 5 connected
9315a33085454e972d2dd489fffa31900d6a9f47 10.128.0.24:30628@31979,redis-cluster-shard-49r-1.redis-cluster-shard-49r-headless.default.svc master - 0 1713520897000 7 connected
88780f6583930a9b4a6f74f1d07d1187af3e56a9 10.128.0.30:30924@31891,redis-cluster-shard-49r-0.redis-cluster-shard-49r-headless.default.svc master - 0 1713520897955 7 connected 6827-10922 15019-16383
8126a4ea2434ab9326e221a38b3e16e14d0dad86 10.128.0.30:31022@31002,redis-cluster-shard-wpv-1.redis-cluster-shard-wpv-headless.default.svc slave 42357755c4a03bb37f759348dfdd132738b6a478 1713520893935 1713520891523 6 connected'
+ current_comp_primary_node=()
+ current_comp_other_nodes=()
++ echo '42357755c4a03bb37f759348dfdd132738b6a478 10.128.0.28:32745@32317,redis-cluster-shard-wpv-0.redis-cluster-shard-wpv-headless.default.svc master - 0 1713520896949 6 connected 0-1364 5461-6826 10923-12287 13654-15018
bf35cb9cdaa340a18f19fa2fc420370b1827ad07 10.128.0.28:30301@32062,redis-cluster-shard-sjh-0.redis-cluster-shard-sjh-headless.default.svc myself,master - 0 1713520897000 5 connected 1365-5460 12288-13653
f5cce503559018a68ffb8f29e2a1ad83f74e9690 10.128.0.30:31843@31771,redis-cluster-shard-sjh-1.redis-cluster-shard-sjh-headless.default.svc slave,fail bf35cb9cdaa340a18f19fa2fc420370b1827ad07 1713520867793 1713520865783 5 connected
9315a33085454e972d2dd489fffa31900d6a9f47 10.128.0.24:30628@31979,redis-cluster-shard-49r-1.redis-cluster-shard-49r-headless.default.svc master - 0 1713520897000 7 connected
88780f6583930a9b4a6f74f1d07d1187af3e56a9 10.128.0.30:30924@31891,redis-cluster-shard-49r-0.redis-cluster-shard-49r-headless.default.svc master - 0 1713520897955 7 connected 6827-10922 15019-16383
8126a4ea2434ab9326e221a38b3e16e14d0dad86 10.128.0.30:31022@31002,redis-cluster-shard-wpv-1.redis-cluster-shard-wpv-headless.default.svc slave 42357755c4a03bb37f759348dfdd132738b6a478 1713520893935 1713520891523 6 connected'
++ wc -l
+ '[' 6 -eq 1 ']'
+ declare -A advertised_ports
+ local using_advertised_ports=false
+ '[' -n redis-cluster-shard-sjh-redis-advertised-0:30301,redis-cluster-shard-sjh-redis-advertised-1:31843 ']'
+ using_advertised_ports=true
+ IFS=,
+ read -ra ADDR
+ for i in "${ADDR[@]}"
++ echo redis-cluster-shard-sjh-redis-advertised-0:30301
++ cut -d: -f2
+ port=30301
+ advertised_ports[$port]=1
+ for i in "${ADDR[@]}"
++ echo redis-cluster-shard-sjh-redis-advertised-1:31843
++ cut -d: -f2
+ port=31843
+ advertised_ports[$port]=1
+ read -r line
++ echo '42357755c4a03bb37f759348dfdd132738b6a478 10.128.0.28:32745@32317,redis-cluster-shard-wpv-0.redis-cluster-shard-wpv-headless.default.svc master - 0 1713520896949 6 connected 0-1364 5461-6826 10923-12287 13654-15018'
++ awk '{print $2}'
+ node_ip_port_fields=10.128.0.28:32745@32317,redis-cluster-shard-wpv-0.redis-cluster-shard-wpv-headless.default.svc
++ echo 10.128.0.28:32745@32317,redis-cluster-shard-wpv-0.redis-cluster-shard-wpv-headless.default.svc
++ awk -F @ '{print $1}'
+ node_ip_port=10.128.0.28:32745
++ echo 10.128.0.28:32745@32317,redis-cluster-shard-wpv-0.redis-cluster-shard-wpv-headless.default.svc
++ awk -F @ '{print $2}'
++ awk -F , '{print $1}'
+ node_bus_port=32317
++ echo 10.128.0.28:32745@32317,redis-cluster-shard-wpv-0.redis-cluster-shard-wpv-headless.default.svc
++ awk -F @ '{print $1}'
++ cut -d: -f2
+ node_port=32745
++ echo '42357755c4a03bb37f759348dfdd132738b6a478 10.128.0.28:32745@32317,redis-cluster-shard-wpv-0.redis-cluster-shard-wpv-headless.default.svc master - 0 1713520896949 6 connected 0-1364 5461-6826 10923-12287 13654-15018'
++ awk '{print $2}'
++ awk -F , '{print $2}'
+ node_fqdn=redis-cluster-shard-wpv-0.redis-cluster-shard-wpv-headless.default.svc
++ echo '42357755c4a03bb37f759348dfdd132738b6a478 10.128.0.28:32745@32317,redis-cluster-shard-wpv-0.redis-cluster-shard-wpv-headless.default.svc master - 0 1713520896949 6 connected 0-1364 5461-6826 10923-12287 13654-15018'
++ awk '{print $3}'
+ node_role=master
+ true
+ [[ -n '' ]]
+ read -r line
++ echo 'bf35cb9cdaa340a18f19fa2fc420370b1827ad07 10.128.0.28:30301@32062,redis-cluster-shard-sjh-0.redis-cluster-shard-sjh-headless.default.svc myself,master - 0 1713520897000 5 connected 1365-5460 12288-13653'
++ awk '{print $2}'
+ node_ip_port_fields=10.128.0.28:30301@32062,redis-cluster-shard-sjh-0.redis-cluster-shard-sjh-headless.default.svc
++ echo 10.128.0.28:30301@32062,redis-cluster-shard-sjh-0.redis-cluster-shard-sjh-headless.default.svc
++ awk -F @ '{print $1}'
+ node_ip_port=10.128.0.28:30301
++ echo 10.128.0.28:30301@32062,redis-cluster-shard-sjh-0.redis-cluster-shard-sjh-headless.default.svc
++ awk -F , '{print $1}'
++ awk -F @ '{print $2}'
+ node_bus_port=32062
++ echo 10.128.0.28:30301@32062,redis-cluster-shard-sjh-0.redis-cluster-shard-sjh-headless.default.svc
++ awk -F @ '{print $1}'
++ cut -d: -f2
+ node_port=30301
++ echo 'bf35cb9cdaa340a18f19fa2fc420370b1827ad07 10.128.0.28:30301@32062,redis-cluster-shard-sjh-0.redis-cluster-shard-sjh-headless.default.svc myself,master - 0 1713520897000 5 connected 1365-5460 12288-13653'
++ awk '{print $2}'
++ awk -F , '{print $2}'
+ node_fqdn=redis-cluster-shard-sjh-0.redis-cluster-shard-sjh-headless.default.svc
++ echo 'bf35cb9cdaa340a18f19fa2fc420370b1827ad07 10.128.0.28:30301@32062,redis-cluster-shard-sjh-0.redis-cluster-shard-sjh-headless.default.svc myself,master - 0 1713520897000 5 connected 1365-5460 12288-13653'
++ awk '{print $3}'
+ node_role=myself,master
+ true
+ [[ -n _ ]]
+ [[ myself,master =~ master ]]
+ current_comp_primary_node+=("$node_ip_port@$node_bus_port")
+ read -r line
++ echo 'f5cce503559018a68ffb8f29e2a1ad83f74e9690 10.128.0.30:31843@31771,redis-cluster-shard-sjh-1.redis-cluster-shard-sjh-headless.default.svc slave,fail bf35cb9cdaa340a18f19fa2fc420370b1827ad07 1713520867793 1713520865783 5 connected'
++ awk '{print $2}'
+ node_ip_port_fields=10.128.0.30:31843@31771,redis-cluster-shard-sjh-1.redis-cluster-shard-sjh-headless.default.svc
++ echo 10.128.0.30:31843@31771,redis-cluster-shard-sjh-1.redis-cluster-shard-sjh-headless.default.svc
++ awk -F @ '{print $1}'
+ node_ip_port=10.128.0.30:31843
++ echo 10.128.0.30:31843@31771,redis-cluster-shard-sjh-1.redis-cluster-shard-sjh-headless.default.svc
++ awk -F @ '{print $2}'
++ awk -F , '{print $1}'
+ node_bus_port=31771
++ echo 10.128.0.30:31843@31771,redis-cluster-shard-sjh-1.redis-cluster-shard-sjh-headless.default.svc
++ awk -F @ '{print $1}'
++ cut -d: -f2
+ node_port=31843
++ echo 'f5cce503559018a68ffb8f29e2a1ad83f74e9690 10.128.0.30:31843@31771,redis-cluster-shard-sjh-1.redis-cluster-shard-sjh-headless.default.svc slave,fail bf35cb9cdaa340a18f19fa2fc420370b1827ad07 1713520867793 1713520865783 5 connected'
++ awk '{print $2}'
++ awk -F , '{print $2}'
+ node_fqdn=redis-cluster-shard-sjh-1.redis-cluster-shard-sjh-headless.default.svc
++ echo 'f5cce503559018a68ffb8f29e2a1ad83f74e9690 10.128.0.30:31843@31771,redis-cluster-shard-sjh-1.redis-cluster-shard-sjh-headless.default.svc slave,fail bf35cb9cdaa340a18f19fa2fc420370b1827ad07 1713520867793 1713520865783 5 connected'
++ awk '{print $3}'
+ node_role=slave,fail
+ true
+ [[ -n _ ]]
+ [[ slave,fail =~ master ]]
+ current_comp_other_nodes+=("$node_ip_port@$node_bus_port")
+ read -r line
++ echo '9315a33085454e972d2dd489fffa31900d6a9f47 10.128.0.24:30628@31979,redis-cluster-shard-49r-1.redis-cluster-shard-49r-headless.default.svc master - 0 1713520897000 7 connected'
++ awk '{print $2}'
+ node_ip_port_fields=10.128.0.24:30628@31979,redis-cluster-shard-49r-1.redis-cluster-shard-49r-headless.default.svc
++ echo 10.128.0.24:30628@31979,redis-cluster-shard-49r-1.redis-cluster-shard-49r-headless.default.svc
++ awk -F @ '{print $1}'
+ node_ip_port=10.128.0.24:30628
++ echo 10.128.0.24:30628@31979,redis-cluster-shard-49r-1.redis-cluster-shard-49r-headless.default.svc
++ awk -F @ '{print $2}'
++ awk -F , '{print $1}'
+ node_bus_port=31979
++ echo 10.128.0.24:30628@31979,redis-cluster-shard-49r-1.redis-cluster-shard-49r-headless.default.svc
++ awk -F @ '{print $1}'
++ cut -d: -f2
+ node_port=30628
++ echo '9315a33085454e972d2dd489fffa31900d6a9f47 10.128.0.24:30628@31979,redis-cluster-shard-49r-1.redis-cluster-shard-49r-headless.default.svc master - 0 1713520897000 7 connected'
++ awk '{print $2}'
++ awk -F , '{print $2}'
+ node_fqdn=redis-cluster-shard-49r-1.redis-cluster-shard-49r-headless.default.svc
++ echo '9315a33085454e972d2dd489fffa31900d6a9f47 10.128.0.24:30628@31979,redis-cluster-shard-49r-1.redis-cluster-shard-49r-headless.default.svc master - 0 1713520897000 7 connected'
++ awk '{print $3}'
+ node_role=master
+ true
+ [[ -n '' ]]
+ read -r line
++ echo '88780f6583930a9b4a6f74f1d07d1187af3e56a9 10.128.0.30:30924@31891,redis-cluster-shard-49r-0.redis-cluster-shard-49r-headless.default.svc master - 0 1713520897955 7 connected 6827-10922 15019-16383'
++ awk '{print $2}'
+ node_ip_port_fields=10.128.0.30:30924@31891,redis-cluster-shard-49r-0.redis-cluster-shard-49r-headless.default.svc
++ echo 10.128.0.30:30924@31891,redis-cluster-shard-49r-0.redis-cluster-shard-49r-headless.default.svc
++ awk -F @ '{print $1}'
+ node_ip_port=10.128.0.30:30924
++ echo 10.128.0.30:30924@31891,redis-cluster-shard-49r-0.redis-cluster-shard-49r-headless.default.svc
++ awk -F @ '{print $2}'
++ awk -F , '{print $1}'
+ node_bus_port=31891
++ echo 10.128.0.30:30924@31891,redis-cluster-shard-49r-0.redis-cluster-shard-49r-headless.default.svc
++ awk -F @ '{print $1}'
++ cut -d: -f2
+ node_port=30924
++ echo '88780f6583930a9b4a6f74f1d07d1187af3e56a9 10.128.0.30:30924@31891,redis-cluster-shard-49r-0.redis-cluster-shard-49r-headless.default.svc master - 0 1713520897955 7 connected 6827-10922 15019-16383'
++ awk '{print $2}'
++ awk -F , '{print $2}'
+ node_fqdn=redis-cluster-shard-49r-0.redis-cluster-shard-49r-headless.default.svc
++ echo '88780f6583930a9b4a6f74f1d07d1187af3e56a9 10.128.0.30:30924@31891,redis-cluster-shard-49r-0.redis-cluster-shard-49r-headless.default.svc master - 0 1713520897955 7 connected 6827-10922 15019-16383'
++ awk '{print $3}'
+ node_role=master
+ true
+ [[ -n '' ]]
+ read -r line
++ echo '8126a4ea2434ab9326e221a38b3e16e14d0dad86 10.128.0.30:31022@31002,redis-cluster-shard-wpv-1.redis-cluster-shard-wpv-headless.default.svc slave 42357755c4a03bb37f759348dfdd132738b6a478 1713520893935 1713520891523 6 connected'
++ awk '{print $2}'
+ node_ip_port_fields=10.128.0.30:31022@31002,redis-cluster-shard-wpv-1.redis-cluster-shard-wpv-headless.default.svc
++ echo 10.128.0.30:31022@31002,redis-cluster-shard-wpv-1.redis-cluster-shard-wpv-headless.default.svc
++ awk -F @ '{print $1}'
+ node_ip_port=10.128.0.30:31022
++ echo 10.128.0.30:31022@31002,redis-cluster-shard-wpv-1.redis-cluster-shard-wpv-headless.default.svc
++ awk -F @ '{print $2}'
++ awk -F , '{print $1}'
+ node_bus_port=31002
++ echo 10.128.0.30:31022@31002,redis-cluster-shard-wpv-1.redis-cluster-shard-wpv-headless.default.svc
++ awk -F @ '{print $1}'
++ cut -d: -f2
+ node_port=31022
++ echo '8126a4ea2434ab9326e221a38b3e16e14d0dad86 10.128.0.30:31022@31002,redis-cluster-shard-wpv-1.redis-cluster-shard-wpv-headless.default.svc slave 42357755c4a03bb37f759348dfdd132738b6a478 1713520893935 1713520891523 6 connected'
++ awk '{print $2}'
++ awk -F , '{print $2}'
+ node_fqdn=redis-cluster-shard-wpv-1.redis-cluster-shard-wpv-headless.default.svc
++ echo '8126a4ea2434ab9326e221a38b3e16e14d0dad86 10.128.0.30:31022@31002,redis-cluster-shard-wpv-1.redis-cluster-shard-wpv-headless.default.svc slave 42357755c4a03bb37f759348dfdd132738b6a478 1713520893935 1713520891523 6 connected'
++ awk '{print $3}'
+ node_role=slave
+ true
+ [[ -n '' ]]
+ read -r line
+ echo 'current_comp_primary_node: 10.128.0.28:30301@32062'
current_comp_primary_node: 10.128.0.28:30301@32062
current_comp_other_nodes: 10.128.0.30:31843@31771
+ echo 'current_comp_other_nodes: 10.128.0.30:31843@31771'
+ '[' 1 -eq 0 ']'
+ primary_node_info=10.128.0.28:30301@32062
++ echo 10.128.0.28:30301@32062
++ awk -F @ '{print $1}'
+ primary_node_endpoint_with_port=10.128.0.28:30301
++ echo 10.128.0.28:30301@32062
++ awk -F @ '{print $1}'
++ awk -F : '{print $1}'
+ primary_node_endpoint=10.128.0.28
++ echo 10.128.0.28:30301@32062
++ awk -F @ '{print $1}'
++ awk -F : '{print $2}'
+ primary_node_port=30301
++ echo 10.128.0.28:30301@32062
++ awk -F @ '{print $2}'
+ primary_node_bus_port=32062
+ is_node_in_cluster 10.128.0.28 30301 redis-cluster-shard-sjh-1
+ local random_node_endpoint=10.128.0.28
+ local random_node_port=30301
+ local node_name=redis-cluster-shard-sjh-1
+ '[' -z O3605v7HsS ']'
++ redis-cli -h 10.128.0.28 -p 30301 -a O3605v7HsS cluster nodes
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
+ cluster_nodes_info='42357755c4a03bb37f759348dfdd132738b6a478 10.128.0.28:32745@32317,redis-cluster-shard-wpv-0.redis-cluster-shard-wpv-headless.default.svc master - 0 1713520896949 6 connected 0-1364 5461-6826 10923-12287 13654-15018
bf35cb9cdaa340a18f19fa2fc420370b1827ad07 10.128.0.28:30301@32062,redis-cluster-shard-sjh-0.redis-cluster-shard-sjh-headless.default.svc myself,master - 0 1713520897000 5 connected 1365-5460 12288-13653
f5cce503559018a68ffb8f29e2a1ad83f74e9690 10.128.0.30:31843@31771,redis-cluster-shard-sjh-1.redis-cluster-shard-sjh-headless.default.svc slave,fail bf35cb9cdaa340a18f19fa2fc420370b1827ad07 1713520867793 1713520865783 5 connected
9315a33085454e972d2dd489fffa31900d6a9f47 10.128.0.24:30628@31979,redis-cluster-shard-49r-1.redis-cluster-shard-49r-headless.default.svc master - 0 1713520897000 7 connected
88780f6583930a9b4a6f74f1d07d1187af3e56a9 10.128.0.30:30924@31891,redis-cluster-shard-49r-0.redis-cluster-shard-49r-headless.default.svc master - 0 1713520897955 7 connected 6827-10922 15019-16383
8126a4ea2434ab9326e221a38b3e16e14d0dad86 10.128.0.30:31022@31002,redis-cluster-shard-wpv-1.redis-cluster-shard-wpv-headless.default.svc slave 42357755c4a03bb37f759348dfdd132738b6a478 1713520893935 1713520891523 6 connected'
++ echo '42357755c4a03bb37f759348dfdd132738b6a478 10.128.0.28:32745@32317,redis-cluster-shard-wpv-0.redis-cluster-shard-wpv-headless.default.svc master - 0 1713520896949 6 connected 0-1364 5461-6826 10923-12287 13654-15018
bf35cb9cdaa340a18f19fa2fc420370b1827ad07 10.128.0.28:30301@32062,redis-cluster-shard-sjh-0.redis-cluster-shard-sjh-headless.default.svc myself,master - 0 1713520897000 5 connected 1365-5460 12288-13653
f5cce503559018a68ffb8f29e2a1ad83f74e9690 10.128.0.30:31843@31771,redis-cluster-shard-sjh-1.redis-cluster-shard-sjh-headless.default.svc slave,fail bf35cb9cdaa340a18f19fa2fc420370b1827ad07 1713520867793 1713520865783 5 connected
9315a33085454e972d2dd489fffa31900d6a9f47 10.128.0.24:30628@31979,redis-cluster-shard-49r-1.redis-cluster-shard-49r-headless.default.svc master - 0 1713520897000 7 connected
88780f6583930a9b4a6f74f1d07d1187af3e56a9 10.128.0.30:30924@31891,redis-cluster-shard-49r-0.redis-cluster-shard-49r-headless.default.svc master - 0 1713520897955 7 connected 6827-10922 15019-16383
8126a4ea2434ab9326e221a38b3e16e14d0dad86 10.128.0.30:31022@31002,redis-cluster-shard-wpv-1.redis-cluster-shard-wpv-headless.default.svc slave 42357755c4a03bb37f759348dfdd132738b6a478 1713520893935 1713520891523 6 connected'
++ wc -l
+ '[' 6 -gt 1 ']'
+ echo '42357755c4a03bb37f759348dfdd132738b6a478 10.128.0.28:32745@32317,redis-cluster-shard-wpv-0.redis-cluster-shard-wpv-headless.default.svc master - 0 1713520896949 6 connected 0-1364 5461-6826 10923-12287 13654-15018
bf35cb9cdaa340a18f19fa2fc420370b1827ad07 10.128.0.28:30301@32062,redis-cluster-shard-sjh-0.redis-cluster-shard-sjh-headless.default.svc myself,master - 0 1713520897000 5 connected 1365-5460 12288-13653
f5cce503559018a68ffb8f29e2a1ad83f74e9690 10.128.0.30:31843@31771,redis-cluster-shard-sjh-1.redis-cluster-shard-sjh-headless.default.svc slave,fail bf35cb9cdaa340a18f19fa2fc420370b1827ad07 1713520867793 1713520865783 5 connected
9315a33085454e972d2dd489fffa31900d6a9f47 10.128.0.24:30628@31979,redis-cluster-shard-49r-1.redis-cluster-shard-49r-headless.default.svc master - 0 1713520897000 7 connected
88780f6583930a9b4a6f74f1d07d1187af3e56a9 10.128.0.30:30924@31891,redis-cluster-shard-49r-0.redis-cluster-shard-49r-headless.default.svc master - 0 1713520897955 7 connected 6827-10922 15019-16383
8126a4ea2434ab9326e221a38b3e16e14d0dad86 10.128.0.30:31022@31002,redis-cluster-shard-wpv-1.redis-cluster-shard-wpv-headless.default.svc slave 42357755c4a03bb37f759348dfdd132738b6a478 1713520893935 1713520891523 6 connected'
+ grep -q redis-cluster-shard-sjh-1
Node redis-cluster-shard-sjh-1 is already in the cluster, skipping...
+ true
+ echo 'Node redis-cluster-shard-sjh-1 is already in the cluster, skipping...'
+ exit 0

Apr 19 '24 10:04 ahjing99

cannot recreate on 0.9.0-beta.17

May 06 '24 09:05 ahjing99

This issue has been marked as stale because it has been open for 30 days with no activity

Jun 10 '24 00:06 github-actions[bot]