Envoy Ratelimt: error loading new configuration from runtime

[murtuza-kolasavala]

I am facing issue setting up rate limiting service in kubernetes cluster. I have prepared deployment file which creates rate limit deployment, service and config map. Pod is coming up but when i try curl localhost:6070/rlconfig I am not getting any output.

There is one error message on output logs but not sure if it is relevant.

level=error msg="error loading new configuration from runtime: rl.yaml: duplicate domain 'rl' in config file"

Pod debug logs

time="2022-10-12T09:30:11Z" level=warning msg="statsd is not in use"
time="2022-10-12T09:30:11Z" level=info msg="Tracing disabled"
time="2022-10-12T09:30:11Z" level=warning msg="runtime: error reading /data/ratelimit/config/..data: read /data/ratelimit/config/..data: is a directory"
time="2022-10-12T09:30:11Z" level=warning msg="connecting to redis on redis.envoy.svc.cluster.local:6379 with pool size 10"
time="2022-10-12T09:30:11Z" level=debug msg="Implicit pipelining enabled: false"
time="2022-10-12T09:30:11Z" level=debug msg="loading domain: rl"
time="2022-10-12T09:30:11Z" level=debug msg="loading descriptor: key=rl.generic_key_instance_1"
time="2022-10-12T09:30:11Z" level=debug msg="Creating stats for key: 'rl.generic_key_instance_1.header_match_get_request'"
time="2022-10-12T09:30:11Z" level=debug msg="loading descriptor: key=rl.generic_key_instance_1.header_match_get_request ratelimit={requests_per_unit=5, unit=MINUTE, unlimited=false, shadow_mode=false}"
time="2022-10-12T09:30:11Z" level=error msg="error loading new configuration from runtime: rl.yaml: duplicate domain 'rl' in config file"
time="2022-10-12T09:30:11Z" level=warning msg="Listening for debug on '0.0.0.0:6070'"
time="2022-10-12T09:30:11Z" level=warning msg="Listening for HTTP on '0.0.0.0:8080'"
time="2022-10-12T09:30:11Z" level=warning msg="Listening for gRPC on '0.0.0.0:8081'"
time="2022-10-12T09:30:11Z" level=debug msg="waiting for runtime update"
time="2022-10-12T09:30:21Z" level=debug msg="[gostats] flushing counter ratelimit.go.frees: 10241"
time="2022-10-12T09:30:21Z" level=debug msg="[gostats] flushing counter ratelimit.go.numGC: 1"
time="2022-10-12T09:30:21Z" level=debug msg="[gostats] flushing counter ratelimit.redis_pool.cx_total: 17"
time="2022-10-12T09:30:21Z" level=debug msg="[gostats] flushing counter ratelimit.redis_pool.cx_local_close: 10"
time="2022-10-12T09:30:21Z" level=debug msg="[gostats] flushing counter ratelimit.go.pauseTotalNs: 127442"
time="2022-10-12T09:30:21Z" level=debug msg="[gostats] flushing counter runtime.load_attempts: 1"
time="2022-10-12T09:30:21Z" level=debug msg="[gostats] flushing counter ratelimit.go.mallocs: 20574"
time="2022-10-12T09:30:21Z" level=debug msg="[gostats] flushing counter ratelimit.go.totalAlloc: 3714120"
time="2022-10-12T09:30:21Z" level=debug msg="[gostats] flushing counter ratelimit.service.config_load_error: 1"
time="2022-10-12T09:30:21Z" level=debug msg="[gostats] flushing counter runtime.load_failures: 1"
time="2022-10-12T09:30:21Z" level=debug msg="[gostats] flushing gauge ratelimit.redis_pool.cx_active: 7"
time="2022-10-12T09:30:21Z" level=debug msg="[gostats] flushing gauge ratelimit.go.heapIdle: 3596288"
time="2022-10-12T09:30:21Z" level=debug msg="[gostats] flushing gauge ratelimit.go.heapObjects: 10333"
time="2022-10-12T09:30:21Z" level=debug msg="[gostats] flushing gauge ratelimit.go.numGoroutine: 15"
time="2022-10-12T09:30:21Z" level=debug msg="[gostats] flushing gauge ratelimit.go.nextGC: 4194304"
time="2022-10-12T09:30:21Z" level=debug msg="[gostats] flushing gauge ratelimit.go.lastGC: 1665567011065341948"
time="2022-10-12T09:30:21Z" level=debug msg="[gostats] flushing gauge ratelimit.go.gcCPUPercent: 1"
time="2022-10-12T09:30:21Z" level=debug msg="[gostats] flushing gauge ratelimit.go.sys: 0"
time="2022-10-12T09:30:21Z" level=debug msg="[gostats] flushing gauge ratelimit.go.heapAlloc: 2078496"
time="2022-10-12T09:30:21Z" level=debug msg="[gostats] flushing gauge ratelimit.go.heapSys: 7241728"
time="2022-10-12T09:30:21Z" level=debug msg="[gostats] flushing gauge runtime.num_values: 2"
time="2022-10-12T09:30:21Z" level=debug msg="[gostats] flushing gauge ratelimit.go.alloc: 2078496"
time="2022-10-12T09:30:21Z" level=debug msg="[gostats] flushing gauge ratelimit.go.heapInuse: 3645440"
time="2022-10-12T09:30:21Z" level=debug msg="[gostats] flushing gauge ratelimit.go.heapReleased: 1392640"

Deployment files

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ratelimit
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ratelimit
  template:
    metadata:
      labels:
        app: ratelimit
    spec:
      containers:
      - name: ratelimit
        image: envoyproxy/ratelimit:master
        command: ["/bin/ratelimit"]
        resources: {}
        env:
        - name: USE_STATSD
          value: "false"
        - name: LOG_LEVEL
          value: "debug"
        - name: REDIS_SOCKET_TYPE
          value: "tcp"
        - name: REDIS_URL
          value: "redis.envoy.svc.cluster.local:6379"
        - name: RUNTIME_ROOT
          value: "/data"
        - name: RUNTIME_SUBDIRECTORY
          value: "ratelimit"
        - name: RUNTIME_WATCH_ROOT
          value: "false"
        ports:
        - name: service
          containerPort: 8081
        - name: other
          containerPort: 6070
        volumeMounts:
        - name: config
          mountPath: /data/ratelimit/config
      volumes:
      - name: config
        configMap:
          name: rl-conf
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: rl-conf
data:
  rl.yaml: |
    domain: rl
    descriptors:
    - key: generic_key
      value: instance_1
      descriptors:
        - key: header_match
          value: get_request
          rate_limit:
            unit: MINUTE
            requests_per_unit: 5

---
apiVersion: v1
kind: Service
metadata:
  name: ratelimit
spec:
  type: ClusterIP
  selector:
    app: ratelimit
  ports:
  - name: ratelimit
    protocol: TCP
    port: 8081
    targetPort: 8081
  - name: config
    protocol: TCP
    port: 6070
    targetPort: 6070

[m-rcl]

The configmap itself looks fine and works for me. The error suggests there is another configuration, referencing the same domain rl. Can you verify that there is no other configuration in your container's /data/ratelimit/config directory? Additionally you can try to set MERGE_DOMAIN_CONFIG=true for your ratelimit container's environment.

[murtuza-kolasavala]

@m-rcl Thanks, Issue is resolved now. Since i was adding it via configmap it was adding additional ../data directory and main config was pointing to this so it was duplicating.

[ShyunnY]

@m-rcl谢谢，问题现已解决。由于我是通过 configmap 添加它的，所以它添加了额外的 ../data 目录，并且主配置指向此目录，因此它是重复的。

@murtuza-kolasavala Hello! I also encountered this problem, can you tell me how you solved it?