chproxy icon indicating copy to clipboard operation
chproxy copied to clipboard

High concurrency set session_id invalid

Open ddddddcf opened this issue 3 years ago • 3 comments

When I start multiple threads to send sql queries, queries with the same session_id are forwarded to different server,but I want let then to then same server log be like:

DEBUG: 2022/04/27 10:11:34 proxy.go:119: [ Id: 16E90FC31005B38F; User "aaa"(4) proxying as "default"(4) to "host1:8123"(3); *** Duration: 5324577 μs]: request success; query: "CREATE TABLE IF NOT EXISTS default.a ENGINE MergeTree() ORDER BY (tuple()) SELECT 1 AS id"; Method: POST; URL: "http://host1:8123/?&query=CREATE TABLE IF NOT EXISTS default.a ENGINE MergeTree() ORDER BY (tuple()) SELECT 1 AS id&query_id=16E90FC31005B38F&session_id=d86d6cfc-3052-43e1-a2a8-70da47f52364&session_timeout=60"
DEBUG: 2022/04/27 10:11:41 proxy.go:121: [ Id: 16E90FC31005B399; User "aaa"(5) proxying as "default"(5) to "host2:8123"(2); *** Duration: 685558 μs]: request failure: non-200 status code 404; query: "SELECT * FROM default.a"; Method: POST; URL: "http://host2:8123/?session_id=d86d6cfc-3052-43e1-a2a8-70da47f52364&query=SELECT * FROM default.a&query_id=16E90FC31005B399&session_id=d86d6cfc-3052-43e1-a2a8-70da47f52364&session_timeout=60"

And config is:

log_debug: false
hack_me_please: false
caches:
  - name: "longterm"
    mode: "file_system"
    file_system:
      dir: "/var/lib/chproxy"
      max_size: 10Gb
    expire: 2h
    grace_time: 100s

network_groups:
  - name: "test"
    networks: ["host"]

param_groups:
  ***
server:
  http:
    listen_addr: ":9090"
    allowed_networks: ["host"]
    read_timeout: 5m
    write_timeout: 10m
    idle_timeout: 15m

users:
  - name: "aaa"
    password: "aaa"
    to_cluster: "cluster"
    to_user: "default"
    deny_http: false
    allow_cors: true
    max_queue_size: 100
    max_queue_time: 60s
    max_concurrent_queries: 5
    max_execution_time: 60s
clusters:
  - name: "cluster"
    scheme: "http"

    nodes: ["host1:8123", "host2:8123"]
    heartbeat:
      interval: 15s
      timeout: 5s
      request: "/?query=SELECT%201%2B1"
      response: "2\n"

    kill_query_user:
      name: "default"
      password: ""
      
    users:
      - name: "default"
        password: ""
        max_queue_size: 100
        max_queue_time: 60s
        max_concurrent_queries: 5
        max_execution_time: 60s

ddddddcf avatar Apr 27 '22 10:04 ddddddcf

I use 10 threads

ddddddcf avatar Apr 27 '22 10:04 ddddddcf

when I use replicasmode , all request will send the same server , but not load balance

ddddddcf avatar Apr 27 '22 11:04 ddddddcf

There must be some bug because when I briefly look at the code, it seems to handle the stickiness based on sessionID https://github.com/ContentSquare/chproxy/blob/bb49c1674a85189b461e72568cc627c8a8e44cc3/scope.go#L59

If you have time to dig and fix the bug, feel free to do it. Otherwise we will do it when we have time (but we have other priority for the moment)

mga-chka avatar Apr 28 '22 13:04 mga-chka

closing this issue since it was fixed by https://github.com/ContentSquare/chproxy/pull/173

mga-chka avatar Sep 02 '22 09:09 mga-chka