[BUG] Request working natively in go-carbon fails with carbonapi

[keliss]

Describe the bug Hi. This query:

asPercent(aliasByNode(removeEmptySeries(removeAboveValue(scale(node1.node2.node3.*.node4.node5.*.node6.node7.node8.node9.node10.p95, 1000), 10000000000000)), 6), timeShift(aliasByNode(removeEmptySeries(removeAboveValue(scale(node1.node2.node3.*.node4.node5.*.node6.node7.node8.node9.node10.p95, 1000), 10000000000000)), 6), '1h'))

fails with the following error:

2021-08-24T16:01:10.044Z        ERROR   access  request failed  {"data": {"handler":"render","carbonapi_uuid":"7bd539fb-abb5-44e7-9f4b-2bdf41212bdc","url":"/render","peer_ip":"<ip>","host":"<host>:8182","format":"json","use_cache":true,"targets":["<metric above>"],"cache_timeout":60,"runtime":0.030559371,"http_code":500,"reason":"function=asPercent: found wildcard where series expected","from":1629820570,"until":1629820870,"from_raw":"-5min","until_raw":"now","uri":"/render","from_cache":false,"used_backend_cache":false,"request_headers":{}}}

CarbonAPI Version carbonapi-0.15.4~1-1

CarbonAPI Configuration:

listen: "0.0.0.0:8182"
cache:
  type: "mem"
  size_mb: 0
  defaultTimeoutSec: 60
cpus: 0
tz: ""
graphite:
  host: "localhost:2003"
  interval: "60s"
  prefix: "carbon.api"
  pattern: "{prefix}.{fqdn}"
pidFile: ""
logger:
  - logger: ""
    file: "/var/log/carbonapi/carbonapi-1.log"
    level: "warn"
    encoding: "console"
upstreams:
  buckets: 10
  slowLogThreshold: 7s
  timeouts:
    find: "10s"
    render: "30s"
    connect: "200ms"
  keepAliveInterval: "30s"
  maxIdleConnsPerHost: 1000
  backendsv2:
    backends:
      - groupName: "victoriametrics"
        protocol: "victoriametrics"
        lbMethod: "rr"
        maxTries: 2
        backendOptions:
          step: "1m"
        servers:
          - "http://storage:8428"

Simplified query (if applicable) Minimal viable query to reproduce the issue:

asPercent(node1.node2.node3.*.node4.node5.*.node6.node7.node8.node9.node10.p95, timeShift(node1.node2.node3.*.node4.node5.*.node6.node7.node8.node9.node10.p95, '1h'))

The same query works on the vanilla go-carbon backend.

[Civil]

Key might be what diffenrece between replies from go-carbon and VM for both node1.node2.node3.*.node4.node5.*.node6.node7.node8.node9.node10.p95 and timeShift(node1.node2.node3.*.node4.node5.*.node6.node7.node8.node9.node10.p95, '1h') (names would matter)

[Civil]

https://github.com/go-graphite/carbonapi/blob/fab6ee9ed2a8bd5e899f87af148f07a68d3a37db/expr/functions/asPercent/function.go#L85-L93

That would be the place where error would be triggered.

Unfortunately error doesn't tell what exactly is the problem here, but I suspect that amount of replies in timeShift and non-timeShit is different in case of VM.

So the error that is returned is wrong and I should definitely fix it.

[keliss]

Ok, thanks for the quick response. I'll try to find out where the difference is coming from and will be looking forward to the fix you mentioned.

[Civil]

Since the issue was opened there were numerous changes in functions, I expect that this issue is now fixed. If not - please reopen with more details on how to reproduce.