apisix icon indicating copy to clipboard operation
apisix copied to clipboard
verified publisher icon apache

help request: router dispatch not working as expected

Open 1renwuruiming opened this issue 3 years ago • 39 comments

Description

nginx存在多种路由的匹配规则，apisix是否存在同样的方式，如果存在，是如何使用的呢

Environment

  • APISIX version (run apisix version):
  • Operating system (run uname -a):
  • OpenResty / Nginx version (run openresty -V or nginx -V):
  • etcd version, if relevant (run curl http://127.0.0.1:9090/v1/server_info):
  • APISIX Dashboard version, if relevant:
  • Plugin runner version, for issues related to plugin runners:
  • LuaRocks version, for installation issues (run luarocks --version):

1renwuruiming avatar Jul 13 '22 02:07 1renwuruiming

@juzhiyuan 我发送了一些邮件到您的邮箱，并提供了一些日志截图，想要咨询一下apisix对于复杂路由的匹配规则是怎样的，是否和nginx一样，又是如何使用的呢

1renwuruiming avatar Jul 13 '22 02:07 1renwuruiming

Hi, please read the following documents:

  1. https://apisix.apache.org/docs/apisix/terminology/router/#configuration
  2. https://apisix.apache.org/docs/apisix/router-radixtree/
  3. https://github.com/api7/lua-resty-radixtree

juzhiyuan avatar Jul 13 '22 02:07 juzhiyuan

Also, could you please paste the screenshot and descriptions into this issue? Then more team members could check. 🙏

NOTE: Don't forget to erase sensitive content.

juzhiyuan avatar Jul 13 '22 03:07 juzhiyuan

我降低版本到2.13.1就没有这样的问题了，所以是否是最近几个版本更改了路由的匹配规则导致的呢？看到其他的issue中也有提到这样的现象

1renwuruiming avatar Jul 15 '22 02:07 1renwuruiming

看到其他的issue中也有提到这样的现象

Could you please paste those issues here? Then I will cc maintainers to check the difference.

juzhiyuan avatar Jul 15 '22 03:07 juzhiyuan

Please provide the minimum reproducible steps

tzssangglass avatar Jul 15 '22 03:07 tzssangglass

开会的时候描述过过程了，我不知道还可以怎么描述，是否检查一下版本更新的diff内容更好呢？对比一下2.13.1和之后的版本的路由这边的代码差别，现象是如果只有一个路由，不会出现404的情况，如果出现多个路由，就会无法路由到正确的环境地址

1renwuruiming avatar Jul 15 '22 03:07 1renwuruiming

开会的时候描述过过程了，我不知道还可以怎么描述，是否检查一下版本更新的diff内容更好呢？对比一下2.13.1和之后的版本的路由这边的代码差别，现象是如果只有一个路由，不会出现404的情况，如果出现多个路由，就会无法路由到正确的环境地址

ok, just give me a route for the problem, I will try it on 2.13.1 and the latest version.

tzssangglass avatar Jul 15 '22 06:07 tzssangglass

我认为这是一个普遍的问题，当你拥有多个请求前缀相似的路由的情况下，新版本的apisix不能很好的转发，你可以模拟这样的场景来进行验证

1renwuruiming avatar Jul 15 '22 06:07 1renwuruiming

我认为这是一个普遍的问题，当你拥有多个请求前缀相似的路由的情况下，新版本的apisix不能很好的转发，你可以模拟这样的场景来进行验证

we have some test cases to cover, see: https://github.com/apache/apisix/blob/master/t/router/radixtree-uri-sanity.t#L256-L306

So I need you to give the exact replication scenario, otherwise I won't verify it blindly.

tzssangglass avatar Jul 15 '22 06:07 tzssangglass

Also, could you please paste the screenshot and descriptions into this issue? Then more team members could check. 🙏

NOTE: Don't forget to erase sensitive content.

@1renwuruiming Again, because your screenshot has some sensitive information, so I can't paste them here instead of you directly.

Also, all people here are volunteers who love this community, and if you can't provide enough information, it's hard to help you resolve this issue.

juzhiyuan avatar Jul 15 '22 06:07 juzhiyuan

@juzhiyuan 我需要将后台服务的地址提供给您吗？

1renwuruiming avatar Jul 15 '22 06:07 1renwuruiming

我以什么样的方式来提供route比较合适呢？

1renwuruiming avatar Jul 15 '22 06:07 1renwuruiming

我以什么样的方式来提供route比较合适呢？

The form of route stored in etcd, text form, desensitized.

And the verification process.

tzssangglass avatar Jul 15 '22 09:07 tzssangglass

好的，我整理一下给您

1renwuruiming avatar Jul 15 '22 10:07 1renwuruiming

@tzssangglass 如何从etcd导出数据呢？我尝试了多种方式仍旧没有成功导出

1renwuruiming avatar Jul 15 '22 11:07 1renwuruiming

@tzssangglass 如何从etcd导出数据呢？我尝试了多种方式仍旧没有成功导出

you can use etcdctl, e.g. etcdctl get /apisix/routes/1 or etcdctl get / --prefix

tzssangglass avatar Jul 15 '22 17:07 tzssangglass

[{ "key": "/apisix/consumers/", "value": "init_dir" }, { "key": "/apisix/global_rules/", "value": "init_dir" }, { "key": "/apisix/global_rules/1", "value": "{"id":"1","create_time":1656912914,"update_time":1657849534,"plugins":{"api-breaker":{"break_response_code":507,"disable":false,"healthy":{"http_statuses":[200],"successes":2},"max_breaker_sec":180,"unhealthy":{"failures":1,"http_statuses":[503,500]}},"limit-req":{"burst":2,"disable":false,"key":"$x_forward_for $http_user_agent $uri $upstream_uri","key_type":"var_combination","nodelay":true,"rate":3,"rejected_code":503,"rejected_msg":"操作太快了，请稍后重试"}}}" }, { "key": "/apisix/plugin_configs/", "value": "init_dir" }, { "key": "/apisix/plugin_metadata/", "value": "init_dir" }, { "key": "/apisix/plugin_metadata/clickhouse-logger", "value": "{"log_format":{"body_bytes_sent":"$body_bytes_sent","route_name":"$route_name","time_local":"$time_local","http_host":"$http_host","time":"$time_iso8601","upstream_uri":"$upstream_uri","request_query":"$args","request_body":"$request_body","method":"$request_method","upstream_addr":"$upstream_addr","request_proto":"$server_protocol","http_referrer":"$name","upstream_host":"$upstream_host","x_forward_for":"$proxy_add_x_forwarded_for","unix_time":"$msec","http_user_agent":"$http_user_agent","upstream_scheme":"$upstream_scheme","request":"$request","request_time":"$request_time","upstream_status":"$upstream_status","vhost":"$host","remote_addr":"$proxy_protocol_addr","path":"$uri","remote_user":"$remote_user"}}" }, { "key": "/apisix/plugins", "value": "[{"name":"api-breaker"},{"name":"authz-casbin"},{"name":"authz-casdoor"},{"name":"authz-keycloak"},{"name":"aws-lambda"},{"name":"azure-functions"},{"name":"basic-auth"},{"name":"batch-requests"},{"name":"clickhouse-logger"},{"name":"client-control"},{"name":"consumer-restriction"},{"name":"cors"},{"name":"csrf"},{"name":"datadog"},{"name":"dubbo-proxy"},{"name":"echo"},{"name":"error-log-logger"},{"name":"example-plugin"},{"name":"ext-plugin-post-req"},{"name":"ext-plugin-post-resp"},{"name":"ext-plugin-pre-req"},{"name":"fault-injection"},{"name":"file-logger"},{"name":"forward-auth"},{"name":"google-cloud-logging"},{"name":"grpc-transcode"},{"name":"grpc-web"},{"name":"gzip"},{"name":"hmac-auth"},{"name":"http-logger"},{"name":"ip-restriction"},{"name":"jwt-auth"},{"name":"kafka-logger"},{"name":"kafka-proxy"},{"name":"key-auth"},{"name":"ldap-auth"},{"name":"limit-conn"},{"name":"limit-count"},{"name":"limit-req"},{"name":"log-rotate"},{"name":"loggly"},{"name":"mocking"},{"name":"node-status"},{"name":"opa"},{"name":"openid-connect"},{"name":"opentelemetry"},{"name":"openwhisk"},{"name":"prometheus"},{"name":"proxy-control"},{"name":"proxy-mirror"},{"name":"proxy-rewrite"},{"name":"public-api"},{"name":"real-ip"},{"name":"redirect"},{"name":"referer-restriction"},{"name":"request-id"},{"name":"request-validation"},{"name":"response-rewrite"},{"name":"rocketmq-logger"},{"name":"server-info"},{"name":"serverless-post-function"},{"name":"serverless-pre-function"},{"name":"sls-logger"},{"name":"splunk-hec-logging"},{"name":"syslog"},{"name":"tcp-logger"},{"name":"traffic-split"},{"name":"ua-restriction"},{"name":"udp-logger"},{"name":"uri-blocker"},{"name":"wolf-rbac"},{"name":"zipkin"},{"name":"jwt-for-1ren"},{"stream":true,"name":"ip-restriction"},{"stream":true,"name":"limit-conn"},{"stream":true,"name":"mqtt-proxy"},{"stream":true,"name":"syslog"}]" }, { "key": "/apisix/plugins/", "value": "init_dir" }, { "key": "/apisix/proto/", "value": "init_dir" }, { "key": "/apisix/routes/", "value": "init_dir" }, { "key": "/apisix/routes/414615654771983046", "value": "{"id":"414615654771983046","create_time":1656659788,"update_time":1657688527,"uri":"/escort/collect/ws","name":"dev_escort_collect_ws","desc":"护驾测试环境采集模块长链接","methods":["GET","POST","PUT","DELETE","PATCH","HEAD","OPTIONS","CONNECT","TRACE"],"host":"mydomain.com","plugins":{"clickhouse-logger":{"database":"default","endpoint_addr":"http://172.16.5.189:8133","logtable":"escort_logger","password":"pd4lJf0Eha1OUPzWUJ","user":"default"},"proxy-rewrite":{"host":"172.16.5.187"}},"service_id":"414615505169547974","labels":{"API_VERSION":"v1.0.0","project":"escort-test"},"enable_websocket":true,"status":1}" }, { "key": "/apisix/routes/414622106198737606", "value": "{"id":"414622106198737606","create_time":1656663633,"update_time":1657256578,"uris":["/dashboard/","/apisix/"],"name":"dev_apisix_dashboard_h5","desc":"dashboard","methods":["GET","POST","PUT","DELETE","PATCH","HEAD","OPTIONS","CONNECT","TRACE"],"upstream":{"nodes":{"172.16.5.189:9001":1},"timeout":{"connect":6,"send":6,"read":6},"type":"roundrobin","scheme":"http","pass_host":"pass","keepalive_pool":{"idle_timeout":60,"requests":1000,"size":320}},"labels":{"API_VERSION":"v1.0.0","project":"apisix"},"status":1}" }, { "key": "/apisix/routes/414628375827579590", "value": "{"id":"414628375827579590","create_time":1656667370,"update_time":1657688577,"uri":"/escort/collect/","name":"dev_escort_collect","desc":"护驾测试环境采集模块接口","methods":["GET","POST","PUT","DELETE","PATCH","HEAD","OPTIONS","CONNECT","TRACE"],"host":"mydomain.com","plugins":{"clickhouse-logger":{"database":"default","endpoint_addr":"http://172.16.5.189:8133","logtable":"escort_logger","password":"pd4lJf0Eha1OUPzWUJ","user":"default"},"proxy-rewrite":{"host":"172.16.5.187","regex_uri":["/escort/collect/","/"]}},"service_id":"414622816210518726","labels":{"API_VERSION":"v1.0.0","project":"escort-test"},"status":1}" }, { "key": "/apisix/routes/414728833116144326", "value": "{"id":"414728833116144326","create_time":1656727247,"update_time":1657784068,"uri":"/escortCommunity/","name":"dev_escort_community_app_interface","desc":"护驾测试环境社区模块app接口","priority":1,"methods":["GET","POST","PUT","DELETE","PATCH","HEAD","OPTIONS","CONNECT","TRACE"],"host":"mydomain.com","plugins":{"clickhouse-logger":{"database":"default","endpoint_addr":"http://172.16.5.189:8133","logtable":"escort_logger","password":"pd4lJf0Eha1OUPzWUJ","ssl_verify":false,"user":"default"},"proxy-rewrite":{"host":"172.16.5.187","regex_uri":["/escortCommunity/","/"]}},"service_id":"414728522402104006","labels":{"API_VERSION":"v1.0.0","project":"escort-test"},"status":1}" }, { "key": "/apisix/routes/414729016054907590", "value": "{"id":"414729016054907590","create_time":1656727356,"update_time":1657688651,"uri":"/escort/web/escortCommunity/","name":"dev_escort_community_web_interface","desc":"护驾测试环境社区模块web接口","methods":["GET","POST","PUT","DELETE","PATCH","HEAD","OPTIONS","CONNECT","TRACE"],"host":"mydomain.com","plugins":{"clickhouse-logger":{"database":"default","endpoint_addr":"http://172.16.5.189:8133","logtable":"escort_logger","password":"pd4lJf0Eha1OUPzWUJ","user":"default"},"proxy-rewrite":{"host":"172.16.5.187","regex_uri":["/escort/web/escortCommunity/","/"]}},"service_id":"414728672440746694","labels":{"API_VERSION":"v1.0.0","project":"escort-test"},"status":1}" }, { "key": "/apisix/routes/414729982036673222", "value": "{"id":"414729982036673222","create_time":1656727932,"update_time":1657853036,"uri":"/escort/service/","name":"dev_escort_service_interface","desc":"护驾测试环境业务模块接口","priority":1,"methods":["GET","POST","PUT","DELETE","PATCH","HEAD","OPTIONS","CONNECT","TRACE"],"host":"mydomain.com","plugins":{"clickhouse-logger":{"database":"default","endpoint_addr":"http://172.16.5.189:8133","logtable":"escort_logger","password":"pd4lJf0Eha1OUPzWUJ","user":"default"},"proxy-rewrite":{"regex_uri":["/escort/service/","/"]}},"service_id":"414729848473256646","labels":{"API_VERSION":"v1.0.0","project":"escort-test"},"status":1}" }, { "key": "/apisix/routes/414730310064800454", "value": "{"id":"414730310064800454","create_time":1656728128,"update_time":1657702182,"uri":"/escort/info/active/","name":"dev_escort_active_interface","desc":"护驾测试环境活动模块接口","methods":["GET","POST","PUT","DELETE","PATCH","HEAD","OPTIONS","CONNECT","TRACE"],"host":"mydomain.com","plugins":{"clickhouse-logger":{"database":"default","endpoint_addr":"http://172.16.5.189:8133","logtable":"escort_logger","password":"pd4lJf0Eha1OUPzWUJ","user":"default"},"proxy-rewrite":{"host":"172.16.5.187"}},"service_id":"414730110919246534","labels":{"API_VERSION":"v1.0.0","project":"escort-test"},"status":1}" }, { "key": "/apisix/routes/414731217343087302", "value": "{"id":"414731217343087302","create_time":1656728668,"update_time":1657702234,"uri":"/escort/info/energy/","name":"dev_escort_energy_interface","desc":"护驾测试环境补能模块接口","methods":["GET","POST","PUT","DELETE","PATCH","HEAD","OPTIONS","CONNECT","TRACE"],"host":"mydomain.com","plugins":{"clickhouse-logger":{"database":"default","endpoint_addr":"http://172.16.5.189:8133","logtable":"escort_logger","password":"pd4lJf0Eha1OUPzWUJ","user":"default"},"proxy-rewrite":{"host":"172.16.5.187"}},"service_id":"414731099516699334","labels":{"API_VERSION":"v1.0.0","project":"escort-test"},"status":1}" }, { "key": "/apisix/routes/414731449472647878", "value": "{"id":"414731449472647878","create_time":1656728807,"update_time":1657702014,"uri":"/escort/crm/","name":"dev_escort_crm_interface","desc":"护驾测试环境crm接口","methods":["GET","POST","PUT","DELETE","PATCH","HEAD","OPTIONS","CONNECT","TRACE"],"host":"mydomain.com","plugins":{"clickhouse-logger":{"database":"default","endpoint_addr":"http://172.16.5.189:8133","logtable":"escort_logger","password":"pd4lJf0Eha1OUPzWUJ","user":"default"},"proxy-rewrite":{"host":"172.16.5.187"}},"service_id":"414731018969285318","labels":{"API_VERSION":"v1.0.0","project":"escort-test"},"status":1}" }, { "key": "/apisix/routes/414731734316221126", "value": "{"id":"414731734316221126","create_time":1656728977,"update_time":1657870946,"uri":"/escort/web/","name":"dev_escort_service_web_interface","desc":"护驾测试环境业务模块web接口","methods":["GET","POST","PUT","DELETE","PATCH","HEAD","OPTIONS","CONNECT","TRACE"],"host":"mydomain.com","plugins":{"clickhouse-logger":{"database":"default","endpoint_addr":"http://172.16.5.189:8133","logtable":"escort_logger","password":"pd4lJf0Eha1OUPzWUJ","user":"default"},"proxy-rewrite":{"host":"172.16.5.187","regex_uri":["/escort/web/","/"]}},"service_id":"414731624811332294","labels":{"API_VERSION":"v1.0.0","project":"escort-test"},"status":1}" }, { "key": "/apisix/routes/414732255399772870", "value": "{"id":"414732255399772870","create_time":1656729287,"update_time":1657689070,"uri":"/escort/oss-sts","name":"dev_escort_ossSts","desc":"护驾测试环境OSS-STS","methods":["GET","POST","PUT","DELETE","PATCH","HEAD","OPTIONS","CONNECT","TRACE"],"host":"mydomain.com","plugins":{"proxy-rewrite":{"host":"172.16.5.174","uri":"/"}},"service_id":"414732092056797894","labels":{"API_VERSION":"v1.0.0","project":"escort-test"},"status":1}" }, { "key": "/apisix/routes/414734439306756806", "value": "{"id":"414734439306756806","create_time":1656730589,"update_time":1657869731,"uri":"/escort/h5/","name":"dev_escort_home_h5","desc":"护驾测试环境H5","priority":1,"methods":["GET","POST","PUT","DELETE","PATCH","HEAD","OPTIONS","CONNECT","TRACE"],"host":"mydomain.com","service_id":"414734269001237190","labels":{"API_VERSION":"v1.0.0","project":"escort-test"},"status":1}" }, { "key": "/apisix/routes/415016511887901382", "value": "{"id":"415016511887901382","create_time":1656898717,"update_time":1657764587,"uri":"/","name":"dev_apisix_dashboard_status","desc":"dashboard静态文件","priority":2,"methods":["GET","POST","PUT","DELETE","PATCH","HEAD","OPTIONS","CONNECT","TRACE"],"vars":[["uri","~~",".(js|css|flash|media|jpg|png|gif|ico|vbs|json|txt|svg)$"]],"upstream":{"nodes":[{"host":"172.16.5.189","port":9001,"weight":1}],"timeout":{"connect":6,"send":6,"read":6},"type":"roundrobin","scheme":"http","pass_host":"pass","keepalive_pool":{"idle_timeout":60,"requests":1000,"size":320}},"labels":{"API_VERSION":"v1.0.0","project":"apisix"},"status":1}" }, { "key": "/apisix/routes/415053802706567878", "value": "{"id":"415053802706567878","create_time":1656920944,"update_time":1657689152,"uri":"/escort/static/","name":"dev_escort_service_static","desc":"护驾测试环境H5静态资源文件\t","methods":["GET","POST","PUT","DELETE","PATCH","HEAD","OPTIONS","CONNECT","TRACE"],"host":"mydomain.com","service_id":"415053596615246534","labels":{"API_VERSION":"v1.0.0","project":"escort-test"},"status":1}" }, { "key": "/apisix/routes/415057043343278790", "value": "{"id":"415057043343278790","create_time":1656922876,"update_time":1657689196,"uri":"/escort/service/escortCommunity/","name":"dev_escort_community_share","desc":"护驾测试环境分享接口","methods":["GET","POST","PUT","DELETE","PATCH","HEAD","OPTIONS","CONNECT","TRACE"],"host":"mydomain.com","plugins":{"clickhouse-logger":{"database":"default","endpoint_addr":"http://172.16.5.189:8133","logtable":"escort_logger","password":"pd4lJf0Eha1OUPzWUJ","user":"default"},"proxy-rewrite":{"host":"172.16.5.187","regex_uri":["/escort/service/escortCommunity/","/"]}},"service_id":"414728522402104006","labels":{"API_VERSION":"v1.0.0","project":"escort-test"},"status":1}" }, { "key": "/apisix/routes/415057404321858246", "value": "{"id":"415057404321858246","create_time":1656923091,"update_time":1657764003,"uri":"/escort/admin/","name":"dev_escort_web_dashboard","desc":"护驾测试环境WEB页面","methods":["GET","POST","PUT","DELETE","PATCH","HEAD","OPTIONS","CONNECT","TRACE"],"host":"mydomain.com","service_id":"415054510755414726","labels":{"API_VERSION":"v1.0.0","project":"escort-test"},"status":1}" }, { "key": "/apisix/routes/415065285234524870", "value": "{"id":"415065285234524870","create_time":1656927788,"update_time":1657791576,"uri":"/escort/energyH5/","name":"dev_escort_energy_h5","desc":"护驾测试环境补能页面","methods":["GET","POST","PUT","DELETE","PATCH","HEAD","OPTIONS","CONNECT","TRACE"],"host":"mydomain.com","service_id":"414734269001237190","labels":{"API_VERSION":"v1.0.0","project":"escort-test"},"status":1}" }, { "key": "/apisix/routes/415173139194970822", "value": "{"id":"415173139194970822","create_time":1656992074,"update_time":1656992074,"uri":"/loki/api/v1/push","name":"dev_loki_promtail_interface","desc":"loki接收日志","methods":["GET","POST","PUT","DELETE","PATCH","HEAD","OPTIONS","CONNECT","TRACE"],"host":"pay.1rendata.com","upstream":{"nodes":{"172.16.5.189:3100":1},"retries":2,"timeout":{"connect":6,"send":6,"read":6},"type":"roundrobin","scheme":"http","pass_host":"pass","keepalive_pool":{"idle_timeout":60,"requests":1000,"size":320},"retry_timeout":2},"labels":{"API_VERSION":"v1.0.0","project":"apisix"},"status":1}" }, { "key": "/apisix/routes/416332727281779398", "value": "{"id":"416332727281779398","create_time":1657683243,"update_time":1657850105,"uri":"/grafana/","name":"grafana","methods":["GET","POST","PUT","DELETE","PATCH","HEAD","OPTIONS","CONNECT","TRACE"],"host":"mydomain.com","upstream":{"nodes":[{"host":"172.16.5.189","port":3000,"weight":1}],"timeout":{"connect":6,"send":6,"read":6},"type":"roundrobin","scheme":"http","pass_host":"pass","keepalive_pool":{"idle_timeout":60,"requests":1000,"size":320}},"status":0}" }, { "key": "/apisix/routes/416612663888446149", "value": "{"id":"416612663888446149","create_time":1657850098,"update_time":1657850122,"uri":"/grafana/","name":"dev_loki_grafana_web","desc":"grafana日志监控可视化","methods":["GET","POST","PUT","DELETE","PATCH","HEAD","OPTIONS","CONNECT","TRACE"],"host":"pay.1rendata.com","upstream":{"nodes":{"172.16.5.189:3000":1},"timeout":{"connect":6,"send":6,"read":6},"type":"roundrobin","scheme":"http","pass_host":"pass","keepalive_pool":{"idle_timeout":60,"requests":1000,"size":320}},"labels":{"API_VERSION":"v1.0.0","project":"apisix"},"status":1}" }, { "key": "/apisix/services/", "value": "init_dir" }, { "key": "/apisix/services/414615505169547974", "value": "{"id":"414615505169547974","create_time":1656659699,"update_time":1657531143,"name":"护驾测试环境采集模块长链接","desc":"护驾测试环境采集模块长链接","upstream":{"nodes":[{"host":"172.16.5.187","port":9001,"weight":10000}],"timeout":{"connect":120,"send":120,"read":120},"type":"roundrobin","scheme":"https","pass_host":"pass","keepalive_pool":{"idle_timeout":60,"requests":100000,"size":320}}}" }, { "key": "/apisix/services/414622816210518726", "value": "{"id":"414622816210518726","create_time":1656664056,"update_time":1656667945,"name":"护驾测试环境采集模块接口服务","desc":"护驾测试环境采集模块接口服务","upstream":{"nodes":[{"host":"172.16.5.187","port":20003,"weight":1}],"timeout":{"connect":6,"send":6,"read":6},"type":"roundrobin","scheme":"http","pass_host":"pass","keepalive_pool":{"idle_timeout":60,"requests":1000,"size":320}}}" }, { "key": "/apisix/services/414629460826915526", "value": "{"id":"414629460826915526","create_time":1656668017,"update_time":1656668017,"name":"护驾测试环境xxl-job","desc":"护驾测试环境xxl-job","upstream":{"nodes":{"172.16.5.187:8090":1},"timeout":{"connect":6,"send":6,"read":6},"type":"roundrobin","scheme":"http","pass_host":"pass","keepalive_pool":{"idle_timeout":60,"requests":1000,"size":320}}}" }, { "key": "/apisix/services/414728522402104006", "value": "{"id":"414728522402104006","create_time":1656727062,"update_time":1657531154,"name":"护驾测试环境社区APP接口","desc":"护驾测试环境社区APP接口","upstream":{"nodes":[{"host":"172.16.5.187","port":20006,"weight":1000000}],"timeout":{"connect":6,"send":6,"read":6},"type":"roundrobin","scheme":"http","pass_host":"pass","keepalive_pool":{"idle_timeout":60,"requests":1000,"size":320}}}" }, { "key": "/apisix/services/414728672440746694", "value": "{"id":"414728672440746694","create_time":1656727151,"update_time":1656727151,"name":"护驾测试环境社区模块web接口","desc":"护驾测试环境社区模块web接口","upstream":{"nodes":{"172.16.5.187:20005":1},"timeout":{"connect":6,"send":6,"read":6},"type":"roundrobin","scheme":"http","pass_host":"pass","keepalive_pool":{"idle_timeout":60,"requests":1000,"size":320}}}" }, { "key": "/apisix/services/414729848473256646", "value": "{"id":"414729848473256646","create_time":1656727852,"update_time":1656728371,"name":"护驾测试环境业务模块接口","desc":"护驾测试环境业务模块接口","upstream":{"nodes":[{"host":"172.16.5.187","port":19001,"weight":1}],"timeout":{"connect":6,"send":6,"read":6},"type":"roundrobin","scheme":"http","pass_host":"pass","keepalive_pool":{"idle_timeout":60,"requests":1000,"size":320}}}" }, { "key": "/apisix/services/414730110919246534", "value": "{"id":"414730110919246534","create_time":1656728009,"update_time":1656728378,"name":"护驾测试环境活动模块接口","desc":"护驾测试环境活动模块接口","upstream":{"nodes":[{"host":"172.16.5.187","port":19003,"weight":1}],"timeout":{"connect":6,"send":6,"read":6},"type":"roundrobin","scheme":"http","pass_host":"pass","keepalive_pool":{"idle_timeout":60,"requests":1000,"size":320}}}" }, { "key": "/apisix/services/414731018969285318", "value": "{"id":"414731018969285318","create_time":1656728550,"update_time":1657531177,"name":"护驾测试环境crm接口","desc":"护驾测试环境crm接口","upstream":{"nodes":[{"host":"172.16.5.187","port":19002,"weight":1000000}],"timeout":{"connect":6,"send":6,"read":6},"type":"roundrobin","scheme":"http","pass_host":"pass","keepalive_pool":{"idle_timeout":60,"requests":1000,"size":320}}}" }, { "key": "/apisix/services/414731099516699334", "value": "{"id":"414731099516699334","create_time":1656728598,"update_time":1656728598,"name":"护驾测试环境补能模块接口","desc":"护驾测试环境补能模块接口","upstream":{"nodes":{"172.16.5.187:19004":1},"timeout":{"connect":6,"send":6,"read":6},"type":"roundrobin","scheme":"http","pass_host":"pass","keepalive_pool":{"idle_timeout":60,"requests":1000,"size":320}}}" }, { "key": "/apisix/services/414731624811332294", "value": "{"id":"414731624811332294","create_time":1656728911,"update_time":1656728911,"name":"护驾测试环境业务模块web接口","desc":"护驾测试环境业务模块web接口","upstream":{"nodes":{"172.16.5.187:19005":1},"timeout":{"connect":6,"send":6,"read":6},"type":"roundrobin","scheme":"http","pass_host":"pass","keepalive_pool":{"idle_timeout":60,"requests":1000,"size":320}}}" }, { "key": "/apisix/services/414732092056797894", "value": "{"id":"414732092056797894","create_time":1656729190,"update_time":1656729190,"name":"护驾测试环境OSS-STS","desc":"护驾测试环境OSS-STS","upstream":{"nodes":{"172.16.5.174:7080":1},"timeout":{"connect":6,"send":6,"read":6},"type":"roundrobin","scheme":"http","pass_host":"pass","keepalive_pool":{"idle_timeout":60,"requests":1000,"size":320}}}" }, { "key": "/apisix/services/414734269001237190", "value": "{"id":"414734269001237190","create_time":1656730487,"update_time":1657869578,"name":"护驾测试环境APPH5","desc":"护驾测试环境APPH5","upstream":{"nodes":{"172.16.5.185:80":1},"retries":3,"timeout":{"connect":6,"send":6,"read":6},"type":"roundrobin","scheme":"http","pass_host":"pass","keepalive_pool":{"idle_timeout":60,"requests":1000,"size":320},"retry_timeout":10}}" }, { "key": "/apisix/services/415053596615246534", "value": "{"id":"415053596615246534","create_time":1656920821,"update_time":1656920821,"name":"护驾测试环境H5静态资源文件","desc":"护驾测试环境H5静态资源文件","upstream":{"nodes":{"172.16.5.185:80":1},"timeout":{"connect":6,"send":6,"read":6},"type":"roundrobin","scheme":"http","pass_host":"pass","keepalive_pool":{"idle_timeout":60,"requests":1000,"size":320}}}" }, { "key": "/apisix/services/415054510755414726", "value": "{"id":"415054510755414726","create_time":1656921366,"update_time":1656921366,"name":"护驾测试环境WEB页面","desc":"护驾测试环境WEB页面","upstream":{"nodes":{"172.16.5.185:80":1},"timeout":{"connect":6,"send":6,"read":6},"type":"roundrobin","scheme":"http","pass_host":"pass","keepalive_pool":{"idle_timeout":60,"requests":1000,"size":320}}}" }, { "key": "/apisix/ssl/", "value": "init_dir" }, { "key": "/apisix/stream_routes/", "value": "init_dir" }, { "key": "/apisix/upstreams/", "value": "init_dir" } ]

1renwuruiming avatar Jul 16 '22 06:07 1renwuruiming

@tzssangglass 我从etcd导出了json数据，并脱敏了相关的信息，请看一下这样的路由提供方式是否有用，感谢

1renwuruiming avatar Jul 16 '22 06:07 1renwuruiming

@tzssangglass 我从etcd导出了json数据，并脱敏了相关的信息，请看一下这样的路由提供方式是否有用，感谢

ok, I will check this issue and #7136 together.

tzssangglass avatar Jul 17 '22 12:07 tzssangglass

My recurrence process is as follows:

apisix version 2.13.1

apisix-base version: nginx version: openresty/1.21.4.1

  1. upstream

use openresty mock upstream, config like:

master_process on;

worker_processes 2;

error_log logs/error.log warn;
pid logs/nginx.pid;

worker_rlimit_nofile 20480;

events {
    accept_mutex off;
    worker_connections 10620;
}

worker_shutdown_timeout 3;

http {
    server {
        listen 1980;
        location /escort/collect/ {
            content_by_lua_block {
                ngx.say("/escort/collect/")
            }
        }

        location /escort/collect/ws {
            content_by_lua_block {
                ngx.say("/escort/collect/ws")
            }
        }

        location /escort/web/ {
            content_by_lua_block {
                ngx.say("/escort/web/")
            }
        }
    }
}

and test

# hit /escort/collect/
$ curl http://127.0.0.1:1980/escort/collect/
/escort/collect/

$ curl http://127.0.0.1:1980/escort/collect/1
/escort/collect/


# hit /escort/collect/ws
$ curl http://127.0.0.1:1980/escort/collect/ws
/escort/collect/ws

$ curl http://127.0.0.1:1980/escort/collect/ws/
/escort/collect/ws


# hit /escort/web/
$ curl http://127.0.0.1:1980/escort/web/
/escort/web/

$ curl http://127.0.0.1:1980/escort/web/1
/escort/web/

As you can see, the route matching here follows the style of Nginx.

  1. route in APISIX
curl http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
  "uri":"/escort/collect/",
  "plugins":{
      "proxy-rewrite":{
          "regex_uri":[
              "/escort/collect/",
              "/"
          ]
      }
  },
  "upstream":{
      "type":"roundrobin",
      "nodes":{
          "127.0.0.1:1980":1
      }
  }
}'
  1. test proxy by APISIX
curl http://127.0.0.1:9080/escort/collect/
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>openresty/1.21.4.1</center>
</body>
</html>

here 404 status was returned by upstream, why? because the request URI was changed by proxy-rewrite from /escort/collect/ to /, the upstream doesn't have this location.

apisix version 2.13.2

apisix-base version: nginx version: openresty/1.21.4.1

Repeat the above process, and get consistent results.

possible errors

APISIX route matching differs from Nginx's location

Your routing configuration is as follows:

  • "uri":"/escort/collect/ws"
  • "uri":"/escort/collect/"
  • "uri":"/escortCommunity/"

According to the above test cases, if the request is curl http://127.0.0.1:9080/escort/collect/, would match the route1, and if the request is curl http://127.0.0.1:9080/escort/collect/1, would get:

 curl http://127.0.0.1:9080/escort/collect/1
{"error_msg":"404 Route Not Found"}

this means that APISIX cannot match the request, it does not match /escort/collect/ locations like Nginx does.

ref: https://github.com/apache/apisix/blob/master/docs/en/latest/terminology/route.md ref: https://github.com/apache/apisix/blob/master/docs/en/latest/terminology/router.md

proxy-rewrite

You need to verify whether it is the 404 Route Not Found returned by APISIX or the 404 returned by the upstream.

You should know exactly which route the request will be caught by and how the request URI will be modified by proxy-rewrite.

reproduce your case

You can describe what you think the routing chaos looks like, based on my reproduction case modification above.

cc @1renwuruiming @Fatpa

tzssangglass avatar Jul 18 '22 03:07 tzssangglass

And for reproducibility, you can test by providing just one route describing the difference between 2.13.1 and 2.13.2 in the same scenario (same route configuration, same upstream), which will allow us to pinpoint the problem faster.

tzssangglass avatar Jul 18 '22 03:07 tzssangglass

Or you think that if two routes routeA and routeB exist at the same time, and the same request is matched by routeA at 2.13.1 and by routeB at 2.13.2, then you can set up two routes.

tzssangglass avatar Jul 18 '22 03:07 tzssangglass

如果是upstream的路由不存在，那么为什么会时好时坏呢？在没有修改过apisix的route的情况下，如果上游路由地址不对，那应该始终是404，不是么

1renwuruiming avatar Jul 18 '22 03:07 1renwuruiming

First, we should know what mainly changed between 2.13.1 and 2.13.2?

And in my test case, my route setting is simple, only /*, and it doesn't happen in a single curl test, but in high concurrency scenes. And in the mean time, some of requests is work fine.

I still believe that the simple test case can't find out the problem of this case.

fatpa avatar Jul 18 '22 03:07 fatpa

During the upgrade processing, we don't change any routes config, all the errors should not lead by the route config. Maybe some performance problem exists?

fatpa avatar Jul 18 '22 03:07 fatpa

我将会重试2.14.1版本时的路由跳转问题，2.13.2版本我未曾进行过验证

1renwuruiming avatar Jul 18 '22 03:07 1renwuruiming

First, we should know what mainly changed between 2.13.1 and 2.13.2?

here is the commit history: https://github.com/apache/apisix/commits/release/2.13

And in my test case, my route setting is simple, only /*, and it doesn't happen in a single curl test, but in high concurrency scenes. And in the mean time, some of requests is work fine.

in APISIX, /* would match any request URI, So I don't think 404 will be generated by APISIX, we should focus on what happens between the request being proxied by APISIX and going upstream, which is unknown to me.

I still believe that the simple test case can't find out the problem of this case.

So can you give steps that can be reproduced and accurately describe the anomaly as I have given? I need to reproduce it in my environment in order to locate it.

tzssangglass avatar Jul 18 '22 04:07 tzssangglass

我将会重试2.14.1版本时的路由跳转问题，2.13.2版本我未曾进行过验证

I also tried it in 2.14.1 and the conclusion is the same.

tzssangglass avatar Jul 18 '22 04:07 tzssangglass

如果是upstream的路由不存在，那么为什么会时好时坏呢？在没有修改过apisix的route的情况下，如果上游路由地址不对，那应该始终是404，不是么

So can you give details of the steps that can reproduce this phenomenon?

tzssangglass avatar Jul 18 '22 04:07 tzssangglass