gateway
gateway copied to clipboard
envoyproxy
feat(SecurityPolicies): add new featureAdding Security Policies for TCPGateways
feat(SecurityPolicies): add new featureAdding Security Policies for TCPGateways
adding security policies to TCPGateways using the existing API's
What this PR does / why we need it:
This PR will allow us to run authorization security policies only on TCPGateways Which issue(s) this PR fixes:
Fixes # https://github.com/envoyproxy/gateway/issues/4908
Release Notes: Yes/No
To add more contexts Here's some logs. Here's the envoy-gateway logs when i apply the security policy. I added a lot of print statement debugging
kubectl -n envoy-gateway-system logs deployments/envoy-gateway | grep -E "(tcp-rbac|Processing RBAC|Created RBAC|Added RBAC|action:|default_action:|Authorization|RBAC|CIDR|principal)
Public IP address have been redacted
2025-04-29T19:20:38.677Z DEBUG gateway-api runner/runner.go:196 {"readyListener":{"ipFamily":"IPv4","address":"0.0.0.0","port":19003,"path":"/ready"},"accessLog":{"json":[{"path":"/dev/stdout"}]},"tcp":[{"name":"default/tcp-gateway/foo","address":"0.0.0.0","port":8088,"routes":[{"name":"tcproute/default/tcp-app-1","destination":{"name":"tcproute/default/tcp-app-1/rule/-1","settings":[{"name":"tcproute/default/tcp-app-1/rule/-1/backend/0","weight":1,"protocol":"TCP","endpoints":[{"host":"10.100.64.92","port":3000,"zone":"0"}],"addressType":"IP","ipFamily":"IPv4"}]}}]},{"name":"default/tcp-gateway/bar","address":"0.0.0.0","port":8089,"routes":[{"name":"tcproute/default/tcp-app-2","destination":{"name":"tcproute/default/tcp-app-2/rule/-1","settings":[{"name":"tcproute/default/tcp-app-2/rule/-1/backend/0","weight":1,"protocol":"TCP","endpoints":[{"host":"10.100.64.132","port":3000,"zone":"0"}],"addressType":"IP","ipFamily":"IPv4"}]},"security":{"authorization":{"rules":[{"name":"securitypolicy/default/tcp-app-2-security-policy/authorization/rule/0","action":"Allow","principal":{"clientCIDRs":[{"cidr":"x.x.x.x/x,"ip":"x.x.x.x","maskLen":x,"isIPv6":false,"distinct":false},{"cidr":"y.y.y.y/y","ip":"y.y.y.y","maskLen": y,"isIPv6":false,"distinct":false}],"UseDownstreamSourceIP":true}}],"defaultAction":"Deny"}}}],"networkFilters":[{"name":"envoy.filters.network.rbac","config":{"rules":[{"name":"securitypolicy/default/tcp-app-2-security-policy/authorization/rule/0","action":"Allow","principal":{"clientCIDRs":[{"cidr":"x.x.x.x/x,"ip":"x.x.x.x","maskLen":x,"isIPv6":false,"distinct":false},{"cidr":"y.y.y.y/y","ip":"y.y.y.y","maskLen": y,"isIPv6":false,"distinct":false}],"UseDownstreamSourceIP":true}}],"defaultAction":"Deny","statPrefix":"tcp_rbac_","sourceIPEnforcement":true}}]}]} {"runner": "gateway-api", "xds-ir": "default/tcp-gateway"}
2025-04-29T19:20:38.678Z INFO provider.tcp-rbac translator/listener.go:603 Processing RBAC filter {"runner": "provider", "filter_name": "envoy.filters.network.rbac", "default_action": "Deny", "num_rules": 1}
2025-04-29T19:20:38.678Z INFO provider translator/listener.go:1129 Converting principals {"runner": "provider", "num_cidrs": 2}
2025-04-29T19:20:38.678Z INFO provider translator/listener.go:1133 Processing CIDR {"runner": "provider", "cidr": "x.x.x.x/x, "ip": "x.x.x.x", "mask_len": x}
2025-04-29T19:20:38.678Z INFO provider.cidr-converter translator/listener.go:1165 Parsed CIDR successfully {"runner": "provider", "ip": "x.x.x.x", "prefix_len": x, "cidr": "x.x.x.x/x}
2025-04-29T19:20:38.678Z INFO provider translator/listener.go:1133 Processing CIDR {"runner": "provider", "cidr": "y.y.y.y/y", "ip": "y.y.y.y", "mask_len": y}
2025-04-29T19:20:38.678Z INFO provider.cidr-converter translator/listener.go:1165 Parsed CIDR successfully {"runner": "provider", "ip": "y.y.y.y", "prefix_len": y, "cidr": "y.y.y.y/y"}
2025-04-29T19:20:38.678Z INFO provider.tcp-rbac translator/listener.go:616 Created RBAC config {"runner": "provider", "action": "ALLOW", "num_policies": 1}
2025-04-29T19:20:38.679Z INFO provider.tcp-rbac translator/listener.go:622 Added RBAC filter to chain {"runner": "provider"}
here is log from the tcpgateway pod. The first one is when i'm not on the list, the second one i am This line seems like it should be blocking, but its not
"connection_termination_details": "rbac_access_denied_matched_policy[none]"
{
":authority": null,
"bytes_received": 0,
"bytes_sent": 0,
"connection_termination_details": "rbac_access_denied_matched_policy[none]",
"downstream_local_address": "10.100.64.83:8089",
"downstream_remote_address": "y.y.y.y:65242",
"duration": 2,
"method": null,
"protocol": null,
"requested_server_name": null,
"response_code": 0,
"response_code_details": null,
"response_flags": "-",
"route_name": null,
"start_time": "2025-04-29T19:22:03.007Z",
"upstream_cluster": "tcproute/default/tcp-app-2/rule/-1",
"upstream_host": "10.100.64.132:3000",
"upstream_local_address": "10.100.64.83:55826",
"upstream_transport_failure_reason": null,
"user-agent": null,
"x-envoy-origin-path": null,
"x-envoy-upstream-service-time": null,
"x-forwarded-for": null,
"x-request-id": null
}
{
":authority": null,
"bytes_received": 0,
"bytes_sent": 0,
"connection_termination_details": null,
"downstream_local_address": "10.100.64.83:8089",
"downstream_remote_address": "y.y.y.y:65534",
"duration": 5,
"method": null,
"protocol": null,
"requested_server_name": null,
"response_code": 0,
"response_code_details": null,
"response_flags": "-",
"route_name": null,
"start_time": "2025-04-29T19:23:05.304Z",
"upstream_cluster": "tcproute/default/tcp-app-2/rule/-1",
"upstream_host": "10.100.64.132:3000",
"upstream_local_address": "10.100.64.83:36492",
"upstream_transport_failure_reason": null,
"user-agent": null,
"x-envoy-origin-path": null,
"x-envoy-upstream-service-time": null,
"x-forwarded-for": null,
"x-request-id": null
}
here's the running config of the section too. from the tcpgateway pod config dump when port forwarded to 19000
},
{
"name": "default/tcp-gateway/bar",
"active_state": {
"version_info": "ce062826bcc75a4f7b1412ed78307d7fffb17c9bcdf978b8d00c7585f1f68d5f",
"listener": {
"@type": "type.googleapis.com/envoy.config.listener.v3.Listener",
"name": "default/tcp-gateway/bar",
"address": {
"socket_address": {
"address": "0.0.0.0",
"port_value": 8089
}
},
"filter_chains": [
{
"filters": [
{
"name": "envoy.filters.network.rbac",
"typed_config": {
"@type": "type.googleapis.com/envoy.extensions.filters.network.rbac.v3.RBAC",
"rules": {
"action": "DENY",
"policies": {
"securitypolicy/default/tcp-app-2-security-policy/authorization/rule/0": {
"permissions": [
{
"any": true
}
],
"principals": [
{
"direct_remote_ip": {
"address_prefix": "x.x.x.x",
"prefix_len": x
}
}
]
}
}
},
"stat_prefix": "tcp_rbac_"
}
},
{
"name": "envoy.filters.network.tcp_proxy",
"typed_config": {
"@type": "type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy",
"stat_prefix": "tcp-8089",
"cluster": "tcproute/default/tcp-app-2/rule/-1",
"access_log": [
{
"name": "envoy.access_loggers.file",
"typed_config": {
"@type": "type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog",
"path": "/dev/stdout",
"log_format": {
"json_format": {
"duration": "%DURATION%",
"downstream_local_address": "%DOWNSTREAM_LOCAL_ADDRESS%",
"connection_termination_details": "%CONNECTION_TERMINATION_DETAILS%",
"bytes_sent": "%BYTES_SENT%",
"x-envoy-upstream-service-time": "%RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)%",
"x-forwarded-for": "%REQ(X-FORWARDED-FOR)%",
"upstream_host": "%UPSTREAM_HOST%",
"start_time": "%START_TIME%",
"upstream_cluster": "%UPSTREAM_CLUSTER%",
"protocol": "%PROTOCOL%",
"route_name": "%ROUTE_NAME%",
"method": "%REQ(:METHOD)%",
"x-envoy-origin-path": "%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%",
"x-request-id": "%REQ(X-REQUEST-ID)%",
"response_code": "%RESPONSE_CODE%",
":authority": "%REQ(:AUTHORITY)%",
"requested_server_name": "%REQUESTED_SERVER_NAME%",
"user-agent": "%REQ(USER-AGENT)%",
"downstream_remote_address": "%DOWNSTREAM_REMOTE_ADDRESS%",
"response_flags": "%RESPONSE_FLAGS%",
"response_code_details": "%RESPONSE_CODE_DETAILS%",
"bytes_received": "%BYTES_RECEIVED%",
"upstream_transport_failure_reason": "%UPSTREAM_TRANSPORT_FAILURE_REASON%",
"upstream_local_address": "%UPSTREAM_LOCAL_ADDRESS%"
}
}
}
}
]
}
}
],
"name": "tcproute/default/tcp-app-2"
}
],
"per_connection_buffer_limit_bytes": 32768,
"access_log": [
{
"name": "envoy.access_loggers.file",
"filter": {
"response_flag_filter": {
"flags": [
"NR"
]
}
},
new log
"name": "default/tcp-gateway/bar",
"active_state": {
"version_info": "cd57adf3500b8189d34161f4db870f7074e0019e750cecd778c909ec8964df6a",
"listener": {
"@type": "type.googleapis.com/envoy.config.listener.v3.Listener",
"name": "default/tcp-gateway/bar",
"address": {
"socket_address": {
"address": "0.0.0.0",
"port_value": 8089
}
},
"filter_chains": [
{
"filters": [
{
"name": "envoy.filters.network.rbac",
"typed_config": {
"@type": "type.googleapis.com/envoy.extensions.filters.network.rbac.v3.RBAC",
"rules": {
"policies": {
"securitypolicy/default/tcp-app-2-security-policy/authorization/rule/0": {
"permissions": [
{
"any": true
}
],
"principals": [
{
"direct_remote_ip": {
"address_prefix": "192.168.254.0",
"prefix_len": 24
}
}
]
}
}
},
"stat_prefix": "tcp_rbac_"
}
},
{
"name": "envoy.filters.network.tcp_proxy",
"typed_config": {
"@type": "type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy",
"stat_prefix": "tcp-8089",
"cluster": "tcproute/default/tcp-app-2/rule/-1",
"access_log": [
{
"name": "envoy.access_loggers.file",
"typed_config": {
"@type": "type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog",
"path": "/dev/stdout",
"log_format": {
"json_format": {
"downstream_remote_address": "%DOWNSTREAM_REMOTE_ADDRESS%",
"route_name": "%ROUTE_NAME%",
"method": "%REQ(:METHOD)%",
"upstream_transport_failure_reason": "%UPSTREAM_TRANSPORT_FAILURE_REASON%",
"requested_server_name": "%REQUESTED_SERVER_NAME%",
"x-forwarded-for": "%REQ(X-FORWARDED-FOR)%",
"duration": "%DURATION%",
"x-request-id": "%REQ(X-REQUEST-ID)%",
":authority": "%REQ(:AUTHORITY)%",
"bytes_received": "%BYTES_RECEIVED%",
"x-envoy-upstream-service-time": "%RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)%",
"protocol": "%PROTOCOL%",
"start_time": "%START_TIME%",
"x-envoy-origin-path": "%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%",
"response_flags": "%RESPONSE_FLAGS%",
"response_code": "%RESPONSE_CODE%",
"upstream_host": "%UPSTREAM_HOST%",
"upstream_cluster": "%UPSTREAM_CLUSTER%",
"upstream_local_address": "%UPSTREAM_LOCAL_ADDRESS%",
"connection_termination_details": "%CONNECTION_TERMINATION_DETAILS%",
"bytes_sent": "%BYTES_SENT%",
"downstream_local_address": "%DOWNSTREAM_LOCAL_ADDRESS%",
"response_code_details": "%RESPONSE_CODE_DETAILS%",
"user-agent": "%REQ(USER-AGENT)%"
}
}
}
}
]
}
}
],
"name": "tcproute/default/tcp-app-2"
}
This pull request has been automatically marked as stale because it has not had activity in the last 30 days. Please feel free to give a status update now, ping for review, when it's ready. Thank you for your contributions!
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}