gateway icon indicating copy to clipboard operation
gateway copied to clipboard

Support gRPC-JSON transcoder

Open quyenhoang96 opened this issue 2 years ago • 8 comments

Description: For now, Envoy-Gateway has GRPCRoute. And in some contexts, We need to call from the client(maybe browser, mobile ...) to service. And it calls with HTTP protocol by Restfull. Can we consider about gRPC-JSON transcoder to make GRPCRoute can call with HTTP.

https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/grpc_json_transcoder_filter

quyenhoang96 avatar Aug 09 '23 00:08 quyenhoang96

can EnvoyPatch help with this?

zirain avatar Aug 09 '23 02:08 zirain

can EnvoyPatch help with this? Hi @zirain How the way can it help?

quyenhoang96 avatar Aug 10 '23 04:08 quyenhoang96

@quyenhoang96 as a workaround you should be able to define any xds resource using https://gateway.envoyproxy.io/v0.5.0/user/envoy-patch-policy.html

arkodg avatar Aug 22 '23 00:08 arkodg

In https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/grpc_json_transcoder_filter . It need to bin file. How to workaround bin file to envoy-proxy can read it.

quyenhoang96 avatar Aug 23 '23 02:08 quyenhoang96

assigning to @AliceProxy who is interested in implementing this one, thanks !

arkodg avatar Aug 30 '23 18:08 arkodg

This issue has been automatically marked as stale because it has not had activity in the last 30 days.

github-actions[bot] avatar Sep 29 '23 20:09 github-actions[bot]

there are two options here

  • we can leverage the existing feature in envoy which is based on the user generating the static proto descriptor sets https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/grpc_json_transcoder_filter#how-to-generate-proto-descriptor-set
  • or rely on a reflection endpoint https://github.com/envoyproxy/envoy/pull/23061 (Envoy PR needs to be taken forward)

arkodg avatar Aug 14 '24 14:08 arkodg

This issue has been automatically marked as stale because it has not had activity in the last 30 days.

github-actions[bot] avatar Sep 13 '24 16:09 github-actions[bot]

This issue has been automatically marked as stale because it has not had activity in the last 30 days.

github-actions[bot] avatar Nov 09 '24 08:11 github-actions[bot]

I'd like to take a crack at this. I'm new to the repo. Are there any somewhat similar feature additions that someone could link to for me to use as a template for navigating the codebase?

ouillie avatar Aug 09 '25 17:08 ouillie

thanks for looking into this one @ouillie, ideally would like https://github.com/envoyproxy/envoy/pull/23061 to land first in upstream so its improves UX here

arkodg avatar Aug 14 '25 17:08 arkodg

FYI the filter has a proto_descriptor_bin inline option as well as the proto_descriptor file reference option, and I believe that could already work fine with a patch policy for small services.

ouillie avatar Aug 14 '25 18:08 ouillie

with the bin option we are limited to the max size of ConfigMap

arkodg avatar Aug 14 '25 18:08 arkodg

@arkodg Given that the gRPC reflection issue has been outstanding for over a year and appears to lack active contributors, perhaps we should avoid holding up this feature on that PR. We might consider proceeding without waiting for progress on grpc reflection, or alternatively raise it separately if it remains important.

nfarhadian avatar Sep 04 '25 00:09 nfarhadian

if the proto descriptor bin for majority cases is less than 1MB and fits in a ConfigMap, this approach will work, hoping someone can pick this issue up and do this analysis and make sure this approach is useful for most cases

arkodg avatar Sep 04 '25 00:09 arkodg

In our setup we have 90+ services, each with its own transcoder. The proto descriptor binaries average ~120 KB in size, with the largest we’ve seen around 360 KB. This suggests the proposed approach should work reliably in practice, since the descriptor size remains well within the ConfigMap limit.

At the moment we use Istio EnvoyFilter to inject transcoders into sidecars, which gives individual teams the flexibility to manage their own transcoders. As we migrate to Envoy Gateway, relying on EnvoyPatchPolicy for this would mean exposing powerful configuration options to all teams, which carries the risk of accidental or insecure changes at the gateway level.

For us, this feature directly addresses that gap by providing a safe, scalable alternative.

nfarhadian avatar Sep 04 '25 18:09 nfarhadian

thanks for sharing this data @nfarhadian, marking this issue as help wanted @ouillie are you still interested in picking this one up ?

arkodg avatar Sep 04 '25 20:09 arkodg