Adding HTTP service support for Envoy external processing

[yanjunxiang-google]

Title: Adding HTTP service support for Envoy external processing

Description: Currently Envoy ext_proc filter can only talk to a gRPC service to do external processing. There is a need for ext_proc filter to be able to talk to a raw HTTP service to do the external processing.

[yanjunxiang-google]

It is planned to have multiple incremental PRs to support this feature:

1. API to add the API service

2. Adding ext_proc HTTP client

3. Refactor the ext_proc filter code to be able to support either gRPC client or HTTP client based on filter configuration.

4. Adding ext_proc HTTP service testing framework.

5. Adding integration test to test ext_proc filter <----> HTTP service end-to-end.

6. Update the API to describe how the HTTP messages will be constructed.

7. Adding HTTP request-ID and session affinity as in this comment:

https://github.com/envoyproxy/envoy/pull/35740#issuecomment-2307266324

x) TBD:

x.1) Do we need to support per-route override of http_service config, like here? https://github.com/envoyproxy/envoy/blob/6277f4601357550b0eba93fa3fd2a4154d63416b/api/envoy/extensions/filters/http/ext_proc/v3/ext_proc.proto#L372

This won't be considered in the MVP of ext_proc HTTP support.

x.2) Logging the HTTP ext_proc call, something like gRPC stream logging here: https://github.com/envoyproxy/envoy/blob/6277f4601357550b0eba93fa3fd2a4154d63416b/source/extensions/filters/http/ext_proc/ext_proc.cc#L898

x.3) Adding test for having the side stream server using HTTP1 protocol.

x.4) Refactoring the code to move the stream_ variable out from Filter class, and put it into ExternalProcessorClientImpl class. So, the Filter::sendRequest() can directly call: client_->sendRequest() for better abstraction.

x.5) change client_impl.[h|cc] into grpc_client_impl.[h|cc] to distinguish it from http client files.

x.6) do we need something for HTTP for logging/monitoring purpose, something like: void Filter::logGrpcStreamInfo().

x.7: do we need to set call_backs_ to nullptr at http_client->cancel() (not necessary, need to revisit)

x.8) Explore the option to refactor client_->start() to not have dynamic_cast there.

x.9) Explore the option to refactore stream_->sendRequest() to avoid dynamic_cast there.

x.10) session affinity support

[soulxu]

cc @gbrail @stevenzzzz @tyxia @mattklein123 @htuch @yanavlasov

[yanjunxiang-google]

The processing flow works this way:

When Envoy needs to send call out messages to the side stream server, either header, body , or trailer, Envoy construct a ProcessingRequest proto message. This ProcessingRequest message is transcoded into a JSON text. This JSON text is then added as a body to a HTTP "POST" message. This HTTP message is sent to the side stream HTTP server. The HTTP server receives the "POST" message. Transcoding the body, i.e, a JSON text, into a ProcessingRequest message. The HTTP server then performs normal header/body/trailer mutation as existing, and put them into a ProcessingResponse proto message. The HTTP server then transcode the ProcessingResponse into JSON text, and sends back a 200 response with this JSON text as body. After receives the 200 response, Envoy then convert the body from JSON text back to a ProcessingResponse proto message. Then continue processing this ProcessingResponse. Overall, the ext_proc gRPC and HTTP share identical state machine. The only difference is that in the transport layer, one using gRPC, and the ProcessingRequest/ProcessingResponse messages are sent as proto messages. The other sends messages using HTTP, with the ProcessingRequest/ProcessingResponse proto messages are transcoded into JSON text, and encoded as the body in the HTTP messages.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

[yanjunxiang-google]

#35740 fixed the issue.