Proposal for enhancing Envoy with single-process multi-tenancy capabilities to optimize resource utilization and cost

[alpha-baby]

Background

Related Discussions：https://twitter.com/mattklein123/status/1522925333053272065

As enterprises increasingly focus on the efficiency and cost of IT infrastructure, optimizing the deployment of foundational components such as proxy servers can yield significant benefits. Envoy, as a key element of the proxy infrastructure, could greatly contribute to cost reduction if managed effectively.

Design

One question that arises is: How can we lower the costs associated with running Envoy?

For instance, Istio has introduced the concept of Ambient Mesh to address this. In Kubernetes environments, Envoy or Ztunnel might be deployed via a DaemonSet across all nodes, necessitating that Envoy or Ztunnel offer a solution to manage resources efficiently.

Feature	mutil-thread	mutil-process
Isolation	Lower, tenants share the same memory space	Higher, each tenant runs in a separate memory space
Resource Utilization	Higher, threads are generally lighter than processes and share application state	Average, each process has its own resource overhead
Performance Overhead	Lower, less overhead for context switching between threads	Higher, more resources needed for context switching between processes
Scalability	Higher, new threads can be easily created to handle more load	Average, new prcesses new more system resources
Deployment & Operational Complexity	Lower, managing a single application instance	Higher, need to manage multiple application instances
Cost Efficiency	Higher, better resource sharing	Average, each process requires separate resources
Fault Isolation	Lower, a crash in one thread could affect the entire application	Higher, a crash in one process usually doesn't affect others
Security & Compliance	Lower, memory sharing could pose security risks	Higher, memory isolation between processes enhances security
Flexibility & Customization	Average, all tenants share the same application code and resources	Higher, processes can be customized for each tenant
Updates & Maintenance	Higher, updating a single application affects all tenants	Average, each process can be updated independently, but with more complex management
XDS & control plane	Lower, reusing one channel of XDS	Higher, each prcoess has a channel of XDS

note: this comparison provides a general perspective. In practice, the specific advantages and disadvantages will be influenced by the architecture of the application, business logic, security requirements, and operational capabilities. In some scenarios, a hybrid model may be adopted, combining the benefits of both multithreading and multiprocessing to meet specific needs.

Our team believes that Envoy should possess single-process multi-tenancy capabilities.

mutil-thread blueprints：

I must admit that I am not thoroughly familiar with the inner workings of Envoy, but I will plan to present a detailed design proposal later.

We would welcome any suggestions or better approaches from the Envoy team regarding this enhancement. Your expertise and insights are highly anticipated and would be invaluable in guiding us toward an effective solution.

Thank you for considering this proposal, and I look forward to a collaborative effort to improve Envoy's cost-efficiency and resource utilization within multi-tenant environments.

[zuercher]

You might try to strike up a discussion in Envoy slack.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

[github-actions[bot]]

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted" or "no stalebot". Thank you for your contributions.