Adding RabbitMQ Adapter

[koladev32]

We propose integrating RabbitMQ as another option for receiving and processing webhooks in the Sendhooks engine. This will involve creating a RabbitMQ adapter similar to our existing Redis adapter, allowing for flexibility and easier implementation of different message brokers in the future.

Motivation

Currently, Sendhooks uses Redis for handling webhooks. While Redis is fast and reliable, integrating RabbitMQ will provide additional flexibility and options for developers who prefer RabbitMQ for its robust messaging capabilities. This integration will make Sendhooks more versatile and appealing to a broader range of users.

Proposed Solution

1. Create a RabbitMQ Adapter: Implement a new adapter in the adapter/rabbitmq_adapter package that follows the same interface as the existing Redis adapter.
2. Configuration: Update the configuration structure to support RabbitMQ settings, such as connection URL, exchange, queue, and routing key.
3. Adapter Manager: Modify the adapter manager to initialize and use the RabbitMQ adapter based on configuration.
4. Testing: Write unit and integration tests for the RabbitMQ adapter to ensure it works correctly with the rest of the Sendhooks system.

RabbitMQ Adapter Implementation

• Connect: Establish a connection to the RabbitMQ server.
• SubscribeToQueue: Subscribe to a RabbitMQ queue to receive webhooks.
• ProcessWebhooks: Process the received webhooks with retry logic and exponential backoff.
• PublishStatus: Publish webhook processing status updates to a RabbitMQ exchange.

Configuration Example

Extend the config.json file to include RabbitMQ settings:

{
  "broker": "rabbitmq",
  "rabbitmq": {
    "url": "amqp://user:password@host:port/vhost",
    "exchange": "webhooks",
    "queue": "webhooks_queue",
    "routingKey": "webhooks_key"
  },
  "numWorkers": 100,
  "channelSize": 10000
}

[cedrickah]

I'm a little perplexed because I saw the motivation for the project. If it's about improving performance, why not first think about using tricks to improve the performance of the code. For example use Cython for Python? It was a parenthesis. To share information between programs on the same machine, trying fast means of transport like Redis, RabbitMQ one by one seems too advanced to me. Why not just look at the shared memory side? If it needs to be distributed within a network, that's another story.