Message accepted on broken payload (invalid JSON)

[ryansiau]

Sending an invalid JSON payload on incoming webhooks (haven't tested outgoing yet) creates an event that does not show on both "Event Deliveries" and "Events Log" page. However, there are some logs related to the event

{
    "error": "unexpected end of JSON input",
    "level": "error",
    "msg": "error find a matching subscription for this source",
    "time": "2023-10-20 10:55:18"
}

Reproduce

This is persistent and can be reproduced easily. for context, i used docker-compose to deploy the instance

1. create a project for incoming webhook
2. create a source, endpoint, and corresponding subscription
3. send any invalid json as payload, for example like this:

{
  "data": ""

after sending the request, the terminal will be filled with error logs similar to these

{"error":"unexpected end of JSON input","level":"error","msg":"error find a matching subscription for this source","time":"2023-10-20 11:15:27"}
{"error":"error find a matching subscription for this source","job":"CreateEventProcessor","level":"error","msg":"job failed","source":"worker","time":"2023-10-20 11:15:27"}

Discussion

I'm wondering if we can avoid this error by adding a payload validator or adding support for custom middleware?

[subomi]

Hi @ryansiau ,

Yeah, this is related to #1767. We should roll out a patch soon.

[ryansiau]

I see, thank you for the confirmation @subomi, will be waiting for the update :D

[RaghavSood]

It would be nice to be able to disable json-filter matching on source that are known to not send JSON messages (such as encrypted payloads or files), rather than force it to only handle JSON - this is currently a blocker for us