Feat: Dynamic (Event) Filters

[mrkaye97]

Description

Introduces dynamic event filters, which are persisted into v1_filter and then queried when we go to trigger a workflow from an event.

Some decisions made here / open questions:

1. I was thinking it'd make sense to have the (tenant_id, id) PK with a unique index on (tenant_id, workflow_id, resource_hint, expression) so that: 1) it's fast to look up + delete filters by the id and 2) it's fast to look up which filters match a workflow.
2. I was thinking we want to "FK" these to the workflow_id and not the workflow version to give the expected behavior to users (basically they know nothing about workflow versions and would expect that e.g. if they add an event trigger to their workflow, the filters would still be there). If we FK to the workflow version, we'd need to migrate these over each time.
3. Should this API be on the REST API or the gRPC one? Went with gRPC since events are pushed via gRPC + we use it for e.g. putting workflows, but don't feel strongly

TODO:

• [x] Implement delete method
• [ ] Write tests for various cases (key used in CEL doesn't exist in input, no input provided, no hint provided, triggering events with no filters works as expected, ...)
• [x] I think that if no hint is provided, we need to make sure that the query for filters returns an empty list
• [x] Change this to use the REST API instead of the gRPC one
• [x] Add the payload to the CEL env

Type of change

• [x] New feature (non-breaking change which adds functionality)

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
hatchet-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	May 16, 2025 7:37pm
hatchet-v0-docs	🛑 Canceled (Inspect)			May 16, 2025 7:37pm

[mrkaye97]

Looking pretty good -- one question that's not immediately clear to me, if we trigger a workflow based on a filter, I assume we'd want both the triggering event payload and the filter's payload to be present in the input. Where is that being propagated at the moment?

~Ah that's a good thought, let me take a stab at this~

Edit: I think there's not necessarily a guarantee that a single filter will apply to any event / workflow run pair, which is also true in the code right now (we loop over the set of filters that match the workflow id and the resource hint). I'm not sure it makes sense to potentially send all of the matching payloads back.

Also, how valuable is the filter payload if a workflow was triggered (meaning the filter did not apply)?