Create CRON to update integrations list for active connectors

[matteobad]

Summary

1. As a user
2. I want to have an updated list of available integrations
3. So that I can use the app to its fullest

Every (day|week|month)? we should have a CRON that triggers an API to fetch all available Integrations for the active Connectors and calculate the delta with a payload as follows:

{
  "connector_id": {
    "added": ["integration_id_1", ...],
    "modified": ["integration_id_2", ...],
    "removed": ["integration_id_3", ...]
  },
  ...
}

Then we can post this message into a queue (UpStash?) that should trigger our webhook to perform the actual update.

Acceptance Criteria

1. Every (day|week|month)? a GitHub action should trigger the CRON API
2. A message should be published to the queue
3. The webhook should be triggered by the queue and update the integrations

Also, here are a few points that need to be addressed:

1. UpStash account and keys
2. Signature verification of webhook
3. Performance and timeouts. Maybe a different API/CRON for every connector?
4. Meaningful logging

Resources

• Vercel CRON secure: https://dev.to/chrisnowicki/how-to-secure-vercel-cron-job-routes-in-nextjs-13-9g8
• UpStash: https://upstash.com/

Notes

• validate the onboarding process
• maybe split into multiple issues?