cal.com icon indicating copy to clipboard operation
cal.com copied to clipboard

feat: outlook cache

Open vijayraghav-io opened this issue 1 year ago • 9 comments

What does this PR do?

  • Fixes #21050 /claim #21050

This PR implements the functionality to -

  • Subscribe for notifications of - create, update or delete events in Microsoft Outlook Calendars (These calendars are pre-connected by team members with their Cal.com accounts, and are mapped to the cal.com events)
  • Receive notifications through a web-hook
  • Validates & filters these notifications and computes the set of outlook calendars (having unique externalIDs) for which the calendar-cache needs invalidation( or update)
  • fetchAvailabilityAndSetCache() is called for above outlook calendars to update the calendar-cache.
  • watch and unwatch functionality is implemented through Microsoft Graph APIs /subscribe , so that the existing cron job can start to subscribe or unsubscribe based on conditions (to avoid duplicate subscriptions).
  • With the calendar-cache available for outlook calendars ,the latest availabilities are shown in real time on public event page with low latency as Graph APIs are not called every time.

Key points: Visit the team public page with cal.cache query parameter set to true, to see the Cache effect or see [Cache Hit] in logs for office365_calendar. Example : http://localhost:3000/team/devs/teamevent?overlayCalendar=true&layout=month_view&cal.cache=true

In this PR, the startWatchingCalendarInMicrosoft() calls /subscription endpoint targeting specific calendars. This avoids subscribing to overly broad resources (e.g., me/events for all calendar events). Instead, target specific calendars or folders (e.g., me/calendars/{calendar_id}/events) to reduce notification volume. Screenshot 2025-05-04 at 11 02 29 PM

Next Steps or Improvements for scaling Given the volume of usage, if we have very high number of teams, team members, team events, simultaneously updating events in outlook - the volume of subscriptions received by the web-hook can be overwhelming even with load balancers.

We can consider Asynchronous Processing: Process notifications asynchronously to avoid blocking the webhook endpoint. For example, queue incoming notifications (using a cloud message queuing system like AWS SQS or RabbitMQ or Azure Queue Storage) and process them in the background with worker nodes. This ensures the endpoint remains responsive under high load.

Also we can consider using Azure Event Hubs or Azure Event Grid as alternative delivery channels. These services are designed for high-throughput event streaming and can buffer notifications, reducing the load on our application.

Video Demo

This demo shows - on creation of an event in outlook calendar the CalendarCache is updated through the webhook Cron job was triggered manually through api - http://localhost:3000/api/calendar-cache/cron?apiKey=

https://www.loom.com/share/468bf851a93d49849b2dab27fe751efd?sid=458f9244-e4fe-4632-897d-b93835535ff6

The second demo shows the effect of caching on public team event page-

https://www.loom.com/share/567c3d5af03e478e97a44299e9bcc8da?sid=9254a7e4-a28b-4a49-b17e-2890489fcf84

Mandatory Tasks (DO NOT REMOVE)

  • [x] I have self-reviewed the code (A decent size PR without self-review might be rejected).
  • [x] I have updated the developer docs in /docs if this PR makes changes that would require a documentation change. If N/A, write N/A here and check the checkbox.
  • [x] I confirm automated tests are in place that prove my fix is effective or that my feature works.

How should this be tested?

  • Are there environment variables that should be set? MICROSOFT_WEBHOOK_TOKEN - This token is sent with subscription requests through /subscriptions Graph API. The notifications received by webhook are verified with this token
  • What are the minimal test data to have? Enable calendar-cache feature flag for the team. Create a team event with hosts having their outlook calendars connected
  • What is expected (happy path) to have (input and output)? At any given point of time Calendar-Cache in cal.com has the latest busy slots in sync with changes in outlook calendar busy times or events. Update or create or delete event in Microsoft outlook calendar , there should be instant update in cal.com calendar-cache On visiting the team event public page with &cal.cahce=true, latest available slots are displayed with low latency
  • Any other important info that could help to test that PR To test locally use port forwarding providers

Summary by mrge

Added caching and webhook support for Outlook (Office365) calendar availability to improve performance and enable real-time updates.

  • New Features
    • Implemented webhook endpoint to receive Outlook calendar change notifications and refresh cache.
    • Added cache layer for Outlook calendar availability queries.
    • Updated database schema to store Outlook subscription info.

vijayraghav-io avatar May 02 '25 05:05 vijayraghav-io

@vijayraghav-io is attempting to deploy a commit to the cal Team on Vercel.

A member of the Team first needs to authorize it.

vercel[bot] avatar May 02 '25 05:05 vercel[bot]

Graphite Automations

"Add consumer team as reviewer" took an action on this PR • (05/02/25)

1 reviewer was added to this PR based on Keith Williams's automation.

"Add community label" took an action on this PR • (05/02/25)

1 label was added to this PR based on Keith Williams's automation.

"Add foundation team as reviewer" took an action on this PR • (05/16/25)

1 reviewer was added to this PR based on Keith Williams's automation.

graphite-app[bot] avatar May 02 '25 06:05 graphite-app[bot]

That was quick! Can you add some unit tests please?

Sure will add 🙏

vijayraghav-io avatar May 03 '25 00:05 vijayraghav-io

@hbjORbj , have added the key unit tests.

vijayraghav-io avatar May 03 '25 18:05 vijayraghav-io

@hbjORbj , also added integrated e2e test, that creates events on actual outlook calendar and verifies the cache 🙏

vijayraghav-io avatar May 16 '25 09:05 vijayraghav-io

@vijayraghav-io Thanks! I will give it a review soon!

hbjORbj avatar May 21 '25 18:05 hbjORbj

@vijayraghav-io Thanks! I will give it a review soon!

🙏

vijayraghav-io avatar May 23 '25 08:05 vijayraghav-io

@hbjORbj , Gentle reminder! 🙏

vijayraghav-io avatar May 30 '25 11:05 vijayraghav-io

@hbjORbj Any chance / is it realistic to get a review for that and get that in the next release? The performance for round-robin outlook team events is disappointing and has a huge impact on our business as we heavily rely on a daily basis for scheduling / finding availability with clients like multiple 10x per hour.

KlotzJesse avatar Jun 16 '25 08:06 KlotzJesse

This PR is being marked as stale due to inactivity.

github-actions[bot] avatar Jul 01 '25 00:07 github-actions[bot]

@hbjORbj , reminding 🙏

vijayraghav-io avatar Jul 01 '25 07:07 vijayraghav-io

Hey @vijayraghav-io, I noticed there are quite a few unrelated changes in the PR (things like added whitespace, new lines, reorganising of imports, and minor formatting like extra commas etc). Could you please revert those so the diff remains clean and it’s easier to focus on the actual changes?

Sure! will update.

vijayraghav-io avatar Jul 14 '25 01:07 vijayraghav-io

@kart1ka , cleaned up the diff.

vijayraghav-io avatar Jul 14 '25 05:07 vijayraghav-io

Walkthrough

This PR introduces Outlook (Office365) calendar cache and webhook support: a Microsoft Graph webhook POST handler (validated by MICROSOFT_WEBHOOK_TOKEN), SelectedCalendar schema fields for outlookSubscriptionId/outlookSubscriptionExpiration, Office365CalendarService enhancements (batched availability fetching, caching layer, watch/unwatch subscription lifecycle, delegation handling), repository/query updates, many unit and E2E tests and test utilities/mocks, plus calendar-cache documentation updates for Outlook.

Assessment against linked issues

Objective Addressed Explanation
Implement Outlook calendar caching mechanism to avoid per-visit live checks (#21050)
Process Microsoft Graph change notifications to update cache (#21050)
Manage Outlook webhook subscriptions (create, renew/expire, unwatch) for selected calendars (#21050)
Integrate cache usage into availability retrieval paths, including team events (#21050)

Assessment against linked issues: Out-of-scope changes

Code Change Explanation
Added optional locations and teamEventLength parameters in Playwright users fixture (apps/web/playwright/fixtures/users.ts) Test fixture API change unrelated to Outlook cache/webhook objectives; affects test helpers only.

Possibly related PRs

  • calcom/cal.com#21377 — Modifies Office365 calendar service; overlaps with subscription and availability logic added here.
  • calcom/cal.com#22532 — Extends calendar-cache feature; likely interacts with cache status and cache management changes in this PR.

[!TIP]

🔌 Remote MCP (Model Context Protocol) integration is now available!

Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats.

✨ Finishing Touches
  • [ ] 📝 Generate Docstrings
🧪 Generate unit tests
  • [ ] Create PR with unit tests
  • [ ] Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Status, Documentation and Community

  • Visit our Status Page to check the current availability of CodeRabbit.
  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

coderabbitai[bot] avatar Jul 16 '25 15:07 coderabbitai[bot]

Reminding...

vijayraghav-io avatar Aug 21 '25 05:08 vijayraghav-io

FYI @volnei this PR since you are working on calendar cache

keithwillcode avatar Aug 27 '25 12:08 keithwillcode

@vijayraghav-io FYI that we have completely restructured how calendar cache will work so we are going to close this PR. Thanks so much for your contribution and effort!

keithwillcode avatar Aug 27 '25 12:08 keithwillcode

/tip 100 @vijayraghav-io

keithwillcode avatar Aug 27 '25 12:08 keithwillcode

🎉🎈 @vijayraghav-io has been awarded $100 by Cal.com, Inc.! 🎈🎊

algora-pbc[bot] avatar Aug 27 '25 12:08 algora-pbc[bot]

@vijayraghav-io FYI that we have completely restructured how calendar cache will work so we are going to close this PR. Thanks so much for your contribution and effort!

Where can I track that / status of the new pr? Currently, in production a first load (for appointment setting it will always be that case and multiple per hour) it still loads 12 seconds on production cal.com with one mandatory team member, using outlook and several calendars and two users.

KlotzJesse avatar Aug 27 '25 12:08 KlotzJesse

@KlotzJesse #22708

keithwillcode avatar Aug 27 '25 14:08 keithwillcode