Replays for React Scan

[RobPruzan]

This PR Introduces a runtime for react scan to:

• collect detailed interaction timing by hacking details from browser event loop callbacks
• associate the custom interaction timing tracking with the PerformanceEntry API to derive the time between commit end and new frame presentation
• collect rrweb session replay events
• runs custom rrweb plugins that:
  • collect react-scan data, allowing the render outlines to be replayed without the user seeing them
  • collect live fps updates (for the purpose of correlating them to the replay)
  • track interaction timings during the session replay

Other changes include:

• an rrweb replayer plugin to render a canvas over the session replay with react scan outlines
• updated render tracking used in react scan monitoring
  • we now only track the blocking component renders
• collect the fiber subtree that re-rendered when ingesting in monitoring
• also collect the subtree of interacted element, incase the target was not the leaf node

What the PR does not include is the infrastructure behind replays for react scan, that will be open sourced at a future date

That functionaliy roughly includes:

• an ingestion server to collect interaction data
• a puppeteer cluster to replay the session replay offline, this allows us to:
  • convert the session replay to an mp4 by taking successive screenshots of the replayed session
  • take screenshots of the element interacted with
  • take screenshots of the render outlines over the page when the interaction happened
  • run expensive dom queries to understand the state of the app that we can't run on the users device
    • for example, querying for every single dom element if its in the viewport. This is not feasible to run on user device, but is trival to run offline in puppeteer
• storage for replays as mp4's, element screenshots, and video thumbnails
• Interaction summary w/ LLM's without using personalized user data
  • we never send data about the state of the application to a third party. This means we are limited in how good the summaries can get using only "static" information about the fiber tree. But, with a good enough prompt, the model does a good job in the general case

Preview of visualized collected data

https://github.com/user-attachments/assets/a6d39b27-991e-4992-8d64-e9952002ab3d

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
react-scan	❌ Failed (Inspect)			Jan 22, 2025 7:41am