AR Foundation Replay

🚧 Work in progress 🏗️

PoC of Recording AR Foundation session into a single mp4 with a binary timeline track / Replaying it in Editor on AR Foundation without any extra settings.

https://github.com/asus4/ARFoundationReplay/assets/357497/8e77ee45-6f2c-442e-a47a-f35f044b8181

↓ Check out the complete project WorldEnsambe utilizing ARFoundationReplay.

https://github.com/asus4/WorldEnsemble/assets/357497/1ff03fd4-01cf-41a6-8aef-42e11f7a67a2

Tested platform

• Only iOS ARKit + macOS Editor for now. Android ARCore is planned.
• Tested with the latest Unity2022 LTS.
• Supported only URP. Built-in is not supported for now.

Implemented Subsystems

• [x] XRSessionSubsystem
• [x] XRCameraSubsystem
• [ ] XRPointCloudSubsystem
• [x] XRPlaneSubsystem
• [ ] XRAnchorSubsystem
• [ ] XRRaycastSubsystem
• [ ] XRHumanBodySubsystem
• [ ] XREnvironmentProbeSubsystem
• [x] XRInputSubsystem
• [ ] XRImageTrackingSubsystem
• [ ] XRObjectTrackingSubsystem
• [ ] XRFaceSubsystem
• [x] XROcclusionSubsystem
• [ ] XRParticipantSubsystem
• [ ] XRMeshSubsystem
• ARCore Extensions
  • [x] Earth Manager
  • [ ] Cloud Anchors
  • [x] Streetscape Geometry

How to use

• Download System.Runtime.CompilerServices.Unsafe NuGet package and rename to zip to extract. Locate lib/netstandard2.0/System.Runtime.CompilerServices.Unsafe.dll to Assets/Plugins/ directory. Or you can copy it from this repository.

• Add the following line to Packages/manifest.json to install AR Foundation Replay package via UPM:

  "dependencies": {
      "com.cysharp.memorypack": "https://github.com/Cysharp/MemoryPack.git?path=src/MemoryPack.Unity/Assets/Plugins/MemoryPack#1.10.0",
      "com.github.asus4.arfoundationreplay": "https://github.com/asus4/ARFoundationReplay.git?path=Packages/com.github.asus4.arfoundationreplay",
      ... other dependencies
  }
  

• Simulation on the Unity Editor is supported only on iOS. Open Build Settings and switch the platform to iOS.

• Activate AR Foundation Replay for Unity Editor:

  1. Open Project Settings/XR Plug-in Management.
  2. Select the PC tab in the XR Plug-in Management.
  3. Activate AR Foundation Replay

• Put the ARRecordButton prefab into the Scene which is located at Packages/com.github.asus4.arfoundationreplay/Prefabs/ARRecordButton.prefab.

• Record AR on the device.

• You can change the replay file from Project Settings/XR Plug-in Management/AR Foundation Replay.

• Play the AR in the Editor.

ARCore Extensions Support

It supports replaying ARCore Geospatial. To test it, please use the forked version of the arcore-unity-extensions and refer to the ARFoundationReplayGeospatial example project.

Dependencies

• MemoryPack - MIT License
• XR SDK - Unity Companion License
• keijiro/Bibcam - Unlicense
• keijiro/Avfi - Unlicense: Metadata recording system is modified from Avfi.