Camera2 Pass Images from ImageReader to MediaRecorder

[mrousavy]

Hi!

I'm trying to create a Camera2 CameraCaptureSession that is capable of four outputs:

1. On-screen preview (SurfaceView, up to 1080p)
2. Photo capture (ImageReader, up to 8k photos)
3. Video Capture (MediaRecorder/MediaCodec, up to 4k videos)
4. Frame Processing (ImageReader, up to 4k video frames)

Unfortunately Camera2 does not support attaching all of those four outputs (Surfaces) at the same time, so I'm going to have to make a compromise.

The compromise that seemed most logical to me was to combine the two video capture pipelines into one, so that the Frame Processing output (#4, ImageReader) redirects the frames into the Video Capture output (#3, MediaRecorder).

How do I write the Images from the ImageReader:

val imageReader = ImageReader.newInstance(4000, 2256, ImageFormat.YUV_420_888, 3)
imageReader.setOnImageAvailableListener({ reader ->
  val image = reader.acquireNextImage() ?: return@setOnImageAvailableListener
  callback.onVideoFrameCaptured(image)
}, queue.handler)

val captureSession = device.createCaptureSession(.., imageReader.surface)

..into the Surface from the MediaRecorder?

val surface = MediaCodec.createPersistentInputSurface()
val recorder = MediaRecorder(context)
..
recorder.setInputSurface(surface)

---

I'm not too familiar with OpenGL, so any help here would be appreciated.

[lbccccc]

1. 在OPENGL环境中创建OES texture->Surfacetexture->Surface ，用这个surface去configSession
2. 预览帧来了之后会绑定到OES texture中，然后你可以绘制到MediaRecorder或者imageReader的surface上