Camera2 API - failed to take photo when TEMPLATE_RECORD is used for preview and video recording and video stabilization is enabled

[anonym24]

Issue was posted on SO as well: https://stackoverflow.com/questions/71712448/camera2-api-failed-to-take-photo-when-template-record-is-used-for-preview-and

Also on Android Issue Tracker: https://issuetracker.google.com/issues/227759163

Here's the official sample Camera2 API app to take photos.

It works fine for all devices and you can take photo by pressing the button without any issue.

But my own app is a little different. It also supports recording videos, that's why I use TEMPLATE_RECORD instead of TEMPLATE_PREVIEW for my viewFinderSurface and recordSurface

So basically code at https://github.com/android/camera-samples/tree/main/Camera2Basic

val captureRequest = camera.createCaptureRequest(
        CameraDevice.TEMPLATE_PREVIEW).apply { addTarget(fragmentCameraBinding.viewFinder.holder.surface) }

will be changed to

val captureRequest = camera.createCaptureRequest(
   CameraDevice.TEMPLATE_PREVIEW).apply { 
      addTarget(fragmentCameraBinding.viewFinder.holder.surface) 
      //addTarget(recordingSurface) 
}

but I also enable video stabilization if devices allow it for Camera2 API (usually annoying device manufactures allow it only for their native camera app)

val captureRequest = camera.createCaptureRequest(
    CameraDevice.TEMPLATE_RECORD
).apply {
    addTarget(fragmentCameraBinding.viewFinder.holder.surface)
    //addTarget(recordingSurface) 
    val modes =
        characteristics.get(CameraCharacteristics.CONTROL_AVAILABLE_VIDEO_STABILIZATION_MODES)
            ?: intArrayOf()
    if (CameraMetadata.CONTROL_VIDEO_STABILIZATION_MODE_ON in modes) {
        set(
            CaptureRequest.CONTROL_VIDEO_STABILIZATION_MODE,
            CameraMetadata.CONTROL_VIDEO_STABILIZATION_MODE_ON
        )
    }
}

Here's the code to setup capture request for taking photo when you press the button https://github.com/android/camera-samples/blob/main/Camera2Basic/app/src/main/java/com/example/android/camera2/basic/fragments/CameraFragment.kt#L338

val captureRequest = session.device.createCaptureRequest(
            CameraDevice.TEMPLATE_STILL_CAPTURE).apply { addTarget(imageReader.surface) }

Everything is OK here

So after I changed TEMPLATE_PREVIEW to TEMPLATE_RECORD and enabled video stabilization then devices which support video stabilization gives me the next error when I try to take a photo. Though camera preview and video recording works fine.

2022-04-01 23:00:31 1529-1762/? E/CameraFragment: Camera 0 error: (4) Fatal (device)
java.lang.RuntimeException: Camera 0 error: (4) Fatal (device)
	at com.example.android.camera2.basic.fragments.CameraFragment$openCamera$2$1.onError(CameraFragment.kt:296)
	at android.hardware.camera2.impl.CameraDeviceImpl$CameraDeviceCallbacks.notifyError(CameraDeviceImpl.java:1922)
	at android.hardware.camera2.impl.CameraDeviceImpl$CameraDeviceCallbacks.lambda$Sm85frAzwGZVMAK-NE_gwckYXVQ(Unknown Source:0)
	at android.hardware.camera2.impl.-$$Lambda$CameraDeviceImpl$CameraDeviceCallbacks$Sm85frAzwGZVMAK-NE_gwckYXVQ.accept(Unknown Source:8)
	at com.android.internal.util.function.pooled.PooledLambdaImpl.doInvoke(PooledLambdaImpl.java:258)
	at com.android.internal.util.function.pooled.PooledLambdaImpl.invoke(PooledLambdaImpl.java:182)
	at com.android.internal.util.function.pooled.OmniFunction.run(OmniFunction.java:77)
	at android.os.Handler.handleCallback(Handler.java:907)
	at android.os.Handler.dispatchMessage(Handler.java:105)
	at android.os.Looper.loop(Looper.java:216)
	at android.os.HandlerThread.run(HandlerThread.java:65)

So it won't take photo until you disable video stabilization. In this case preview, video recording and taking (capture) photos works fine

Another way to fix it is keep video stabilization enabled but use TEMPLATE_PREVIEW instead of TEMPLATE_RECORD, in this case it also works fine. But mb video stabilization doesn't even work in this case when template is set to preview instead of recording. Would be great to get an answer of Google Camera2 API developers about it.

But I have recording video feature in my app, I want video stabilization to be enabled and use TEMPLATE_RECORD to get good FPS. So annoying!

I don't know if this issue happens for all devices which allow video stabilization with Camera2 (exposed by manufactures for all developers and can be used for third party apps).

Usually many devices don't support video stabilization...

I tested Huawei Mate 20 RS, Huawei P30 and they have this issue, my users report it usually for Huawei, so I think most of Huawei devices have this issue.

Also my users report that in some cases their Samsung phones reboot when trying to take photos when it's TEMPLATE_RECORD and video stabilization enabled, so different devices behavior differently for such camera setup.

So from all the next ways to setup camera2 preview works 1, 2, 3 when taking a photo and 4 doesn't work:

1. TEMPLATE_PREVIEW + video stabilization off - (V)
2. TEMPLATE_PREVIEW + video stabilization on - (V), mb stabilization doesn't work in preview template
3. TEMPLATE_RECORD + video stabilization off - (V)
4. TEMPLATE_RECORD + video stabilization on - (X)

p.s. it's like my ten question on SO about Camera2 API, so many issues with that. Of course mb it's not really problem of API, and manufactures are in the wrong with implementing it correctly for their devices but who knows, mb something wasn't clear with API for them

You can test it with official camera2 sample app (change the code to use recording template and enable video stabilization) and with phones which exposes video stabilization for Camera2 API