pedroSG94
Inconsistent FPS during recording to mp4
Describe the bug We have seen footage recorded have a transient bug which causes the FPS to drop while recording to an MP4 file. We are using h264 encoding w/ 1280x720p @ 30fps (android fps Range [30, 30]) w/ 4Mbps bitrate with AAC audio encoding @ 96Kbps bitrate. Normally, all the footage recorded follows the consistent 30fps setting, with ffprobe reporting ~29.82 fps or similar values. However, footage which have the bug present, will report an average fps from ffprobe of ~22fps or even as low as ~5 or ~6fps, which is below even the variable FPS range supported on our device (the lowest FPS range with a 30fps maximum is [7, 30]). This has become a problem because when footage is finished being recorded, the audio and video appear out of sync. For now, we have a process which detects footage with this bug, and simply trans-codes it to keep both tracks in sync by inserting duplicate frames. This makes the footage appear choppy or slowed, which is not ideal for our customers.
I know this is not a great description in terms of debugging, but we have not been able to reliably reproduce the problem yet. We are still trying, but currently we have only seen the issue on customer devices, and we cannot provide their footage as example. We are working to get example footage of our own, and to narrow down steps to reproduce the bug.
The devices which are recording the footage are all the same model of device, so this is not a device variation issue. The device runs AOSP version 9 (SDK 28).
Expected behavior Recorded footage should maintain avg. fps @ 30 ([30, 30] on android).
Smartphone (please complete the following information):
- Library version: 2.5.5 & 2.6.0
- Device: Cammpro C8
- OS: Android 9
- Media server: nginx-rtmp
- Class used: GenericStream w/ Camera2Source.
Hello,
It is really weird. Did you try using the device with low battery without connect the device to charge? Maybe the problem is related with the battery saver mode.
I have confirmed that the devices were not at low battery at the time, often they are >50% battery, and we have even seen this at >80% battery, which i would not expect any throttling from.
I have just emailed you an example video where the bug is present. The FPS will drop and then correct later. We removed the audio track for customer privacy, but the audio is fine. I am happy to add any profiling or logging that could help troubleshoot, but its a very transient bug and we still haven't nailed down a way to reproduce it. We thought it may be related footage that has the monochrome filter applied, but it has also happened on footage without the filter applied.
Could the triple-A algorithms of the device cause this? Like if the lighting conditions are changing frequently, and the camera is adjusting auto-focus, auto white-balance, and auto exposure, would we expect the FPS to drop like that?
@NerdySouth Check the temperature of the product, it is likely to rise too much and reduce performance.
