RootEncoder icon indicating copy to clipboard operation
RootEncoder copied to clipboard
verified publisher icon pedroSG94

Audio & Video out of sync on Oppo

Open linhlq58 opened this issue 4 years ago • 9 comments

Audio & video get out of sync when using Camera1 on Oppo. Audio came faster than video after some minutes of livestream. I tested on multiple devices but this issue only happens on Oppo. Could you tell me how to fix it?

linhlq58 avatar Sep 23 '21 08:09 linhlq58

Hello,

  • Can you tell me model and API?
  • Which view are you using?
  • RTMP or RTSP? if RTSP, with TCP or UDP?
  • Did you test with other player and server?
  • If you are using RTMP, can you try use forceAkamaiTs(true)? call it before startStream

pedroSG94 avatar Sep 23 '21 08:09 pedroSG94

  • I'm testing on Oppo Reno 4. My customers who have this error also use Oppo but I don't have more information about model they used. All of them are API 29 & 30. I also tested on Xiaomi, Samsung and everything is fine.
  • I'm using GLSurfaceView. It is included in FaceUnity SDK. I just extract output YUV frames and send them through function videoEncoder.inputYUVData() in Camera1Base.
  • I'm using RTMP
  • I tested output on Facebook, Youtube and have the same result. About server, I only tested with my server
  • Yes, I will try to use forceAkamaiTs(true) before startStream.

linhlq58 avatar Sep 23 '21 09:09 linhlq58

Ok

I need 3 confirmation if forceAkamaiTs doesn't works:

  • Compile my app example and test with OpenGlView example. This is using VideoEncoder surface to buffer mode, it is to discard problems only with buffer to buffer mode.
  • Compile my app example and test with Default example. This is using VideoEncoder buffer to buffer mode like you are using in your implementation.
  • If the second test work and you haven't the last version of the library, update it and test your implementation

pedroSG94 avatar Sep 23 '21 09:09 pedroSG94

Ok

I need 3 confirmation if forceAkamaiTs doesn't works:

  • Compile my app example and test with OpenGlView example. This is using VideoEncoder surface to buffer mode, it is to discard problems only with buffer to buffer mode.
  • Compile my app example and test with Default example. This is using VideoEncoder buffer to buffer mode like you are using in your implementation.
  • If the second test work and you haven't the last version of the library, update it and test your implementation

pedroSG94 avatar Sep 23 '21 09:09 pedroSG94

  • Audio is distorted when using forceAkamaiTs(true)
  • OpenGlView example works normally
  • Default example works normally
  • After updating the library to 2.1.2, I tested with my implementation again and the out-sync still happened

linhlq58 avatar Oct 05 '21 03:10 linhlq58

  • Audio is distorted when using forceAkamaiTs(true)
  • OpenGlView example works normally
  • Default example works normally
  • After updating the library to 2.1.2, I tested with my implementation again and the out-sync still happened

linhlq58 avatar Oct 05 '21 03:10 linhlq58

So this is a problem that only happens in your implementation with Oppo device. This will be difficult to fix.

Default example should works similar to your implementation so the only difference could be that you have a delay in your implementation. Can you check Video/Audio packets timestamp to know if you have a similar delay?: https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/blob/master/rtplibrary/src/main/java/com/pedro/rtplibrary/base/Camera1Base.java#L896 https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/blob/master/rtplibrary/src/main/java/com/pedro/rtplibrary/base/Camera1Base.java#L896 To check timestamp in audio and video:

//Video
    Log.e("Video timestamp", "" + info.presentationTimeUs);
//Audio
    Log.e("Audio timestamp", "" + info.presentationTimeUs);

After that, share me logcat in the moment that you have out of sync and other logcat working fine to check the difference

pedroSG94 avatar Oct 07 '21 10:10 pedroSG94

So this is a problem that only happens in your implementation with Oppo device. This will be difficult to fix.

Default example should works similar to your implementation so the only difference could be that you have a delay in your implementation. Can you check Video/Audio packets timestamp to know if you have a similar delay?: https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/blob/master/rtplibrary/src/main/java/com/pedro/rtplibrary/base/Camera1Base.java#L896 https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/blob/master/rtplibrary/src/main/java/com/pedro/rtplibrary/base/Camera1Base.java#L896 To check timestamp in audio and video:

//Video
    Log.e("Video timestamp", "" + info.presentationTimeUs);
//Audio
    Log.e("Audio timestamp", "" + info.presentationTimeUs);

After that, share me logcat in the moment that you have out of sync and other logcat working fine to check the difference

pedroSG94 avatar Oct 07 '21 10:10 pedroSG94

@pedroSG94

This is the logcat when livestream is working fine: E/Audio timestamp: 59617243 E/Video timestamp: 59279817

And this is the logcat when livestream is out of sync: E/Audio timestamp: 660099908 E/Video timestamp: 659678784

As you can see, the difference of audio and video is about 0.4 second from the start. When livestream is out of sync, that difference still does not change.

linhlq58 avatar Oct 19 '21 09:10 linhlq58

@pedroSG94

This is the logcat when livestream is working fine: E/Audio timestamp: 59617243 E/Video timestamp: 59279817

And this is the logcat when livestream is out of sync: E/Audio timestamp: 660099908 E/Video timestamp: 659678784

As you can see, the difference of audio and video is about 0.4 second from the start. When livestream is out of sync, that difference still does not change.

linhlq58 avatar Oct 19 '21 09:10 linhlq58

Ok, I did a branch for this issue with a change in timestamp. Can you compile it in your Oppo, check if error persist and share me a logcat with filter "Pedro"?: https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/tree/i943 A extended logcat to know if out of sync increase with time or it is constant.

pedroSG94 avatar Oct 19 '21 09:10 pedroSG94

Here is the logcat when I started stream: 2021-10-21 16:08:17.116 22725-22725/com.botbanhang.botlive I/Pedro: creating ts: 510333833105 2021-10-21 16:08:17.168 22725-22725/com.botbanhang.botlive I/Pedro: ts already created: 510333833105 2021-10-21 16:08:17.471 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 354240 2021-10-21 16:08:17.486 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 370203 2021-10-21 16:08:17.517 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 400467 2021-10-21 16:08:17.527 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 411149 2021-10-21 16:08:17.544 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 428201 2021-10-21 16:08:17.566 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 449266 2021-10-21 16:08:17.567 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 450856 2021-10-21 16:08:17.568 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 452021 2021-10-21 16:08:17.569 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 453240 2021-10-21 16:08:17.571 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 454569 2021-10-21 16:08:17.572 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 456167 2021-10-21 16:08:17.574 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 457443 2021-10-21 16:08:17.574 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 457446 2021-10-21 16:08:17.575 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 458379 2021-10-21 16:08:17.576 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 459591 2021-10-21 16:08:17.664 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 547555 2021-10-21 16:08:17.698 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 581523 2021-10-21 16:08:17.752 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 635486 2021-10-21 16:08:17.765 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 649152 2021-10-21 16:08:17.767 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 650777 2021-10-21 16:08:17.778 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 661441 2021-10-21 16:08:17.808 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 691994 2021-10-21 16:08:17.809 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 692623 2021-10-21 16:08:17.830 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 713567 2021-10-21 16:08:17.847 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 730410 2021-10-21 16:08:17.848 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 731695 2021-10-21 16:08:17.849 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 732704

And here is the logcat when it is out of sync: 2021-10-21 16:21:00.676 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 763560040 2021-10-21 16:21:00.678 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 763561955 2021-10-21 16:21:00.712 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 763595809 2021-10-21 16:21:00.745 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 763628559 2021-10-21 16:21:00.779 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 763663045 2021-10-21 16:21:00.781 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 763665070 2021-10-21 16:21:00.842 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 763725689 2021-10-21 16:21:00.846 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 763729728 2021-10-21 16:21:00.869 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 763752464 2021-10-21 16:21:00.881 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 763765179 2021-10-21 16:21:00.914 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 763798002 2021-10-21 16:21:00.924 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 763807449 2021-10-21 16:21:00.926 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 763810235 2021-10-21 16:21:00.929 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 763812848 2021-10-21 16:21:00.931 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 763814664 2021-10-21 16:21:00.933 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 763817060 2021-10-21 16:21:00.935 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 763818271 2021-10-21 16:21:00.937 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 763821010 2021-10-21 16:21:00.940 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 763823825 2021-10-21 16:21:00.942 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 763826203

The error still persist. Additional, audio is laggy like this video: https://www.facebook.com/2204445623215564/videos/255327933076306/

linhlq58 avatar Oct 21 '21 09:10 linhlq58

Here is the logcat when I started stream: 2021-10-21 16:08:17.116 22725-22725/com.botbanhang.botlive I/Pedro: creating ts: 510333833105 2021-10-21 16:08:17.168 22725-22725/com.botbanhang.botlive I/Pedro: ts already created: 510333833105 2021-10-21 16:08:17.471 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 354240 2021-10-21 16:08:17.486 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 370203 2021-10-21 16:08:17.517 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 400467 2021-10-21 16:08:17.527 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 411149 2021-10-21 16:08:17.544 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 428201 2021-10-21 16:08:17.566 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 449266 2021-10-21 16:08:17.567 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 450856 2021-10-21 16:08:17.568 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 452021 2021-10-21 16:08:17.569 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 453240 2021-10-21 16:08:17.571 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 454569 2021-10-21 16:08:17.572 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 456167 2021-10-21 16:08:17.574 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 457443 2021-10-21 16:08:17.574 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 457446 2021-10-21 16:08:17.575 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 458379 2021-10-21 16:08:17.576 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 459591 2021-10-21 16:08:17.664 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 547555 2021-10-21 16:08:17.698 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 581523 2021-10-21 16:08:17.752 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 635486 2021-10-21 16:08:17.765 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 649152 2021-10-21 16:08:17.767 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 650777 2021-10-21 16:08:17.778 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 661441 2021-10-21 16:08:17.808 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 691994 2021-10-21 16:08:17.809 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 692623 2021-10-21 16:08:17.830 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 713567 2021-10-21 16:08:17.847 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 730410 2021-10-21 16:08:17.848 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 731695 2021-10-21 16:08:17.849 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 732704

And here is the logcat when it is out of sync: 2021-10-21 16:21:00.676 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 763560040 2021-10-21 16:21:00.678 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 763561955 2021-10-21 16:21:00.712 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 763595809 2021-10-21 16:21:00.745 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 763628559 2021-10-21 16:21:00.779 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 763663045 2021-10-21 16:21:00.781 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 763665070 2021-10-21 16:21:00.842 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 763725689 2021-10-21 16:21:00.846 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 763729728 2021-10-21 16:21:00.869 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 763752464 2021-10-21 16:21:00.881 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 763765179 2021-10-21 16:21:00.914 22725-23094/com.botbanhang.botlive I/Pedro: video ts: 763798002 2021-10-21 16:21:00.924 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 763807449 2021-10-21 16:21:00.926 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 763810235 2021-10-21 16:21:00.929 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 763812848 2021-10-21 16:21:00.931 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 763814664 2021-10-21 16:21:00.933 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 763817060 2021-10-21 16:21:00.935 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 763818271 2021-10-21 16:21:00.937 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 763821010 2021-10-21 16:21:00.940 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 763823825 2021-10-21 16:21:00.942 22725-23095/com.botbanhang.botlive I/Pedro: audio ts: 763826203

The error still persist. Additional, audio is laggy like this video: https://www.facebook.com/2204445623215564/videos/255327933076306/

linhlq58 avatar Oct 21 '21 09:10 linhlq58

I did other change. Can you test it and share me a logcat? https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/tree/i943

pedroSG94 avatar Oct 21 '21 10:10 pedroSG94