Issue with setCustomAudioEffect when replacing audio buffer

[maddygoround]

Hi Pedro,

We are on a work around with display stream (with video)

to do this we start displaying the video on overlay with display stream and tries to get audio pcmbuffer from file stream's audio decoder, then replace the pcm buffer of display stream with file streams PCMBuffer in "setCustomAudioEffect" (Not starting the FileStream but only Audio Decoder).

Size of File stream's pcmBuffer is 4096, we are converting that into 5127 as that's the size of Display stream's pcmBuffer. Size of Display stream's pcmBuffer is 5127 in setCustomAudioEffect --> process

When tries to replace the pcmBuffer in setCustomAudioEffect --> process, process get stopped after 4 iterations, and then no audio get transmitted.

[maddygoround]

Hi Pedro,

Any suggestions ?

[pedroSG94]

Hello,

I did this branch to test it: https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/tree/i988

You can use it like this:

rtmpCamera1.setCustomAudioEffect(new CustomAudioEffect() {
      @Override
      public Frame process(Frame pcmFrame) {
        byte[] bufferModified = new byte[4096];
        //check if your buffer is valid to set.
        if (bufferModified.length <= pcmFrame.getSize()) {
          pcmFrame.setBuffer(bufferModified);
          pcmFrame.setSize(bufferModified.length);
          pcmFrame.setOffset(0); //if you have offset. use the correct value
        }
        return pcmFrame;
      }
    });

With this, the library should support buffers with length less or equal to original buffer. Let me know if it is working fine. If it is not working, share me the way you are getting audio from a file and the file used to reproduce the case.

[maddygoround]

Sure. Thanks. I will let you know the result.

[GauravSTripathi]

Hi Pedro, Thanks, Above modifications worked 👍

But there is some distortion in the file Audio with display Stream, I think it is because of the difference in PCMBuffer size. Please have a look at attached video, also please share your thoughts. Wondering if there is a way to make both buffer of same size?

Video: https://drive.google.com/file/d/1i56KYB_ARReMtk_99k_tn8uj2_51QUCC/view?usp=sharing

[pedroSG94]

I will think about it but in this case the distortion seems normal. Keep in mind that PCM buffer size is related with timestamp. Use that method to modify audio source is not the real objetive. It was designed to apply filters to PCM original buffer. I'm already working in a stream mode that allow you change audio and video source on fly. For now, the only workaround is get exactly the same PCM buffer size from file, you can try concatenate part of the next buffer with the current buffer to get the same size.

[GauravSTripathi]

Thanks for the quick reply and suggestion. Sure, Will try to add all the chunks in one list (Append as it comes) from the file stream's audio decoder, then get 5127 bytes from the list and see if it works (Without distortion). So basically instead of adjusting 4096 byte will take whole 5127 byte from the list and then remove the consumed data. Also will try to restart the file Stream audio decoder as soon as it finish the job, currently decoder stopes after one iteration.