media icon indicating copy to clipboard operation
media copied to clipboard
verified publisher icon androidx

AAC with channel config 0 fails to play from TS files

Open lawadr opened this issue 2 years ago • 4 comments

Version

Media3 main branch

More version details

a879bae1ee2c684e8100f50bcff39655d46a2e8d

Devices that reproduce the issue

Chromecast with Google TV HD (Android TV 12) Pixel 3a (Android 12) Emulator (Android 14)

Devices that do not reproduce the issue

None?

Reproducible in the demo app?

Yes

Reproduction steps

  1. Play the attached media in the demo app

Expected result

The media plays successfully

Actual result

Playback fails straight away, every time, with:

10-04 21:11:07.536  4775  4899 E MediaCodecAudioRenderer: Audio codec error
10-04 21:11:07.536  4775  4899 E MediaCodecAudioRenderer:   java.lang.IllegalStateException: queueInputBuffer() is valid only at Executing states; currently at Released state
10-04 21:11:07.536  4775  4899 E MediaCodecAudioRenderer:
10-04 21:11:07.536  4775  4899 E MediaCodecAudioRenderer:       at android.media.MediaCodec.native_queueInputBuffer(Native Method)
10-04 21:11:07.536  4775  4899 E MediaCodecAudioRenderer:       at android.media.MediaCodec.queueInputBuffer(MediaCodec.java:2831)
10-04 21:11:07.536  4775  4899 E MediaCodecAudioRenderer:       at androidx.media3.exoplayer.mediacodec.SynchronousMediaCodecAdapter.queueInputBuffer(SynchronousMediaCodecAdapter.java:145)
10-04 21:11:07.536  4775  4899 E MediaCodecAudioRenderer:       at androidx.media3.exoplayer.mediacodec.MediaCodecRenderer.feedInputBuffer(MediaCodecRenderer.java:1435)
10-04 21:11:07.536  4775  4899 E MediaCodecAudioRenderer:       at androidx.media3.exoplayer.mediacodec.MediaCodecRenderer.render(MediaCodecRenderer.java:864)
10-04 21:11:07.536  4775  4899 E MediaCodecAudioRenderer:       at androidx.media3.exoplayer.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:1079)
10-04 21:11:07.536  4775  4899 E MediaCodecAudioRenderer:       at androidx.media3.exoplayer.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:529)
10-04 21:11:07.536  4775  4899 E MediaCodecAudioRenderer:       at android.os.Handler.dispatchMessage(Handler.java:102)
10-04 21:11:07.536  4775  4899 E MediaCodecAudioRenderer:       at android.os.Looper.loopOnce(Looper.java:205)
10-04 21:11:07.536  4775  4899 E MediaCodecAudioRenderer:       at android.os.Looper.loop(Looper.java:294)
10-04 21:11:07.536  4775  4899 E MediaCodecAudioRenderer:       at android.os.HandlerThread.run(HandlerThread.java:67)
10-04 21:11:07.543  4775  4899 E ExoPlayerImplInternal: Playback error
10-04 21:11:07.543  4775  4899 E ExoPlayerImplInternal:   androidx.media3.exoplayer.ExoPlaybackException: MediaCodecAudioRenderer error, index=4, format=Format(1/257, null, null, audio/mp4a-latm, mp4a.40.2, -1, null, [-1, -1, -1.0, null], [0, 48000]), format_supported=YES
10-04 21:11:07.543  4775  4899 E ExoPlayerImplInternal:       at androidx.media3.exoplayer.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:608)
10-04 21:11:07.543  4775  4899 E ExoPlayerImplInternal:       at android.os.Handler.dispatchMessage(Handler.java:102)
10-04 21:11:07.543  4775  4899 E ExoPlayerImplInternal:       at android.os.Looper.loopOnce(Looper.java:205)
10-04 21:11:07.543  4775  4899 E ExoPlayerImplInternal:       at android.os.Looper.loop(Looper.java:294)
10-04 21:11:07.543  4775  4899 E ExoPlayerImplInternal:       at android.os.HandlerThread.run(HandlerThread.java:67)
10-04 21:11:07.543  4775  4899 E ExoPlayerImplInternal:   Caused by: androidx.media3.exoplayer.mediacodec.MediaCodecDecoderException: Decoder failed: c2.android.aac.decoder
10-04 21:11:07.543  4775  4899 E ExoPlayerImplInternal:       at androidx.media3.exoplayer.mediacodec.MediaCodecRenderer.createDecoderException(MediaCodecRenderer.java:1011)
10-04 21:11:07.543  4775  4899 E ExoPlayerImplInternal:       at androidx.media3.exoplayer.mediacodec.MediaCodecRenderer.render(MediaCodecRenderer.java:883)
10-04 21:11:07.543  4775  4899 E ExoPlayerImplInternal:       at androidx.media3.exoplayer.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:1079)
10-04 21:11:07.543  4775  4899 E ExoPlayerImplInternal:       at androidx.media3.exoplayer.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:529)
10-04 21:11:07.543  4775  4899 E ExoPlayerImplInternal:       ... 4 more
10-04 21:11:07.543  4775  4899 E ExoPlayerImplInternal:   Caused by: java.lang.IllegalStateException: queueInputBuffer() is valid only at Executing states; currently at Released state
10-04 21:11:07.543  4775  4899 E ExoPlayerImplInternal:
10-04 21:11:07.543  4775  4899 E ExoPlayerImplInternal:       at android.media.MediaCodec.native_queueInputBuffer(Native Method)
10-04 21:11:07.543  4775  4899 E ExoPlayerImplInternal:       at android.media.MediaCodec.queueInputBuffer(MediaCodec.java:2831)
10-04 21:11:07.543  4775  4899 E ExoPlayerImplInternal:       at androidx.media3.exoplayer.mediacodec.SynchronousMediaCodecAdapter.queueInputBuffer(SynchronousMediaCodecAdapter.java:145)
10-04 21:11:07.543  4775  4899 E ExoPlayerImplInternal:       at androidx.media3.exoplayer.mediacodec.MediaCodecRenderer.feedInputBuffer(MediaCodecRenderer.java:1435)
10-04 21:11:07.543  4775  4899 E ExoPlayerImplInternal:       at androidx.media3.exoplayer.mediacodec.MediaCodecRenderer.render(MediaCodecRenderer.java:864)
10-04 21:11:07.543  4775  4899 E ExoPlayerImplInternal:       ... 6 more

Media

Media will be emailed

Bug Report

  • [X] You will email the zip file produced by adb bugreport to [email protected] after filing this issue.

lawadr avatar Oct 04 '23 20:10 lawadr

The original MKV works. And when converting the TS back to an MKV, that MKV also works. The AAC streams are exactly the same in each container (copied with ffmpeg).

From limited debugging, it seems that the initializationData set inside the Format object (that is ultimately passed to MediaCodec inside a MediaFormat as "csd-0") differs between MKV and TS.

For this piece of media, "csd-0" receives {0x11, 0x80} for the TS but {0x11, 0x80, 0x04, 0xC8, 0x09, 0x00, 0x01, 0x08, 0xC8, 0x00, 0x00} for the MKV. As you can see, the first two bytes are the same. Hardcoding to use the MKV's data makes the TS play back perfectly.

The AdtsReader always builds this 2 byte version using the AacUtil.buildAudioSpecificConfig function, whereas the MKV and MP4 extractors grab this data block straight from the container itself.

Digging a bit deeper, it seems that these extra bytes are needed when the channel config is set to 0. A channel config of 0 means that the channel config is defined inside the AAC stream itself in a program config element (PCE). In fact, the extra bytes are merely the appended PCE payload!

What would be the best way to handle this? An "easy" solution would be for the AdtsReader to delay submitting the Format object to the output when the channel config is 0 until after it has found and appended a PCE from the AAC frame.

Note that the exception in MediaCodec's AAC decoder is thrown before MediaCodec ever receives any sample data from the stream! It happens during the first function call after the config and start. This indicates that it will always fail upon receiving a 2 byte "csd-0" when the channel config is 0, meaning all AAC streams with a channel config of 0 will fail when inside a TS container.

lawadr avatar Oct 04 '23 22:10 lawadr

Did you ever figure out how to fix this error? I'm getting it with an mkv that is probably not supported due to a codec but the Unexpected runtime error is not very helpful to users.

casolorz avatar Feb 04 '24 17:02 casolorz

Did you ever figure out how to fix this error? I'm getting it with an mkv that is probably not supported due to a codec but the Unexpected runtime error is not very helpful to users.

The problem doesn't happen when playing an MKV or MP4. Only TS files (e.g. from an HLS stream). Are you by any chance using software that's remuxing the MKV into an HLS? If so, then yes I have partially fixed it. See my PR #722.

lawadr avatar Feb 05 '24 01:02 lawadr

The problem doesn't happen when playing an MKV or MP4. Only TS files (e.g. from an HLS stream). Are you by any chance using software that's remuxing the MKV into an HLS? If so, then yes I have partially fixed it. See my PR #722.

No, I just have an mkv getting this error. Every so often it throws a codec error but most of the time this runtime error. When it gives a codec error it says index=0, format = Format(1, null, null, video/dolby-vision, hev1.08.06x -1, null, [1920,1080,23.976023, ColorInfo(BT2020, Limited range, ST2084 PQ, false)], [-1, -1]), format_supported=Yes

casolorz avatar Feb 05 '24 16:02 casolorz