iOS 9.2 creates audio issue

[TheDucc]

After the release of iOS 9.2 the web audio frame work has a bug on this platform related to the sample rate being set incorrectly when the buffer is created. For more information see this post: http://www.html5gamedevs.com/topic/17326-ios-9-webaudio-issues/

The symptoms are audio playing to slow, or distorted because the sample rate is set to 48000 by default in iOS.

When you create the buffer you need to be able to set the default sample rate with creteBuffer. Something like this:

  var buffer = g_WebAudioContext.createBuffer(1, 1, 44100);
  var source = g_WebAudioContext.createBufferSource();
  source.buffer = buffer; 

So, could we talk about a patch too pass in the sample rate when you are creating the buffer in the options?

[TheDucc]

As part of my research (and a more eloquent explanation), this issues has a solution I have found here https://github.com/Jam3/ios-safe-audio-context

It is the source of the sample code I provided above.

[nehz]

Thanks, I'll have a look at it

[TheDucc]

I also tried my hand at adding new code

I added a 'desiredSampleRate' to the options object in function WebAudio()

if (options.desiredSampleRate === undefined) options.desiredSampleRate = 44100;

And then tried to re-write the buffer, as in the example code EDIT: I was forgetting to copy over the audio from the original buffer

    req.onload = function() {
      if (req.status !== 200 && audioBuffers[src]) return req.onerror();
      audioCtx.decodeAudioData(req.response, function(buffer) {
        if (desiredSampleRate != buffer.sampleRate) {
          var newBuffer = audioCtx.createBuffer(buffer.numberOfChannels, buffer.length, desiredSampleRate);
          var j = 0;
          var channelData = new Array(buffer.numberOfChannels);
          for (j = 0; j < buffer.numberOfChannels; ++j) {
            channelData[j] = buffer.getChannelData(j);
            newBuffer.copyToChannel(channelData[j], j, 0);
          }
          audioBuffers[src] = newBuffer;
        }else {
          audioBuffers[src] = buffer;
        }


        // Fire onBuffered event
        var handlers = eventHandlers[src].buffered;
        if (handlers) {
          // We can safely clean up all onBuffered event handlers, as once the
          // src media is cached, the onBuffered event can be fired immediately
          // for any new audio objects that are created henceforth
          eventHandlers[src].buffered = null;

          for (var i = 0; i < handlers.length; i++) {
            deferredApply(handlers[i]);
          }
        }
      });
    };

This gives much better results; but Sometimes the audio won't play. But I am not 100% sure that the intermittent play issue is related yet.

[nehz]

Can't reproduce it on iOS 9.1, will update my device over the weekend to test this

[nehz]

Tried using test.mp3 -> 32k and 44.1k re-sampled using Audacity on iOS (9.2.1), which seems to work fine. Can you share the audio that's causing issues? And have you tried the latest iOS update ?

Also I just pushed an update which should help with iOS playing issues, give that a shot to see if it fixes the media not playing issue.

[chrisschaub]

Reviewing this code, line 124 throws a can't set read only property error:

self.audioSrc.onended = null;

[nehz]

Nice catch, thanks. e446e6235d59e03766764f8d274c606d4ee80765 should fix it

[chrisschaub]

And now a new error: Error: Failed to execute 'stop' on 'AudioBufferSourceNode': cannot call stop without calling start first. at Error (native) at WebAudio.stop (http://localhost:8100/lib/ngWebAudio/src/ngWebAudio.js:127:47)

I should add that this code was working flawlessly before the ios9.2 update mess. The code that calls stop in this case is unchanged from when ngWebAudio was working pre 9.2. It just does a stop of any playing audio file (and one is playing).

[nehz]

Thanks! Sigh... WebAudio API is just horrible to work with. Pushed a new commit that should address this new bug hopefully fca6b1f12aee348678a3d01624a173205c1bcebe. Thanks for hanging in there!

[chrisschaub]

Thanks, error gone. The only issue I see now is that

onEnd

seems to get fired on "stop" as well as being at the end of the file. Thanks for your help, much appreciated!

[nehz]

That was the original intention of onEnd (e.g the event fires when ever the media "stops"; was a bug that prevent this, which I fixed along with other changes), however might refactor this into two events, onStop and onEnd? Let me know if you think it would be useful.

[chrisschaub]

I think both are valuable. Detecting a finish of play is a unique event in many ways.

On Tuesday, February 2, 2016, Zhen Wang [email protected] wrote:

That was the original intention of onEnd (e.g the event fires when ever the media "stops"; was a bug that prevent this, which I fixed along with other changes), however might refactor this into two events, onStop and onEnd? Let me know if you think it would be useful.

— Reply to this email directly or view it on GitHub https://github.com/nehz/ngWebAudio/issues/2#issuecomment-178954777.

Christopher Schaub http://chris.schaub.com

[nehz]

I've pushed an update for it and created a separate issue to keep track of any regressions or unintuitive behaviours #7

[chrisschaub]

Thanks, seems much better. Testing.

[chrisschaub]

Ok, there is still a distortion bug on iOS 9.2. The issue is compounded by the fact that copyToChannel is not available on iOS right now, even with the webkit prefix. This gets at the issue:

https://github.com/Jam3/ios-safe-audio-context/blob/master/index.js

[nehz]

For some reason I can't seem to reproduce the distortion (iPad2-32bit, iOS 9.2.1). Is it possible to get a copy your audio (and maybe code if possible)? Else I could just use the code linked, however I would like to avoid using desiredSampleRate as a workaround as ideally you shouldn't need to worry about it and the API shoud just "work"

[chrisschaub]

The issue is only on iOS 9.2, on a device. You won't hear it on desktop safari or in chrome, unfortunately.

On Thu, Feb 4, 2016 at 7:07 AM, Zhen Wang [email protected] wrote:

For some reason I can't seem to reproduce the distortion. Is it possible to get a copy your audio (and maybe code if possible), else I could just use the code linked. However I would like to avoid using desiredSampleRate as a workaround as ideally you shouldn't need to worry about it and the API shoud just "work"

— Reply to this email directly or view it on GitHub https://github.com/nehz/ngWebAudio/issues/2#issuecomment-179823673.

Christopher Schaub http://chris.schaub.com

[nehz]

Weird my iPad2 (32bit 9.2.1) does not seem reproduce this, however I can observe it on the iOS simulators (9.2). So it could either be 32bit vs 64bit, or iOS version difference. Do you happen to have any devices with 9.2.1 or 9.3 to see if the bug is still there?

I've tried replicating the safe context code both at init time (current implementation) and inside a touchend handler (wrangled all the init code and put it into play just for testing), and it does not seem to fix it on the simulator(could be workaround doesn't work on simiulator). I'll keep trying to poke at it. Really wish Apple would stop breaking things haha..

[chrisschaub]

I have a theory. A bunch of the WebAudio webkit functions are not present as of ios 9.2. I wonder if we should be trying just the non prefixed WebAudio api as of 9.2, maybe apple merged them into ios safari?

[chrisschaub]

Ok, tested. If you use change this line:

var AudioContext = window.AudioContext || window.webkitAudioContext;

to

var AudioContext = window.AudioContext;

you will notice distortion is gone.

[nehz]

Sadly, that's because it will fallback to html5 audio (AudioContext == undefined).

Did the ios-safe-audio-context code work for you? Might just add it to the code, or allow a way for users to replace the AudioContext (WebAudio.setContext()), so you use the ios-safe-audio-context lib as a dependency.

Wish Apple had a issue tracking for this bug ...

[chrisschaub]

Ahh, ok, I havent tried that yet. But would test if you did.

On Friday, February 5, 2016, Zhen Wang [email protected] wrote:

That's because it will fallback to html5 audio (AudioContext == undefined).

Did the ios-safe-context code work for you? Might just add it to the code, or allow a way for users to replace the AudioContext (WebAudio.setContext()), so you use the ios-safe-context lib as a dependency.

Wish Apple had a issue tracking this bug ...

— Reply to this email directly or view it on GitHub https://github.com/nehz/ngWebAudio/issues/2#issuecomment-180651018.

Christopher Schaub http://chris.schaub.com

[nehz]

It didn't work for me, but maybe because I only can test the bug with iOS simulators

[chrisschaub]

Yes, you don't see the bug on emulators or simulators. On an actual iPhone 6, iOS 9.2, the audio context sample rate gets set to 48800 -- even though the audio file is clearly 44100! I'm working on something like this ...

 audioCtx.decodeAudioData(req.response, function(buffer) {
        // ios as of 9.2 needs this to fix distortion - maybe just compare rates
        if (/(iPhone|iPad)/i.test(navigator.userAgent)) {
          var newBuffer = audioCtx.createBuffer(buffer.numberOfChannels, 1, desiredSampleRate);
          for (var j = 0; j < buffer.numberOfChannels; j++) {
            newBuffer.getChannelData(j).set(buffer.getChannelData(j), 0);
          } 
          audioBuffers[src] = newBuffer;
        } else {
          audioBuffers[src] = buffer;
        }

I can create a new buffer at 44100, but when i copy the data in, the player is still somehow thinking it's 48800.

[chrisschaub]

Sorry, bug ....

audioCtx.decodeAudioData(req.response, function(buffer) {
        // ios as of 9.2 needs this to fix distortion - maybe just compare rates
        if (/(iPhone|iPad)/i.test(navigator.userAgent)) {
          var newBuffer = audioCtx.createBuffer(buffer.numberOfChannels, buffer.length, desiredSampleRate);
          for (var j = 0; j < buffer.numberOfChannels; j++) {
            newBuffer.getChannelData(j).set(buffer.getChannelData(j), 0);
          } 
          audioBuffers[src] = newBuffer;
        } else {
          audioBuffers[src] = buffer;
        }

[chrisschaub]

Possible fix here: https://github.com/nehz/ngWebAudio/pull/8

[nehz]

decodeAudioData() should automatically resample to the device's sample rate (using audio ctx) (https://developer.mozilla.org/en-US/docs/Web/API/AudioContext/decodeAudioData)

I think the fix will be setting up the audio context correctly