ScriptProcessorNode is deprecated, need help with an AudioWorklet example

[davay42]

I'm using ToneJS in my web-app for synths and effects and it already establishes an AudioContext instance on it's own. In order to use it's effects I need to have AMY in the same context.

We can import { context } from 'tone', yet it's a fitted into modern web standard and there's no ScriptProcessorNode present there. Even context.rawContext!

I can see that it's playing a minor role in the whole HTML setup and it should be easily replaceable with AudioWorklet.

I'm still new to these low level concepts, so still just trying to figure out the exact setup. Here's what I have as reference:

• https://www.warpdesign.fr/webaudio-from-scriptprocessornode-to-the-new-audioworklet-api
• https://gist.github.com/louisgv/f210a1139d955baf511ff35f58fc8db1

Any help with a minimal example of that to be put into the www folder?

[davay42]

For now I just can't figure out what's happening at this line of code:

      scriptNode.onaudioprocess = function (audioProcessingEvent) {
        fn(audioProcessingEvent.outputBuffer.getChannelData(0));
      };

And how the buffer is actually connected to the synth. Help needed.

[davay42]

My explorations live here https://github.com/davay42/amy/tree/main/src/es6

[davay42]

@octetta May be you can help with that?

[octetta]

@octetta May be you can help with that?

I'll try to look at this, but I'm a newb on all things web audio.

A thought though, what if we could figure out how to add effects inside AMY?

I suspect the authors might be more interested in this, because it would work with their embedded stuff.

Could you share with me what basic effects you need from tone.js, as a design thought experiment?

[davay42]

Just reverbs and delays are enough to profoundly enrich the sound of the synth. And another thing is that implementing AudioWorklet glue we can eliminate any issues with clicks and audio freezes on low end hardware because we get rid of even touching the main JS thread.

[davay42]

I also got some pretty cool audio analysis tools and would like to integrate them in a future proof way, which AudioWorklets seem to be now. I tried to do it myself several times with AubioJs, yet from a web-designer perspective it's not that easy to start managing memory buffers and processor threads 😅

But I believe that once we find a solution , it can be implemented widely. Devs at Meyda could appreciate that too!

[octetta]

To be honest, I'm a C/embedded engineer, and have poor skills in the web domain (aside from creating ugly monster code when I need a quick UI for exotic hardware control). I've been very interested in the Erlang and Elixir ecosystem, and will write up a simple Elixir + AMY integration article soon to foster interest and sharing.

[octetta]

I've not looked into this yet, but while messing around with this article, I noticed that the sound library I introduced here (miniaudio) has its own Web Audio backend that might be cleaner than what Brian/DAn did.

I'll try to look at it after I get caught up with $DAYJOB.

Also, a thought experiment: I wonder if it's possible generate RTSP stream (or some other subscribe-able audio format) directly from AMY?

[bwhitman]

Love this article @octetta !

I have to admit to being out of my league on web audio stuff. I got it working just enough for the demo, but would love help modernizing it so others can use it easier.

[bwhitman]

@davay42 @octetta check out the latest AudioWorklet example, in the docs folder here and at https://shorepine.github.com/amy -- works great for me on Chrome & Safari. Not yet on iOS, but i think that's because i'm not handling the first onClick right to give permission to play audio.

[davay42]

Great progress! Makes AMY future-proof and quite promising in web context. One last thing lacking is ES6 module export, that is easy to fix with a single export amyModule line at the end of the file. It enables us to import the script (async, exactly where needed and able to be bundled with our apps). I might use the modified version in my code, but we actually might collaborate to release it as a proper NPM package if you're interested. I'm currently enjoying https://github.com/elemaudio/elementary with their wasm build that's readily usable in JS environment (and no need bother with the wasm files - they're simply embedded in JS).

Thanks for your update! @bwhitman ✊