ValveSoftware
C API: ITD/ILD support?
I couldn't see anything in the API about modelling ITD/ILD, is this planned or is there another technology that can be used in conjunction with steam audio to achieve it?
@chrisvarns - we perform HRTF based 3D audio modeling. It is a super of modeling ITD/ILD essentially. What are you trying to achieve with modeling ITD/ILD?
We're developing VR content, and are currently evaluating the tech and middleware available. I assumed hrtf was independent of itd and ild. Thinking about it i can see how the level differences would be built in, but not necessarily the time differences. I'll do some more reading.
I made a quick demo using the binaural effect, with a sound rotating around the listener, and i could tell it was working but the effect wasn't as obviously good as some of the videos i've seen showing it off, so figured there might be some extra effects i am missing. I should test it more with different sounds.
@chrisvarns - HRTF basically contains both the ITD and ILD information in it and other spectral content avoid cone of confusion.
It is hard for us to comment more on quality without seeing your implementation and resulting output.
I'll upload it tomorrow if you are interested, its only a couple hundred lines, but based on your comments im sure its working fine and i just need to run a bunch more content through it to thoroughly test it out. Thanks!
Your call. If you need us to look into something, we are happy to take a quick look and give it a listen if you want.
There's definitely multiple approximations involved, since the only data passed into iplApplyBinauralEffect() is a single unit vector. Obviously the vectors from a point source to the two ears will not be exactly equal in angle and rarely equal in distance, and these difference will increase the closer a sound gets to a listener.
Differences in distance should be easy to capture by inserting delays prior to HRTF. However you'd need to run the HRTF twice, once per ear, since iplApplyBinauralEffect() only takes a mono input signal. I tried this, and it certainly does add quite perceptible width to the sound.
The angular difference is only important when greater than the angular resolution of the HRTF data itself (I'm not sure if that's documented for phonon?). I was curious, so I tried computing distinct direction vectors per ear and rendering each as a different HRTF, picking the left channel of the left ear and the right channel of the right ear, and compared it with a normal single-direction/single-HRTF render. I put the sound source at a 45 degree angle from front. The difference was imperceptible at 3m distance, noticeable but hard to characterize at 1m, starts to become quite apparent at about 55cm, and quite strong at 25cm (which would be <15cm from your actual face). What it feels like is that when using a single direction vector the sound is still somewhat inside your head, whereas using a direction vector per ear the sound feels more clearly outside the head. But I wouldn't really consider it at all significant for sounds more than a meter away from the listener. Moreover if sound sources are significantly larger than the listener, the approaching waves may still be plane-like even at close distances. So its really only for very close and small sound emitters.
Still I think it is worth requesting an iplBinauralEffect that can accept a stereo input buffer representing the audio signals arriving at each ear as well as a direction vectors per ear.
Except that I just recalled that both delays and HRTF are linear time invariant, so it doesn't matter which order they happen -- time delays could be added to the outputs of the iplApplyBinauralEffect, so there's no need for stereo input. There is still the possible need for separate direction vectors per ear for nearby small sources though.