microsoft-graph-comms-samples Call Status is always 'Establishing' but never turned to established - Calling bot for Microsoft Team- Application Hosted Media

I am really new to bot framework, I am currently working on a bot that is supposed to receive incoming (peer-to-peer) calls from Microsoft Teams using [Graph Calling SDK] and [Real time media library] and play an audio file (stored in a folder in my application) and get details of incoming audio media stream. My application is basically a .net framework 4.7.2 console application and i am trying to follow 'AVPFrontend' and 'HueBot Samples. Now the problem is when i don't involve any media configuration, bot answers the call and call is never disconnected. But when i include media platform settings, call status is never turned to 'Established' but it's always 'Establishing'

Following are my media platform settings:

builder.SetNotificationUrl(new Uri(ConfigurationManager.AppSettings["BotBaseURI"] + "api/calling"));
            builder.SetMediaPlatformSettings(new MediaPlatformSettings()
            {
                MediaPlatformInstanceSettings = new MediaPlatformInstanceSettings()
                {
                    CertificateThumbprint = "1c9bfea46besdfjsdk234234230e217b821666cf3f6",
                    InstanceInternalPort = 8500,
                    InstancePublicPort = 10119,
                    InstancePublicIPAddress = new IPAddress(0x0),
                    ServiceFqdn = "0.bot.conntoso.com",

                },

                ApplicationId = AppID,
            }
            );
            builder.SetServiceBaseUrl(PlaceCallEndpointUrl);

I am using ngrok to test and debug ngrok locally, following are my settings:

Expected behavior The Call Status should turn to 'Established'

Additional context From AVPFrontEnd sample, I have implemented PlatformCallController.cs (which addresses incoming call) and DemoController.cs (this should handle webhooks once call is accepted, according to my understanding). but code in DemoController.cs is never executed. What am i missing, please help me to address this issue. Thanks!

May 20 '20 10:05 muradhaider5

It seems we cannot connect to your bot at the "0.bot.conntoso.com:10119" endpoint. Are you sure that "0.bot.conntoso.com" resolves to "0.tcp.ngrok.io"? After your bot service starts up and is ready to receive incoming calls, verify that TCP connectivity to "0.bot.conntoso.com:10119" is working; e.g., using Test-NetConnection -ComputerName 0.bot.conntoso.com -Port 10119 (substituting "0.bot.conntoso.com" with the value for MediaPlatformInstanceSettings.ServiceFqdn and 10119 with the value for InstancePublicPort.

May 21 '20 07:05 ssulzer

Thanks for your response. TCP test is succeeded: But Call's state is never established

May 21 '20 08:05 muradhaider5

Can you share a call-id or scenario-id of a test call please.

May 21 '20 15:05 ssulzer

just found that, I was using a self signed certificate which was causing the issue, once i used the certificate issued by the CA (let's encrypt), issue was resolved.

May 21 '20 15:05 muradhaider5

just an off-topic question btw, Can bot access the audio stream of a Caller in peer-to-peer incoming call?

May 21 '20 15:05 muradhaider5

Do you mean in a 1:1 call where there is just the Caller and the bot? Yes, the bot will receive the audio from the peer/caller.

Or do you mean in a case where person A has called person B in a 1:1 call? In that case, adding the bot to the conversation automatically turns it into a multiparty group call. In a multiparty conversation, by default the bot will hear a mix of the other active speakers. If the bot wishes to hear the active speakers separately, or 'unmixed', it should set ReceiveUnmixedMeetingAudio = true in its AudioSocketSettings. https://microsoftgraph.github.io/microsoft-graph-comms-samples/docs/bot_media/Microsoft.Skype.Bots.Media.AudioSocketSettings.html#properties

AudioMediaBuffer.UnmixedAudioBuffers property: https://microsoftgraph.github.io/microsoft-graph-comms-samples/docs/bot_media/Microsoft.Skype.Bots.Media.AudioMediaBuffer.html#Microsoft_Skype_Bots_Media_AudioMediaBuffer_UnmixedAudioBuffers

May 21 '20 17:05 ssulzer

Yes, I mean 1:1 Call with just caller and bot, can you please give a hint about accessing the incoming media stream from caller? Thanks!

May 22 '20 07:05 muradhaider5

And for my info, incoming call audio media buffers be accessed in AudioSocket.AudioMediaReceived event? Is there any way that i can receive only caller's media buffers not the bot's? (in the context of 1:1 call between a Caller and bot, where caller makes the call).

May 22 '20 08:05 muradhaider5

Yes, use the AudioSocket.AudioMediaReceived event to receive the audio media from the peer. The event will be raised 50 times per second (every 20 milliseconds). The event handler must consume each buffer and return quickly. Typically the bot app should just copy the audio frame content of each AudioMediaBuffer, enqueue the audio frame copy for asynchronous processing (from some other worker thread), dispose the AudioMediaBuffer object, and return. The bot will hear only the peer (caller) and not itself. The AudioMediaReceived event never includes any audio sent by the bot (so there's no echo).

May 22 '20 09:05 ssulzer

ok thanks, but one thing is confusing me, AudioSocket.AudioMediaReceived event is getting fired even if my microphone is muted, and its continuously returning the audio buffer.

May 22 '20 09:05 muradhaider5

If the peer is muted and not sending any audio content, the media SDK will generate buffers containing silence (aka "comfort noise") so the AudioSocket.AudioMediaReceived event continuously delivers a steady stream of audio frames to the bot app. Such generated silence buffers are indicated with the AudioMediaBuffer.IsSilence property. Thanks.

May 22 '20 18:05 ssulzer

@muradhaider5, @ssulzer Can you please help me, to run the ComplianceRecordingBot sample locally using ngrok to test and debug ngrok locally? or Can you please provide the steps to do it locally? I saw you are running sample locally using ngrok.

Thanks in advance, Jeetendra

Jun 15 '20 03:06 JSGund

@JSGund you may check this: https://docs.microsoft.com/en-us/microsoftteams/platform/bots/calls-and-meetings/debugging-local-testing-calling-meeting-bots

Jun 24 '20 14:06 muradhaider5

Riding in here to say if you've found this issue, your call may remain in "establishing" state if you've forgotten to update the settings around the URLs that have been configured. For example the bot base url. Check the logs, and double-check your config. (This might be a message for future me).

Sep 10 '20 07:09 irwinwilliams

Hi @ssulzer , I ran into a similar situation and require your help! The problem is when i don't involve any media configuration, bot answers the call and call is never disconnected. But when i include media platform settings, call status is never turned to 'Established' but it's always 'Establishing'

Following are my media platform settings: