UDP Server: An asynchronous socket operation is already in progress using this SocketAsyncEventArgs instance.

[dj-curls]

I am getting this error a couple of times a day from our server.

Any ideas where to start looking into this issue?

v5.0.10

System.InvalidOperationException: An asynchronous socket operation is already in progress using this SocketAsyncEventArgs instance.  
   at System.Net.Sockets.SocketAsyncEventArgs.ThrowForNonFreeStatus(Int32 status)
   at System.Net.Sockets.SocketAsyncEventArgs.SetBuffer(Byte[] buffer, Int32 offset, Int32 count)
   at NetCoreServer.UdpServer.TryReceive()
   at NetCoreServer.UdpServer.Send(EndPoint endpoint, Byte[] buffer, Int64 offset, Int64 size)
   at Shield.Service.CommsEngineCore.Transports.NetCoreServerUdpSession.Send(Byte[] buffer) in /home/vsts/work/1/s/Shield.Service.CommsEngineCore/Transports/NetCoreServerUdpSession.cs:line 75

(I notice someone using akka.net had a similar issue [https://github.com/akkadotnet/akka.net/issues/3572] - but I am unfamiliar with akka.net)

[dj-curls]

Getting the same exception thrown from a different stack as well now.

System.InvalidOperationException: An asynchronous socket operation is already in progress using this SocketAsyncEventArgs instance.
   at System.Net.Sockets.SocketAsyncEventArgs.ThrowForNonFreeStatus(Int32 status)
   at System.Net.Sockets.SocketAsyncEventArgs.SetBuffer(Byte[] buffer, Int32 offset, Int32 count)
   at NetCoreServer.UdpServer.TryReceive()
   at NetCoreServer.UdpServer.ProcessReceiveFrom(SocketAsyncEventArgs e)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Net.Sockets.SocketAsyncContext.OperationQueue`1.ProcessSyncEventOrGetAsyncEvent(SocketAsyncContext context, Boolean skipAsyncEvents, Boolean processAsyncEvents)
   at System.Net.Sockets.SocketAsyncContext.HandleEvents(SocketEvents events)
   at System.Net.Sockets.SocketAsyncEngine.System.Threading.IThreadPoolWorkItem.Execute()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()

[gribunin]

Same exception when sending data from parallel threads:

CoreCLR Version: 5.0.521.16609
.NET Version: 5.0.5
Description: The process was terminated due to an unhandled exception.
Exception Info: System.InvalidOperationException: An asynchronous socket operation is already in progress using this SocketAsyncEventArgs instance.
   at System.Net.Sockets.SocketAsyncEventArgs.ThrowForNonFreeStatus(Int32 status)
   at System.Net.Sockets.SocketAsyncEventArgs.SetBuffer(Byte[] buffer, Int32 offset, Int32 count)
   at NetCoreServer.UdpClient.TrySend()
   at NetCoreServer.UdpClient.SendAsync(EndPoint endpoint, Byte[] buffer, Int64 offset, Int64 size)
   at NetCoreServer.UdpClient.SendAsync(Byte[] buffer, Int64 offset, Int64 size)
   at SonicDispatcher.Services.BroadcastService.OnNtpTimer(Object target)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.TimerQueueTimer.CallCallback(Boolean isThreadPool)
   at System.Threading.TimerQueueTimer.Fire(Boolean isThreadPool)
   at System.Threading.TimerQueue.FireNextTimers()

[gribunin]

Obviously in my case this happens when I am using UdpClient.SendAsync from multiple threads. It seems that Updclient.SendAsync is not thread-safe (inside UdpClient implementation there is a _sending flag which is not thread-safe), so in my case the workaround is to marshal all calls to SendAsync to the same thread.

Not sure if it's mentioned anywhere in NetCoreServer documentation.

[chronoxor]

If you handle all the logic carefully in your code you are safe to use SendAsync: Recieve->SendAsync->Recieve->SendAsync Otherwise just use sync Send() method. For UDP it won't block the socket for long.

[dj-curls]

What about the original stack trace where Send() was being used?

[JocPelletier]

I experienced this issue, I have thousands of TCP clients sending data to thousands of UDP clients. Everything is Async and after a code review I see 2 potential issues:

1. UdpServer._sending is not thread safe, many threads can call my UdpServer.Send concurrently in my server, so I will have to add my own check. Also the flag is set to _sending = false at the beginning of ProcessSendTo, but the buffer is cleared at the end of the call. It's possible that another SendAsync() already appended data to the buffer and got cleared.

2. UdpServer._sendBuffer can be mixed between UdpClient! If a SocketError occur, the buffer is not cleared and the next SendAsync() call will append the data to existing buffer and will send that to the client. I've not experienced this but from code review that's what I see, am I wrong @chronoxor ?

To me UdpServer doesn't seems safe to use