Cannot run program "Microsoft.Spark.Worker": error=2, No such file or directory when using debug

[selvavm]

I have setup dotnet Spark in MacOS by following MacOS Getting Started. I am able to debug MyJob from Visual Studio by running below command,

spark-submit \
  --class org.apache.spark.deploy.dotnet.DotnetRunner \
  --master local \
  ~/csharp/MyJob/bin/Debug/net6.0/microsoft-spark-3-0_2.12-2.1.1.jar \
  debug

When I added a UDF, I got below exception in the terminal window, Cannot run program "Microsoft.Spark.Worker": error=2, No such file or directory

I was surprised and after trying different combinations, I figured out removing debug from spark-submit works as expected.

spark-submit \
  --class org.apache.spark.deploy.dotnet.DotnetRunner \
  --master local \
  ~/csharp/MyJob/bin/Debug/net6.0/microsoft-spark-3-0_2.12-2.1.1.jar \
  dotnet ~/csharp/MyJob/bin/Debug/net6.0/MyJob.dll

Below is my environment variables,

export SPARK_HOME=~/spark-3.0.1-bin-hadoop2.7/
export DOTNET_WORKER_DIR=~/Microsoft.Spark.Worker-2.1.1/
export DOTNET_ASSEMBLY_SEARCH_PATHS=~/csharp/MyJob/bin/Debug/net6.0/
export DOTNET_WORKER_DEBUG=1 
export PATH="$SPARK_HOME/bin:~/Microsoft.Spark.Worker-2.1.1/:/usr/local/opt/node@16/bin:$PATH"