infer ExecutionEngineException in Unity3d IL2CPP iOS

Running code using Infer.NET fails with an ExecutionEngineException on iOS when built with IL2CPP from Unity3d. Some of the instantiations of generic types used internally don't exist during compile time which causes the corresponding iOS AOT code to be missing at runtime.

Details: Unity 2018.3 Microsoft.ML.Probabilistic.dll Microsoft.ML.Probabilistic.Compiler.dll

ExecutionEngineException: Attempting to call method 'Microsoft.ML.Probabilistic.Models.ModelBuilder::SearchExpression<System.Boolean>' for which no ahead of time (AOT) code was generated.
  at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.ML.Probabilistic.Utilities.Util.Invoke (System.Reflection.MethodBase method, System.Object target, System.Object[] parameters) [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.ML.Probabilistic.Models.ModelBuilder.Build (Microsoft.ML.Probabilistic.Models.InferenceEngine engine, System.Boolean inferOnlySpecifiedVars, System.Collections.Generic.IEnumerable`1[T] vars) [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.ML.Probabilistic.Models.InferenceEngine.BuildAndCompile (System.Boolean inferOnlySpecifiedVars, System.Collections.Generic.IEnumerable`1[T] vars) [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.ML.Probabilistic.Models.InferenceEngine.GetCompiledInferenceAlgorithm (System.Boolean inferOnlySpecifiedVars, Microsoft.ML.Probabilistic.Models.IVariable var) [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.ML.Probabilistic.Models.InferenceEngine.Infer (Microsoft.ML.Probabilistic.Models.IVariable var) [0x00000] in <00000000000000000000000000000000>:0 
  at Main.TwoCoins () [0x00000] in <00000000000000000000000000000000>:0 
  at Main.Start () [0x00000] in <00000000000000000000000000000000>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.ML.Probabilistic.Utilities.Util.Invoke (System.Reflection.MethodBase method, System.Object target, System.Object[] parameters) [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.ML.Probabilistic.Models.ModelBuilder.Build (Microsoft.ML.Probabilistic.Models.InferenceEngine engine, System.Boolean inferOnlySpecifiedVars, System.Collections.Generic.IEnumerable`1[T] vars) [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.ML.Probabilistic.Models.InferenceEngine.BuildAndCompile (System.Boolean inferOnlySpecifiedVars, System.Collections.Generic.IEnumerable`1[T] vars) [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.ML.Probabilistic.Models.InferenceEngine.GetCompiledInferenceAlgorithm (System.Boolean inferOnlySpecifiedVars, Microsoft.ML.Probabilistic.Models.IVariable var) [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.ML.Probabilistic.Models.InferenceEngine.Infer (Microsoft.ML.Probabilistic.Models.IVariable var) [0x00000] in <00000000000000000000000000000000>:0 
  at Main.TwoCoins () [0x00000] in <00000000000000000000000000000000>:0 
  at Main.Start () [0x00000] in <00000000000000000000000000000000>:0 
 
(Filename: currently not available on il2cpp Line: -1)```

May 18 '19 09:05 mjokipii

Does this error occur only on the device and not in the simulator? Also, do you have a small repro project?

On 18 May 2019, at 10:21, mjokipii [email protected] wrote:

Running code using Infer.NET fails with an ExecutionEngineException on iOS when built with IL2CPP from Unity3d. Some of the instantiations of generic types used internally don't exist during compile time which causes the corresponding iOS AOT code to be missing at runtime.

Details: Unity 2018.3 Microsoft.ML.Probabilistic.dll Microsoft.ML.Probabilistic.Compiler.dll

ExecutionEngineException: Attempting to call method 'Microsoft.ML.Probabilistic.Models.ModelBuilder::SearchExpression<System.Boolean>' for which no ahead of time (AOT) code was generated. at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00000] in <00000000000000000000000000000000>:0 at Microsoft.ML.Probabilistic.Utilities.Util.Invoke (System.Reflection.MethodBase method, System.Object target, System.Object[] parameters) [0x00000] in <00000000000000000000000000000000>:0 at Microsoft.ML.Probabilistic.Models.ModelBuilder.Build (Microsoft.ML.Probabilistic.Models.InferenceEngine engine, System.Boolean inferOnlySpecifiedVars, System.Collections.Generic.IEnumerable1[T] vars) [0x00000] in <00000000000000000000000000000000>:0 at Microsoft.ML.Probabilistic.Models.InferenceEngine.BuildAndCompile (System.Boolean inferOnlySpecifiedVars, System.Collections.Generic.IEnumerable1[T] vars) [0x00000] in <00000000000000000000000000000000>:0 at Microsoft.ML.Probabilistic.Models.InferenceEngine.GetCompiledInferenceAlgorithm (System.Boolean inferOnlySpecifiedVars, Microsoft.ML.Probabilistic.Models.IVariable var) [0x00000] in <00000000000000000000000000000000>:0 at Microsoft.ML.Probabilistic.Models.InferenceEngine.Infer (Microsoft.ML.Probabilistic.Models.IVariable var) [0x00000] in <00000000000000000000000000000000>:0 at Main.TwoCoins () [0x00000] in <00000000000000000000000000000000>:0 at Main.Start () [0x00000] in <00000000000000000000000000000000>:0 --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x00000] in <00000000000000000000000000000000>:0 at Microsoft.ML.Probabilistic.Utilities.Util.Invoke (System.Reflection.MethodBase method, System.Object target, System.Object[] parameters) [0x00000] in <00000000000000000000000000000000>:0 at Microsoft.ML.Probabilistic.Models.ModelBuilder.Build (Microsoft.ML.Probabilistic.Models.InferenceEngine engine, System.Boolean inferOnlySpecifiedVars, System.Collections.Generic.IEnumerable1[T] vars) [0x00000] in <00000000000000000000000000000000>:0 at Microsoft.ML.Probabilistic.Models.InferenceEngine.BuildAndCompile (System.Boolean inferOnlySpecifiedVars, System.Collections.Generic.IEnumerable1[T] vars) [0x00000] in <00000000000000000000000000000000>:0 at Microsoft.ML.Probabilistic.Models.InferenceEngine.GetCompiledInferenceAlgorithm (System.Boolean inferOnlySpecifiedVars, Microsoft.ML.Probabilistic.Models.IVariable var) [0x00000] in <00000000000000000000000000000000>:0 at Microsoft.ML.Probabilistic.Models.InferenceEngine.Infer (Microsoft.ML.Probabilistic.Models.IVariable var) [0x00000] in <00000000000000000000000000000000>:0 at Main.TwoCoins () [0x00000] in <00000000000000000000000000000000>:0 at Main.Start () [0x00000] in <00000000000000000000000000000000>:0

(Filename: currently not available on il2cpp Line: -1)``` — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

May 18 '19 12:05 a-vishar

This error occurs only when ran on a device that executes only AOT code like the IL2CPP compiled Xcode build on any iOS device. The library performs correctly in the editor that executes managed code. I have a small Unity3d repro. Where should i deliver the project?

May 18 '19 12:05 mjokipii

You can either attach it here (25MB limit) or use a third party host.

Please include a full set of steps required to repro.

I’m not sure this is something we can fix, it seems like it is either a limitation, or a bug in another tool. However, we can certainly take a look!

On 18 May 2019, at 13:09, mjokipii [email protected] wrote:

This error occurs only when ran on a device that executes only AOT code like the IL2CPP compiled Xcode build on any iOS device. The library performs correctly in the editor that executes managed code. I have a small Unity3d repro. Where should i deliver the project?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

May 18 '19 12:05 a-vishar

Yes, this is due to a known limitation of the IL2CPP compiler used in Unity3d and concerns only generic types with value type parameter. Please see:

https://stackoverflow.com/questions/56183606/invoke-generic-method-via-reflection-in-c-sharp-il2cpp-on-ios/56185978#56185978

Unfortunately that includes just about every use of Infer.Net.

I'll post the repro in a minute.

May 18 '19 12:05 mjokipii

InferAheadOfTimeException.zip

Unzip the InferAheadOfTimeException.zip archive
Open the contained folder in Unity3d
In Unity editor, open the Assets/Scenes/Main scene
Open the build settings from File/Build Settings
Select Platform iOS and click Switch Platform
Press Play button in editor
Observe correct output from Scripts/Main.cs in Console
Press Play button again to exit playmode in editor
Click Build button in the Build Settings window and choose the target location
After the build is finished, open the resulting Unity-iPhone.xcodeproj in Xcode
Build and run on an iOS device
Observe the reported error appear in the debug panel

Reproduced on iOS 12.1.4 and iOS 10.2.1 using Unity3d 2018.3.10f1 and Xcode 10.2.1.

May 18 '19 13:05 mjokipii

Hi there, any chance for a fix or a workaround to allow Infer.NET on Unity iOS?

Jun 22 '19 09:06 mjokipii

Hi! I see, the error happens during the compilation of the Infer.Net model. Do you actually need to run the compilation on the iOS device or, maybe, inference would be enough? In the latter case you could compile the model into C# code on your development machine, and then only put that code and Infer.Net runtime into the iOS code. This way all the necessary generic instantiations would probably (I don't have an iOS device anywhere nearby to check) exist at compile time.

See User guide on precompiling models

Jun 25 '19 17:06 msdmkats

That sounds promising. After a quick look to the instructions, if ican can update the model parameters and data without recompiling, i think inference is all i need in the iOS side. I will look into this and see if i can make this work. Thanks a lot!

Jun 28 '19 08:06 mjokipii