Exceptions thrown when Python.Runtime sees dynamic assemblies

[dmitry-a-morozov]

@dsyme described an issue with dynamic assemblies in comments here: https://github.com/fsprojects/FSharp.Interop.PythonProvider/blob/master/src/PythonTypeProviderProxy/Program.fs#L11

To fix the issue following code in Python.Runtime

https://github.com/pythonnet/pythonnet/blob/develop/pythonnet/src/runtime/assemblymanager.cs#L90

    static void AssemblyLoadHandler(Object ob, AssemblyLoadEventArgs args){
        Assembly assembly = args.LoadedAssembly;
        assemblies.Add(assembly);
        ScanAssembly(assembly);
    }

Needs to be replace with:

    static void AssemblyLoadHandler(Object ob, AssemblyLoadEventArgs args){
        Assembly assembly = args.LoadedAssembly;
        assemblies.Add(assembly);
        if ( !(assembly is System.Reflection.Emit.AssemblyBuilder) )
        {
            ScanAssembly(assembly);
        }            
    }

That said I still agree with Don that going forward it's better to have Python runtime out-of-process proxy for stability and 32 vs 64 bits issue reasons.

But I thought issue still can be opened maybe to start discussion on in-process vs out-process TP hosting.

It will be interesting to hear from guys who worked on R Type Provide (@hmansell & Co). Because I believe they dealt with same kind of problem.

[hmansell]

I assume you are looking for input specifically on the in-process vs out-of-process thing. With the R type provider, I chose to make everything in-process because I wanted to have a very rich but low-overhead way of exploring the values that are returned by calling R functions. Fortunately, the developer of RDotNet had done most of the work already. Calling an R function does not marshal anything back - it all stays in native R form. You can traverse and decompose the result easily using the RDotNet API, without any overhead of marshalling results back.

We also have a extensible conversion layer for converting the whole result into some native .NET type, which is very useful and important. This is what allows you to do things like pass a Deedle frame directly into an R function, and convert the result of some R function that returns a data.frame into a Deedle Frame.

Partly, this a conceptual viewpoint. If you look at the R Provider as a way of interacting with some R session, then you might expect that session to run out-of-process or remotely. If you look at it as a way of calling into R libraries, then it's more likely you would want it to run in-process.

That said, I think there is value in being able to run out-of-process, so I have a high-level design of some mechanism for running R functions in a separate/remote process, with values living in the external process being represented with some separate type from the local SymbolicExpression. As it stands, I still expect a local R DLL and local SymbolicExpressions for interacting with the results, so this doesn't avoid an R session in the process. But it is useful for the 'automate a remote session' use case.

[dmitry-a-morozov]

@hmansell Thanks for your valuable feedback. We'll continue discussion on the upcoming call of F# Data Science group.

[ghost]

Note the metadata is currently acquired out-of-process, the runtime is in-process.

[hmansell]

That's probably a good way of doing it, to avoid potentially crashing Visual Studio!