CppSharp icon indicating copy to clipboard operation
CppSharp copied to clipboard

wrap template with variable arguments to C#

Open AhmedZero opened this issue 4 years ago • 37 comments

Brief Description

continue to previous issue #1640 , now we need to parse the template with variable arguments and wrap it to C#

template<std::size_t N>
class DLL_API FloatArrayF
{
public:
    template<typename... V, class = typename std::enable_if_t<sizeof...(V) == N>>
    FloatArrayF(V... x)  { }

};

OS: Windows 11 (dotnet 5.0, last commit of CPPSharp)

Used settings

Target: C#

AhmedZero avatar Nov 29 '21 11:11 AhmedZero

This is quite difficult because there's no direct correspondence to C#. We have considered mapping such non-type template parameters to regular parameters in a constructor but the idea is still vague and it would take some effort to implement. However, if it's too important to you, we can start a discussion about it.

ddobrev avatar Nov 29 '21 11:11 ddobrev

ok, let's start.

AhmedZero avatar Nov 29 '21 13:11 AhmedZero

The biggest problem - to a fully automated solution - is that any specialisations we use must be prebuilt. This is because templates in C++ are resolved at the time of compilation unlike generics in C# which are resolved at run-time. What we do for type template parameters is that we scan the public API of the wrapped library and generate a C++ file which exports all found specialisations. We can extend this logic to include non-type template parameters but then we must warn users to only use those numeric non-type template parameters already used in the original C++. That is, if the latter only uses N = 2 and N = 4, 1 or 3 would cause a crash. If you understand the problem, please tell me: do you need freely selectable N-s or are a few predefined enough?

ddobrev avatar Nov 29 '21 14:11 ddobrev

I understand the problem, and I need freely selectable N-s. I know it's not easy and takes a long time to wrap without problems.

AhmedZero avatar Nov 29 '21 14:11 AhmedZero

This makes matters much more difficult - maybe even impossible with automatic code. Do you have the option of changing the C++ itself - replacing N with a regular field?

ddobrev avatar Nov 29 '21 14:11 ddobrev

I know it's the crazy way. we can write inline assembly code in C# depending on the templates in C++ and in C# we construct the best and fit way code to call them.

AhmedZero avatar Nov 29 '21 14:11 AhmedZero

Assembly code in C#? I didn't know it was possible, or perhaps I'm misunderstanding. Can you please write some pseudo-code for my clarity?

ddobrev avatar Nov 29 '21 14:11 ddobrev

like that and we can write assembly code with https://github.com/icedland/iced/tree/master/src/csharp/Intel

        [SuppressUnmanagedCodeSecurity]
        [UnmanagedFunctionPointer(CallingConvention.Cdecl)]
        private delegate int AssemblyAddFunction(int x, int y);

        [DllImport("kernel32.dll")]
        private static extern bool VirtualProtectEx(IntPtr hProcess, IntPtr lpAddress, UIntPtr dwSize, uint flNewProtect, out uint lpflOldProtect);

        ....................................

        byte[] assembledCode =
        {
            0x55,               // 0 push ebp            
            0x8B, 0x45, 0x08,   // 1 mov  eax, [ebp+8]   
            0x8B, 0x55, 0x0C,   // 4 mov  edx, [ebp+12]  
            0x01, 0xD0,         // 7 add  eax, edx       
            0x5D,               // 9 pop  ebp            
            0xC3                // A ret                 
        };

        int returnValue;
        unsafe
        {
            fixed (byte* ptr = assembledCode)
            {
                var memoryAddress = (IntPtr) ptr;

                // Mark memory as EXECUTE_READWRITE to prevent DEP exceptions
                if (!VirtualProtectEx(Process.GetCurrentProcess().Handle, memoryAddress,
                    (UIntPtr) assembledCode.Length, 0x40 /* EXECUTE_READWRITE */, out uint _))
                {
                    throw new Win32Exception();
                }

                var myAssemblyFunction = Marshal.GetDelegateForFunctionPointer<AssemblyAddFunction>(memoryAddress);
                returnValue = myAssemblyFunction(10, -15);
            }               
        }

AhmedZero avatar Nov 29 '21 15:11 AhmedZero

So this library of iced will give you the correct ASM to execute for a certain specialisation of FloatArrayF with a certain N?

ddobrev avatar Nov 29 '21 15:11 ddobrev

yes, I think ... we need to know how to call the FloatArrayF template in ASM code because it's standard in ASM code.

AhmedZero avatar Nov 29 '21 15:11 AhmedZero

If you like, give it a try. We cannot help directly, however. I am not sure this can work and I cannot invest the time to verify either way. I am not even sure it can become standard enough for inclusion in C++#. For example, it seems this library only works with x86 ASM while we try to support other architectures such as ARM as well. What we can do is give you directions and get you acquainted with our code so that you experience less difficulties working.

ddobrev avatar Nov 29 '21 15:11 ddobrev

there is alternatively called Capstone http://www.capstone-engine.org/ it supports ARM, ARM-64 (aka ARMv8/AArch64), Mips, PowerPC, Sparc, SystemZ, XCore, and X86 (16-bit, 32-bit & 64-bit). anyway, I test to write code with ASM and write the template and tell you the results.

AhmedZero avatar Nov 29 '21 15:11 AhmedZero

Excellent, I will definitely be happy to see them.

ddobrev avatar Nov 29 '21 15:11 ddobrev

after analysis, there is no APIS for FloatArrayF with freely selectable N-s in Exports Table. image

AhmedZero avatar Nov 30 '21 14:11 AhmedZero

In such a case it might be doable. As I said, non-type template parameters (N) would be converted to regular parameters passed in the constructor. In this case we'll need a check if the value passed is supported (one of yours in this screen-shot) and then proceed. More details will follow if you'd like to try implementing this feature.

ddobrev avatar Nov 30 '21 15:11 ddobrev

Ok. Let's try

AhmedZero avatar Nov 30 '21 17:11 AhmedZero

You already have a test case: yours. Now you need to have it generated and called from a unit test to ensure it actually works. This involves 3 steps. The first step is to stop ignoring of those kinds (integral) of non-type template parameters you want to support: search for NonTypeTemplateParameter in our code and you'll find the piece. Next, you'll need a pass which transforms such templates into a mappable entities. In this pass you need to remove any integral non-type template parameters and replace them with additional numeric parameters in each constructor. You have 2 options for this: actually modify the existing types or create new classes and ignore the old ones. Either has significant disadvantages so up to you. The final step is just to generate the binding and call your newly generated class in a unit test - to verify it crashes when it should, doesn't crash when it shouldn't and returns proper values.

ddobrev avatar Nov 30 '21 18:11 ddobrev

the first step: i think that code used to ignore NonTypeTemplateParameter.

   if (@class.TemplateParameters.Any(param => param is NonTypeTemplateParameter))
                foreach (var specialization in @class.Specializations.Where(s => s.IsGenerated))
                    specialization.ExplicitlyIgnore();

there is any other code to disable it to start to read them???

AhmedZero avatar Dec 01 '21 09:12 AhmedZero

Yes, this is the piece. I don't think there's any other. In case you still don't get it generated, go to Declaration.GenerationKind and add a setter like this:

set
{
    if (Name == "mine" /* USR == "mine" */)
    {
        System.Diagnostics.Debugger.Break();
    }
    base.GenerationKind = value;
}

This way if something tried ignoring your declaration, you'll stop there automatically and the stack trace will tell you what's happening.

ddobrev avatar Dec 01 '21 10:12 ddobrev

I think that is the problem, it ignores

  1. "FloatArrayF<N>" ------- "c:@ST>1#Nk@FloatArrayF@FT@>1#pTFloatArrayF#Pt1.0#v#"
  2. "FloatArrayF<6> [ImplicitInstantiation]" ["ClassTemplateSpecialization"] ----- "c:@S@FloatArrayF>#Vk6@FT@>1#pTFloatArrayF#Pt0.0#v#"
            if (decl.TemplatedFunction.IsDependent && !decl.IsExplicitlyGenerated)
            {
                decl.TemplatedFunction.GenerationKind = GenerationKind.None;
                Diagnostics.Debug("Decl '{0}' was ignored due to dependent context",
                    decl.Name);
                return true;
            }

AhmedZero avatar Dec 01 '21 15:12 AhmedZero

I've rechecked your original code. This is an unrelated issue, I'm afraid - templated functions. We only support templated types at present. If you need such functions, then you'd have to implement not one but two features: integral template parameters and templated functions.

ddobrev avatar Dec 01 '21 15:12 ddobrev

ok, I want to contribute to making cppsharp better. I know it's not easy. anyway, we know "FloatArrayF<N>" ------- "c:@st>1#Nk@FloatArrayF@FT@>1#pTFloatArrayF#Pt1.0#v#" that it ignores to add to Export Table of Library. the other one it's added.

AhmedZero avatar Dec 01 '21 15:12 AhmedZero

I understand. I am moved by your commitment and excitement and ready to help you along the way, these really are important features. My suggestion is to first proceed with the integral template parameters since we've already put some thought into it. You could add a regular function to your test and then work to expose the class itself. Once this works, we can close it and start with the second feature of templated functions. What are your thoughts?

ddobrev avatar Dec 01 '21 15:12 ddobrev

Ok, let's start with the first feature.

AhmedZero avatar Dec 01 '21 16:12 AhmedZero

What help do you need at present?

ddobrev avatar Dec 01 '21 16:12 ddobrev

I need both :D. but start step by step, I'm not in a hurry. let's start with integral template parameters.

AhmedZero avatar Dec 01 '21 16:12 AhmedZero

OK. So, you have the test, you've found where non-type template parameters are ignored. First of all you need to stop ignoring these types of non-type template parameters you want to support: namely, integral. Check NonTypeTemplateParameter, it has a property named similar to Kind, an enum. When done, you need the new pass I described earlier. When you get there, we'll talk more about creating passes.

ddobrev avatar Dec 01 '21 16:12 ddobrev

I think that it ignores the class

     Declaration decl = null;
            if (specialization.Arguments.Any(a =>
                a.Kind != TemplateArgument.ArgumentKind.Type ||
                (a.Type.Type.TryGetDeclaration(out decl) == true) &&
                 decl.Ignore))
            {
                specialization.ExplicitlyIgnore();
                return false;
            }

AhmedZero avatar Dec 01 '21 23:12 AhmedZero

I'm not sure what you mean, is this a question? You can see in here the Kind I mentioned, you need to extend the logic to allow integral template parameters as well.

ddobrev avatar Dec 02 '21 13:12 ddobrev

you mean the DeclarationKind called NonTypeTemplateParm. sorry for the misunderstand.

case DeclarationKind.NonTypeTemplateParm:
                  {
                      var _decl = NonTypeTemplateParameter.__CreateInstance(decl.__Instance);
                      return VisitNonTypeTemplateParameter(_decl);
                  }

AhmedZero avatar Dec 02 '21 17:12 AhmedZero