11.7, 12.X Conversions from extension methods to delegate types

[VladimirReshetnikov]

This issue was reported by Neal Gafter [email protected].

Consider this code:

using System;

static class S
{
    static void Main()
    {
        string s = "";
        Action a = s.Ext;
    }

    static void Ext(this string s) { }
}

It compiles both on Roslyn and Mono, and creates a delegate to the extension method Ext encapsulating the empty string as a value to be provided as the first argument to the extension method. Unfortunately, the spec does not explain why it should work this way.

Relevant parts of the spec are:

• §12.7 Method group conversions
• §13.2.1 Expression classifications — General
• §13.5.1 Member lookup — General
• §13.7.5.1 Member access — General
• §13.7.6.3 Extension method invocations

§12.7 Method group conversions An implicit conversion (§12.2) exists from a method group (§13.2) to a compatible delegate type. Given a delegate type D and an expression E that is classified as a method group, an implicit conversion exists from E to D if E contains at least one method that is applicable in its normal form (§13.6.4.2) to an argument list constructed by use of the parameter types and modifiers of D

Note that a prerequisite for applying rules from this section is to have an expression that is already classified as a method group. §13.2.1 Expression classifications — General gives the following definition:

An expression is classified as one of the following: <...> • A method group, which is a set of overloaded methods resulting from a member lookup (§13.5).

Do we know that s.Ext is a method group? Because s.Ext is syntactically a member-access, we need rules from §13.7.5.1 Member access — General to determine its meaning.

The member-access is evaluated and classified as follows: • If K is zero and E is a namespace... (NO) • Otherwise, if E is a namespace... (NO) • If E is a predefined-type or a primary-expression classified as a type... (NO) • If E is a property access, indexer access, variable (YES), or value, the type of which is T, and a member lookup (§13.5) of I in T with K type arguments produces a match... (NO) • Otherwise, an attempt is made to process E.I as an extension method invocation (§13.7.6.3). If this fails, E.I is an invalid member reference, and a binding-time error occurs.

First of all, how an expression of the form E.I could be a method invocation? Where is its argument list? OK, suppose it meant to say

• Otherwise, if E.I occurs as part of an invocation expression E.I(...), an attempt is made to process that expression as an extension method invocation (§13.7.6.3).

But even in this form this bullet would not apply in our case — we do not have an invocation. §13.7.6.3 requires an argument list to start searching for extension methods, but we do not have an argument list. Moreover, §13.7.6.3 does not mention any member lookup, but we know from §13.2.1 that a method group can only result from a member lookup. So, we have to conclude that s.Ext is not a method group and its processing according to §13.7.5.1 must result in a compile-time error.

We could try to save the situation by saying that §12.7 describes processing of an implicit conversion to a delegate type by constructing an imaginary method invocation with an argument list built based on the delegate signature, and this argument list can be used as an argument list required in §13.7.6.3. But as I said initially, the current wording in the spec requires that we already must have a method group as a prerequisite for the processing descibed in §12.7.

We need to fix the spec in several places if we want to break this loop.

[jskeet]

It also compiles with csc, with the same meaning (I believe).

[Nigel-Ecma]

This problem stems from proposed changes to the existing Standard introduced to cover extension methods. Unfortunately as this example shows these changes may not be rigorously and correctly defined in a way which meshes with the Standard.

A quick read suggests the issue is in the new text in either member lookup or member access, but I wouldn't trust that without a review of all the text added to support extension methods, and what the intended semantics are - "it compiles" doesn't tell you that. Given issues like this a piecemeal approach looking at each change in isolation could easily mask other problems.

This is a case where I think we probably need a topic, rather than linear, based approach to reviewing the changes.

[jskeet]

Assigning to Neal now that he's in the group to either:

• Come up with a proposal
• Determine that it would be just about bearable to leave this "extra feature" unspecified in C# 5, and change the milestone to C# 6

[jskeet]

Noting that the feature isn't completely unspecified (we refer to it in 12.8), we're choosing to punt this to C# 6.

[jskeet]

All comments prior to this one were before we removed section 7. I've updated the title only.

[Nigel-Ecma]

Let's consider this route through the clauses (11.8 is the old 12.7 referenced above):

11.8 Method group conversions ... The run-time evaluation of a method group conversion proceeds as follows: • If the method selected at compile-time is an instance method, or it is an extension method

So here we have extension methods capable of being part of a method group, which makes sense.

As before we go through 12.2.1 (new 13.2.1):

12.2.1 General ... • A method group, which is a set of overloaded methods resulting from a member lookup (§12.5).

To arrive at 12.5 (new 13.5) and in subclause 12.5.1 (new 13.5.1) we have:

A member lookup of a name N with K type arguments in a type T is processed as follows: • First, a set of accessible members named N is determined:

o If T is a type parameter, then the set is the union of the sets of accessible members named N in each of the types specified as a primary constraint or secondary constraint (§15.2.5) for T, along with the set of accessible members named N in object. o Otherwise, the set consists of all accessible(§8.5) members named N in T, including inherited members and the accessible members named N in object. If T is a constructed type, the set of members is obtained by substituting type arguments as described in §15.3.3. Members that include an override modifier are excluded from the set.

Is the phrase "N in T" where things do wrong? The current interpretation is that an extension method is not "in" T, but is it not "virtually in" T? Looking at what the compilers are doing they appear to be following this latter interpretation.

So maybe this last bullet could be:

o Otherwise, the set consists of all accessible(§8.5) members named N in T; including inherited members, extension members, and the accessible members named N in object. If T is a constructed type, the set of members is obtained by substituting type arguments as described in §15.3.3. Members that include an override modifier are excluded from the set.

Does this produce the desired semantics? Does it break anything?

It might break §12.7.6.3 Extension method invocations, as in making it partially redundant as "if the normal processing of the invocation finds no applicable methods" wouldn't occur if there was an applicable extension method. What would remain valid is the stuff about calling the static method. Here combining §12.7.6.2 Method invocations and the non-redundant parts of §12.7.6.3 could be the solution.

However an unexplored question is would including in member lookup the (virtual) extension members fundamentally change the result so that a different method would now be chosen?

This seems to be a big enough issue revolving around the addition of extension methods that it should be resolved as a priority before we move forward (feel free to counter-argue).

[gafter]

However an unexplored question is would including in member lookup the (virtual) extension members fundamentally change the result so that a different method would now be chosen?

Yes, it certainly would change the result. Here is an example:

using System;
public class C {
    static void Main()
    {
        new C().M(1);
    }
    
    public void M(long l)
    {
        Console.WriteLine("instance");
    }
}

public static class X
{
    public static void M(this C c, int i)
    {
        Console.WriteLine("extension");
    }
}

The extension method is a better candidate because of the exact match on the type of the argument.

[Nigel-Ecma]

On 2/07/2020, at 11:28 am, Neal Gafter [email protected] wrote:

Yes, it certainly would change the result. Here is an example:

Well that’s a fail then.

(There is little point in asking now whether the user might have intended the extension method to be the chosen candidate, I doubt the semantics are going to change.)

[KalleOlaviNiemitalo]

This also needs changes in 20.1 Delegates / General), which says

For instance methods, a callable entity consists of an instance and a method on that instance. For static methods, a callable entity consists of just a method.

leaving no possibility for a callable entity to consist of an extension method and its first argument. Similar in 20.5 Delegate instantiation.

(In .NET, the reflection API can also create a delegate that references a static non-extension method and its first argument, or an instance method without an instance. However, I don't think the C# standard should require those abilities.)

Also, the type must apparently be a reference type, rather than a value type or an unconstrained type parameter; this does not do the "boxing operation" mentioned in 10.8 Method group conversions:

using System;

public struct S {
}

static class Exts {
    static Action M(S arg) {
        return arg.N; // error CS1113
    }
    public static void N(this S arg) {}

    static Action O<T>(T arg) {
        return arg.P; // error CS1113
    }
    public static void P<T>(this T arg) {}
}