Programatically added method selector is ignored if it has Priority 10

[l-1squared]

TestNG Version

7.4.0

Expected behavior

Addition of a method selector with priority 10 filters methods

Actual behavior

method selector gets ignored

Is the issue reproductible on runner?

• [ ] Shell
• [ ] Maven
• [x] Gradle
• [ ] Ant
• [ ] Eclipse
• [ ] IntelliJ
• [ ] NetBeans

Test case sample

package com.tngtech.testng.error.example;

import org.testng.IMethodSelector;
import org.testng.IMethodSelectorContext;
import org.testng.ITestNGMethod;
import org.testng.TestNG;
import org.testng.annotations.Test;

import java.util.List;

public class Application {

    public static void main(String[] args){
        TestNG runningWithSelectorWithPriorityEleven = runWithPriority(11); //will run only doNothing
        TestNG runningWithSelectorWithPriorityTen = runWithPriority(10); //will run both test methods

        if (runningWithSelectorWithPriorityEleven.hasFailure() != runningWithSelectorWithPriorityTen.hasFailure()){
            throw new AssertionError("Expected both runs to be equal, but they were not");
        }

    }

    private static TestNG runWithPriority(int priority){
        TestNG testng = new TestNG();
        testng.setTestClasses( new Class<?>[] { DummyTestClass.class} );
        testng.addMethodSelector( MethodSelector.class.getName(), priority );
        testng.run();
        return testng;
    }

    public static class MethodSelector implements IMethodSelector {
        @Override
        public boolean includeMethod(IMethodSelectorContext context, ITestNGMethod method, boolean isTestMethod) {
            return method.getMethodName().equals("doNothing");
        }

        @Override
        public void setTestMethods(List<ITestNGMethod> testMethods) {}
    }

    public static class DummyTestClass {

        @Test
        public void throwError(){
            throw new AssertionError("I shall fail");
        }

        @Test
        public void doNothing(){
            assert true;
        }
    }
}

[l-1squared]

In transition from version 7.3.0 to 7.4.0 the class org.testng.internal.RunInfo was changed. Specifically the type of the field m_methodSelectors was changed from a List<MethodSelectorDescriptor> to a TreeSet<MethodSelectorDescriptor>.

The class MethodSelectorDescriptor implements Comparable, where the compare method only takes the method selector's priorites into account. This compare method is used by the TreeMap backing TreeSet to determine how to order various Entries. If comparator resolves to equal, the Map replaces the value under the Key, which, in case of Tree Set, is a dummy.

TestNG generates an XmlMethodSelector from somewhere, that is always added first to the RunInfo instance and thus to the tree set that is implemented therein. Selectors that were defined manually via TestNG::addMethodSelector are added later to the instance. However, if they happen to have the same priority as that XmlMethodSelector, the tree map will regard them as equal to it. This is in fact true for any selector that is already contained in the tree set. Hence, they will be ignored, because the tree map throws away keys it regards as equal to an existing key.

[l-1squared]

An easy workaround is obviously to just change the priority

[l-1squared]

Attached the complete project from which I produced the posted Application.java file. testng-error-example.zip