wpf icon indicating copy to clipboard operation
wpf copied to clipboard

Windows Automation tree breaking change

Open Tobiidave opened this issue 2 years ago • 15 comments

Description

Dotnet 8 renders the controls in the Windows automation tree differently compared to earlier dotnet versions. Buttons in an itemscontrol can't be found anymore, breaking the existing automation.

Using the "Accessabilitiy Insights for Windows" shows a different graph compared to dotnet 6/7 (see below)

Reproduction Steps

WpfApp3.zip

Expected behavior

With <TargetFramework>net7.0-windows</TargetFramework> , the automation tree looks like pane "Desktop 1" window "MainWindow" title bar '' button "TopButton" button "ItemButton1"

Actual behavior

With <TargetFramework>net8.0-windows</TargetFramework>, the automation tree looks like

pane "Desktop 1" window "MainWindow" title bar '' button "TopButton" list view #note that here was the "ItemButton1" - now it is not present anymore!

Regression?

Yes

Known Workarounds

No response

Impact

The UI Tests for our applications breaks; need workarounds

Configuration

Microsoft.WindowsDesktop.App 8.0.1 Microsoft.NETCore.App 8.0.1 windows 10.19045.3803 x64

Vs2022

Other information

No response

Tobiidave avatar Jan 12 '24 09:01 Tobiidave

@Tobiidave Maybe this behavior be changed by https://github.com/dotnet/wpf/pull/6862

I'm not sure.

lindexi avatar Jan 12 '24 11:01 lindexi

I don't know how the 6862 affected this. But the problem exists in both 8.0.0 and 8.0.1

Tobiidave avatar Jan 12 '24 14:01 Tobiidave

I do not think you can expect automation trees to stay intact between major version upgrades - I would consider it business-as-usual these getting broken on major upgrade.

psmulovics avatar Jan 12 '24 19:01 psmulovics

@psmulovics In this case the WPF controls change type and are not usable anymore, I edited and tried to emphasize this by a small comment. It is not only a tree re-ordering issue, but elements that were clickable can not be found anymore! Ofc, any suggested workarounds are welcome! We have paused dotnet8 migration due to this, the automation tech is central to our development.

Tobiidave avatar Jan 15 '24 10:01 Tobiidave

We have paused dotnet8 migration due to this

@Tobiidave Sorry hear that. I still haven't found the exact change code for this issues.

lindexi avatar Jan 19 '24 09:01 lindexi

I've created a test app and I've tried to restore the original behavior simply by setting this in OnStartup:

System.AppContext.SetSwitch("Switch.System.Windows.Controls.ItemsControlDoesNotSupportAutomation", true);

Unfortunately, it looks like it's too late and the value is already set and cached. So I had to use reflection instead:

var accessibilitySwitches = Type.GetType("System.Windows.AccessibilitySwitches, WindowsBase, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089");
var field = accessibilitySwitches.GetField("_ItemsControlDoesNotSupportAutomation", System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Static | BindingFlags.DeclaredOnly);
field.SetValue(null, 1);

This seems to fix the issue and restore the original pre-NET8 behavior. To test it, I used the following piece of code in the test app from this bug report:

public static void VerifyMainWindowAccessibility()
{
    AutomationElement mainWindow = AutomationElement.RootElement.FindFirst(TreeScope.Children, new PropertyCondition(AutomationElement.NameProperty, "MainWindow"));
    Debug.WriteLine(DumpUIATree(mainWindow));
}

private static string DumpUIATree(AutomationElement element)
{
    var s = element.Current.Name + " : " + element.Current.ControlType.ProgrammaticName;
    DumpChildrenRecursively(element, 1, ref s);
    return s;
}

private static List<AutomationElement> GetChildNodes(AutomationElement automationElement)
{
    var children = new List<AutomationElement>();
    TreeWalker walker = TreeWalker.ControlViewWalker;
    AutomationElement child = walker.GetFirstChild(automationElement);
    while (child != null)
    {
        children.Add(child);
        child = walker.GetNextSibling(child);
    }
    return children;
}

private static void DumpChildrenRecursively(AutomationElement node, int level, ref string s)
{
    var children = GetChildNodes(node);
    foreach (AutomationElement child in children)
    {
        if (child != null)
        {
            for (int i = 0; i < level; i++)
                s += "-";
            s += " " + child.Current.Name + " : " + child.Current.ControlType?.ProgrammaticName + "\r\n";
            DumpChildrenRecursively(child, level + 1, ref s);
        }
    }
}

NET8 before the fix:

- TopButton : ControlType.Button
-- TopButton : ControlType.Text
-  : ControlType.List
-- ItemButton1 : ControlType.DataItem
--- ItemButton1 : ControlType.Text

NET8 after the fix:

- TopButton : ControlType.Button
-- TopButton : ControlType.Text
- ItemButton1 : ControlType.Button
-- ItemButton1 : ControlType.Text

NET6 behavior:

- TopButton : ControlType.Button
-- TopButton : ControlType.Text
- ItemButton1 : ControlType.Button
-- ItemButton1 : ControlType.Text

jimm98y avatar Jan 20 '24 01:01 jimm98y

I am investigating the issue from my end.

Kuldeep-MS avatar Jan 23 '24 10:01 Kuldeep-MS

Good work guys - but be sure to test this with the ms app "Accessibilities for Windows" to make sure it looks the same. It shows larger differences than what I see from your inspection using the TreeWalker-code above!

Tobiidave avatar Jan 23 '24 13:01 Tobiidave

I've tested this code below to set the AppContext switch and find it worked.

public class Program
{
    [STAThread]
    static void Main()
    {
        AppContext.SetSwitch("Switch.System.Windows.Controls.ItemsControlDoesNotSupportAutomation", true);

        var app = new App();
        app.InitializeComponent();
        app.Run();
    }
}

This means that if it is late on startup, it can be set before the WPF initialization which reflection is not needed. @jimm98y

walterlv avatar Jan 30 '24 02:01 walterlv

Hi. So now when .Net7 is out of support and this issue is blocking from moving to .Net8, what do think we should do with our app and +1000 customers?

Tobiidave avatar Aug 20 '24 14:08 Tobiidave

@Tobiidave Sorry, I do not think the PR can enter .NET 9

lindexi avatar Aug 21 '24 01:08 lindexi

So how would we handle these changes in the windows automation tree?

Tobiidave avatar Aug 21 '24 12:08 Tobiidave

@Tobiidave - Could you please take a look at this comment, where the same scenario is explained in detail? Please let us know if it isn't working for you.

Kuldeep-MS avatar Aug 21 '24 14:08 Kuldeep-MS

Hi - I took the test project attached to this thread, added

    public App()
    {
        AppContext.SetSwitch("Switch.System.Windows.Controls.ItemsControlDoesNotSupportAutomation", true);
        
        // Create the startup window
        MainWindow mainWindow = new MainWindow();
        mainWindow.Show();
    }

But the automation tree did not change.

Tobiidave avatar Aug 23 '24 07:08 Tobiidave

@Tobiidave It is too late to do it in the App ctor. And thanks for creating this issue. It helped me to figure out the workaround. We also have a problem with dataitems. It is random, but all or few might be empty sometimes in ItemsControls with item template. Use this one in the startup WPF project:

<ItemGroup>
  <RuntimeHostConfigurationOption Include="Switch.System.Windows.Controls.ItemsControlDoesNotSupportAutomation" Value="True" />
</ItemGroup>

syromiatnikov avatar Feb 06 '25 15:02 syromiatnikov

Just checking in - any updates on this issue? We've been hearing from a few customers at DevExpress who are running into problems because of this breaking change

Alexgoon avatar Apr 30 '25 12:04 Alexgoon