Testing PowerToys - Microsoft PowerToys

Test Types

PowerToys uses multiple types of testing to ensure quality:

Test Type	Purpose	Tools	Requirements
Unit Tests	Test individual functions and classes	MSTest, xUnit	Standard dev environment
UI Tests	Test user interface and interactions	WinAppDriver, MSTest	WinAppDriver v1.2.1, Developer Mode
Fuzz Tests	Find security vulnerabilities	OneFuzz, SharpFuzz	.NET 8, OneFuzz service

Unit Tests

Finding Unit Tests

Unit test projects follow naming conventions:

<Product>UnitTests
<Product>*UnitTests

Examples:

FancyZonesUnitTests
AdvancedPasteUnitTests
PowerLauncherUnitTests

Typical locations:

Same folder as the module: src/modules/<Module>/Tests/
Sibling folder: src/modules/<Module>/<Module>UnitTests/
Separate test directory: src/tests/<Module>UnitTests/

Running Unit Tests

Using Visual Studio Test Explorer

Open Test Explorer:
- Menu: Test > Test Explorer
- Keyboard: Ctrl+E, T

Build the test project first:

cd src\modules\<Module>\Tests\
..\..\..\..\tools\build\build.ps1

Wait for exit code 0 (build success)
Run tests in Test Explorer:
- Click Run All to run all tests
- Right-click specific test to run individually
- Use search/filter to find specific tests

Do not use dotnet test in this repository. PowerToys uses MSBuild and Visual Studio Test platform. Use Test Explorer or vstest.console.exe instead.

Using vstest.console.exe

Run tests from command line:

# Run all tests in assembly
vstest.console.exe x64\Debug\FancyZonesUnitTests\FancyZonesUnitTests.dll

# Run specific test
vstest.console.exe x64\Debug\FancyZonesUnitTests\FancyZonesUnitTests.dll /Tests:TestName

# Run tests matching filter
vstest.console.exe x64\Debug\FancyZonesUnitTests\FancyZonesUnitTests.dll /TestCaseFilter:"Priority=1"

Writing Unit Tests

C# Unit Tests

Create test class with MSTest:

using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace PowerToys.ModuleName.UnitTests
{
    [TestClass]
    public class ModuleFeatureTests
    {
        [TestMethod]
        public void TestMethodName_Scenario_ExpectedResult()
        {
            // Arrange
            var module = new ModuleClass();
            var input = "test input";

            // Act
            var result = module.ProcessInput(input);

            // Assert
            Assert.IsNotNull(result);
            Assert.AreEqual("expected output", result);
        }

        [TestMethod]
        [ExpectedException(typeof(ArgumentNullException))]
        public void TestMethodName_NullInput_ThrowsException()
        {
            // Arrange
            var module = new ModuleClass();

            // Act
            module.ProcessInput(null); // Should throw
        }
    }
}

C++ Unit Tests

Using Microsoft Native Unit Test Framework:

#include "CppUnitTest.h"
#include "ModuleClass.h"

using namespace Microsoft::VisualStudio::CppUnitTestFramework;

namespace ModuleUnitTests
{
    TEST_CLASS(ModuleFeatureTests)
    {
    public:
        TEST_METHOD(TestMethodName_Scenario_ExpectedResult)
        {
            // Arrange
            ModuleClass module;
            std::wstring input = L"test input";

            // Act
            auto result = module.ProcessInput(input);

            // Assert
            Assert::IsFalse(result.empty());
            Assert::AreEqual(L"expected output", result.c_str());
        }
    };
}

UI Tests

Prerequisites

Install WinAppDriver v1.2.1:
- Download from WinAppDriver releases
- Install to default location: C:\Program Files (x86)\Windows Application Driver
Enable Developer Mode:
- Settings > Update & Security > For developers
- Select “Developer mode”
Close PowerToys before running UI tests

Running UI Tests

Build PowerToys solution:

.\tools\build\build.ps1 -Configuration Debug

Build UI test project:

cd src\modules\<Module>\Tests\
..\..\..\..\tools\build\build.ps1

Open Test Explorer (Ctrl+E, T)
Run UI tests from Test Explorer

UI Test Project Structure

Follow the naming convention:

{ModuleFolder}/Tests/{ModuleName}-UITests

Example: src/modules/Hosts/Tests/Hosts-UITests/

Project File Template

<Project Sdk="Microsoft.NET.Sdk">
  <Import Project="..\..\..\Common.Dotnet.CsWinRT.props" />

  <PropertyGroup>
    <ProjectGuid>{YOUR-GUID-HERE}</ProjectGuid>
    <RootNamespace>PowerToys.YourModule.UITests</RootNamespace>
    <AssemblyName>PowerToys.YourModule.UITests</AssemblyName>
    <IsPackable>false</IsPackable>
    <IsTestProject>true</IsTestProject>
    <Nullable>enable</Nullable>
    <OutputType>Library</OutputType>

    <!-- This is a UI test, so don't run as part of MSBuild -->
    <RunVSTest>false</RunVSTest>
  </PropertyGroup>
  
  <PropertyGroup>
    <OutputPath>$(SolutionDir)$(Platform)\$(Configuration)\tests\YourModule.UITests\</OutputPath>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="MSTest" />
    <ProjectReference Include="..\..\..\common\UITestAutomation\UITestAutomation.csproj" />
  </ItemGroup>
</Project>

Writing UI Tests

Inherit from UITestBase class:

using Microsoft.VisualStudio.TestTools.UnitTesting;
using PowerToys.UITestAutomation;

namespace PowerToys.YourModule.UITests
{
    [TestClass]
    public class ModuleUITests : UITestBase
    {
        // Constructor sets scope
        public ModuleUITests()
            : base(PowerToysModule.YourModule, WindowSize.Large)
        {
        }

        [TestMethod("YourModule.UI.BasicTest")]
        [TestCategory("YourModule #1")]
        public void TestBasicFunctionality()
        {
            // Find UI element
            var button = this.Find<Button>("AddButton");
            
            // Interact with element
            button.Click();

            // Verify result
            Assert.IsTrue(this.Has<TextBox>("ResultTextBox"), 
                "Result should be displayed");
        }

        [TestMethod("YourModule.UI.VisualTest")]
        public void TestVisualAppearance()
        {
            var container = this.Find("MainContainer");
            
            // Visual assertion - compares screenshot
            VisualAssert.AreEqual(this.TestContext, container, "ExpectedView");
        }
    }
}

UI Test Scope Options

// Default: Starts from PowerToys Settings UI
public ModuleUITests() : base() { }

// Module-specific: Starts from your module's UI
public ModuleUITests() 
    : base(PowerToysModule.YourModule, WindowSize.Small_Vertical) { }

Window size options:

WindowSize.Small_Vertical
WindowSize.Large
WindowSize.Maximized

Common UI Test Patterns

Finding Elements

// By AutomationId
var element = this.Find("ElementAutomationId");

// By type and name
var button = this.Find<Button>("ButtonName");

// Check if exists
bool exists = this.Has<TextBox>("TextBoxName");

// Check for single element
bool hasOne = this.HasOne<ComboBox>("ComboBoxName");

Interacting with Elements

// Click button
var button = this.Find<Button>("SaveButton");
button.Click();

// Enter text
var textBox = this.Find<TextBox>("InputField");
textBox.SendKeys("test input");

// Select combo box item
var comboBox = this.Find<ComboBox>("ModeSelector");
comboBox.SelectItemByName("Advanced");

// Toggle switch
var toggle = this.Find<ToggleButton>("EnableFeature");
toggle.Toggle();

Visual Assertions

// Compare element appearance to baseline
var container = this.Find("MainView");
VisualAssert.AreEqual(this.TestContext, container, "ExpectedState");

// Baseline images stored in TestData folder
// Named: ExpectedState.png

UI Test Pipeline

UI tests run automatically in the build pipeline: Pipeline URL: https://microsoft.visualstudio.com/Dart/_build?definitionId=161438 Pipeline options:

buildSource:
  - latestMainOfficialBuild  # Test against latest release
  - buildNow                 # Test current source code
  - specificBuildId          # Test specific build

specificBuildId: "12345"    # Build ID when using specificBuildId

uiTestModules:
  - 'UITests-FancyZones'     # Run specific module tests
  - 'MouseUtils.UITests'     # Or multiple modules
  - []                       # Empty = run all tests

Fuzz Tests

Fuzz testing identifies security vulnerabilities by providing random/malformed input.

When to Use Fuzz Tests

Required for modules that:

Handle file I/O (reading, writing, parsing files)
Process user input (text, commands, data)
Parse external data formats (JSON, XML, images, etc.)
Interface with external APIs or services

Fuzz Test Implementation

C# Fuzz Tests with SharpFuzz

Add NuGet packages:

<PackageReference Include="SharpFuzz" Version="2.1.1" />
<PackageReference Include="Microsoft.OneFuzz.SDK" Version="1.0.0" />

Create fuzz target:

using SharpFuzz;
using Microsoft.OneFuzz.SDK;

public class FuzzTarget
{
    public static void FuzzMethod(string input)
    {
        try
        {
            // Code to test with fuzzing input
            var result = ProcessInput(input);
        }
        catch (Exception ex)
        {
            // Expected exceptions should be caught
            // Unexpected exceptions will be reported
        }
    }

    public static void Main(string[] args)
    {
        // SharpFuzz entry point
        Fuzzer.Run(() =>
        {
            var input = Console.ReadLine();
            FuzzMethod(input);
        });
    }
}

C++ Fuzz Tests with libFuzzer

#include <stdint.h>
#include <stddef.h>
#include "ModuleToTest.h"

extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
{
    if (size < 4) return 0;

    try
    {
        // Convert fuzzing input to usable format
        std::string input(reinterpret_cast<const char*>(data), size);
        
        // Test your code with fuzzing input
        ProcessInput(input);
    }
    catch (...)
    {
        // Expected exceptions
    }

    return 0;
}

Running Fuzz Tests Locally

# Build fuzz target
dotnet build src\modules\<Module>\FuzzTests\

# Run with SharpFuzz
sharpfuzz .\x64\Debug\<Module>.FuzzTests.dll

# Provide seed corpus (optional)
sharpfuzz .\x64\Debug\<Module>.FuzzTests.dll --corpus=.\testdata\corpus

OneFuzz Integration

PowerToys integrates with Microsoft’s OneFuzz service for continuous fuzzing:

Runs automatically in CI/CD pipeline
Generates crash reports for security team
Provides coverage metrics
See Fuzzing Testing docs

Test Requirements

When to Update Tests

Always update tests when:

Adding new functionality
Changing existing behavior
Fixing bugs
Refactoring code

Acceptable to skip tests when:

Comment-only changes
Documentation updates
String/resource changes (with justification)
Whitespace/formatting fixes

Test Coverage Expectations

New modules must have unit tests
UI-based modules should have UI tests
File/input handling modules must have fuzz tests
Bug fixes should include regression tests

Testing Discipline

Before Opening a PR

Build succeeds with exit code 0
All existing tests pass locally
New tests added for new functionality
Tests pass in Test Explorer
Fuzz tests implemented if handling file I/O or user input

Test Checklist

Unit tests written for new code
UI tests written for UI changes (if applicable)
Fuzz tests written for file/input handling (if applicable)
All tests pass locally
Tests documented with clear names
Edge cases covered
Exception handling tested
Performance considered for test execution time

Special Testing Requirements

Multi-Monitor Testing

Utilities requiring multi-monitor testing:

FancyZones
Display Fuser
Mouse Jump

Test configurations:

2+ monitors with different resolutions
Different DPI settings (100%, 125%, 150%, 200%)
Primary monitor on left/right
Vertical arrangements
Monitor disconnect/reconnect

Multi-Computer Testing

Mouse Without Borders:

Requires 2+ physical computers
Cannot be adequately tested with VMs
Test across different Windows versions

Elevation Testing

Test both scenarios:

PowerToys running as normal user
PowerToys running elevated (administrator)

Ensure graceful degradation when not elevated.

Debugging Test Failures

Test Explorer Debugging

Right-click test in Test Explorer
Select Debug Selected Tests
Breakpoints will be hit during test execution

Test Output

View test output in Test Explorer:

Select failed test
Click “Open additional output for this result” link
Review stack traces and error messages

UI Test Debugging

// Add delays for visual inspection
System.Threading.Thread.Sleep(2000);

// Take screenshots on failure
if (!condition)
{
    var screenshot = this.TakeScreenshot();
    this.TestContext.AddResultFile(screenshot);
    Assert.Fail("Condition not met");
}

Next Steps

Building

Build PowerToys from source

Debugging

Debug PowerToys modules effectively

Coding Style

Follow PowerToys coding standards

Creating New Utility

Create a new PowerToys utility

​Test Types

​Unit Tests

​Finding Unit Tests

​Running Unit Tests

​Using Visual Studio Test Explorer

​Using vstest.console.exe

​Writing Unit Tests

​C# Unit Tests

​C++ Unit Tests

​UI Tests

​Prerequisites

​Running UI Tests

​UI Test Project Structure

​Project File Template

​Writing UI Tests

​UI Test Scope Options

​Common UI Test Patterns

​Finding Elements

​Interacting with Elements

​Visual Assertions

​UI Test Pipeline

​Fuzz Tests

​When to Use Fuzz Tests

​Fuzz Test Implementation

​C# Fuzz Tests with SharpFuzz

​C++ Fuzz Tests with libFuzzer

​Running Fuzz Tests Locally

​OneFuzz Integration

​Test Requirements

​When to Update Tests

​Test Coverage Expectations

​Testing Discipline

​Before Opening a PR

​Test Checklist

​Special Testing Requirements

​Multi-Monitor Testing

​Multi-Computer Testing

​Elevation Testing

​Debugging Test Failures

​Test Explorer Debugging

​Test Output

​UI Test Debugging

​Next Steps

Building

Debugging

Coding Style

Creating New Utility

Test Types

Unit Tests

Finding Unit Tests

Running Unit Tests

Using Visual Studio Test Explorer

Using vstest.console.exe

Writing Unit Tests

C# Unit Tests

C++ Unit Tests

UI Tests

Prerequisites

Running UI Tests

UI Test Project Structure

Project File Template

Writing UI Tests

UI Test Scope Options

Common UI Test Patterns

Finding Elements

Interacting with Elements

Visual Assertions

UI Test Pipeline

Fuzz Tests

When to Use Fuzz Tests

Fuzz Test Implementation

C# Fuzz Tests with SharpFuzz

C++ Fuzz Tests with libFuzzer

Running Fuzz Tests Locally

OneFuzz Integration

Test Requirements

When to Update Tests

Test Coverage Expectations

Testing Discipline

Before Opening a PR

Test Checklist

Special Testing Requirements

Multi-Monitor Testing

Multi-Computer Testing

Elevation Testing

Debugging Test Failures

Test Explorer Debugging

Test Output

UI Test Debugging

Next Steps