Skip to content

Extends xUnit to expose extra context and simplify logging

License

Notifications You must be signed in to change notification settings

KevinSmall/XunitContext

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

XunitContext

Build status NuGet Status

Extends xUnit to expose extra context and simplify logging.

Redirects Trace.Write, Debug.Write, and Console.Write and Console.Error.Write to ITestOutputHelper. Also provides static access to the current ITestOutputHelper for use within testing utility methods.

Uses AsyncLocal to track state.

Support is available via a Tidelift Subscription.

Contents

NuGet package

https://nuget.org/packages/XunitContext/

ClassBeingTested

using System;
using System.Diagnostics;

static class ClassBeingTested
{
    public static void Method()
    {
        Trace.WriteLine("From Trace");
        Console.WriteLine("From Console");
        Debug.WriteLine("From Debug");
        Console.Error.WriteLine("From Console Error");
    }
}

snippet source | anchor

XunitContextBase

XunitContextBase is an abstract base class for tests. It exposes logging methods for use from unit tests, and handle the flushing of logs in its Dispose method. XunitContextBase is actually a thin wrapper over XunitContext. XunitContexts Write* methods can also be use inside a test inheriting from XunitContextBase.

using Xunit;
using Xunit.Abstractions;

public class TestBaseSample  :
    XunitContextBase
{
    [Fact]
    public void Write_lines()
    {
        WriteLine("From Test");
        ClassBeingTested.Method();

        var logs = XunitContext.Logs;

        Assert.Contains("From Test", logs);
        Assert.Contains("From Trace", logs);
        Assert.Contains("From Debug", logs);
        Assert.Contains("From Console", logs);
        Assert.Contains("From Console Error", logs);
    }

    public TestBaseSample(ITestOutputHelper output) :
        base(output)
    {
    }
}

snippet source | anchor

Logging

XunitContext provides static access to the logging state for tests. It exposes logging methods for use from unit tests, however registration of ITestOutputHelper and flushing of logs must be handled explicitly.

using System;
using Xunit;
using Xunit.Abstractions;

public class XunitLoggerSample :
    IDisposable
{
    [Fact]
    public void Usage()
    {
        XunitContext.WriteLine("From Test");

        ClassBeingTested.Method();

        var logs = XunitContext.Logs;

        Assert.Contains("From Test", logs);
        Assert.Contains("From Trace", logs);
        Assert.Contains("From Debug", logs);
        Assert.Contains("From Console", logs);
        Assert.Contains("From Console Error", logs);
    }

    public XunitLoggerSample(ITestOutputHelper testOutput)
    {
        XunitContext.Register(testOutput);
    }

    public void Dispose()
    {
        XunitContext.Flush();
    }
}

snippet source | anchor

XunitContext redirects Trace.Write, Console.Write, and Debug.Write in its static constructor.

Trace.Listeners.Clear();
Trace.Listeners.Add(new TraceListener());
#if (NETSTANDARD)
DebugPoker.Overwrite(
text =>
{
    if (string.IsNullOrEmpty(text))
    {
        return;
    }

    if (text.EndsWith(Environment.NewLine))
    {
        WriteLine(text.TrimTrailingNewline());
        return;
    }

    Write(text);
});
#else
Debug.Listeners.Clear();
Debug.Listeners.Add(new TraceListener());
#endif
var writer = new TestWriter();
Console.SetOut(writer);
Console.SetError(writer);

snippet source | anchor

These API calls are then routed to the correct xUnit ITestOutputHelper via a static AsyncLocal.

Logging Libs

Approaches to routing common logging libraries to Diagnostics.Trace:

Filters

XunitContext.Filters can be used to filter out unwanted lines:

using Xunit;
using Xunit.Abstractions;

public class FilterSample :
    XunitContextBase
{
    static FilterSample()
    {
        Filters.Add(x => x != null && !x.Contains("ignored"));
    }

    [Fact]
    public void Write_lines()
    {
        WriteLine("first");
        WriteLine("with ignored string");
        WriteLine("last");
        var logs = XunitContext.Logs;

        Assert.Contains("first", logs);
        Assert.DoesNotContain("with ignored string", logs);
        Assert.Contains("last", logs);
    }

    public FilterSample(ITestOutputHelper output) :
        base(output)
    {
    }
}

snippet source | anchor

Filters are static and shared for all tests.

Context

For every tests there is a contextual API to perform several operations.

  • Context.TestOutput: Access to ITestOutputHelper.
  • Context.Write and Context.WriteLine: Write to the current log.
  • Context.LogMessages: Access to all log message for the current test.
  • Counters: Provide access in predicable and incrementing values for the following types: Guid, Int, Long, UInt, and ULong.
  • Context.Test: Access to the current ITest.
  • Context.SourceFile: Access to the file path for the current test.
  • Context.SourceDirectory: Access to the directory path for the current test.
  • Context.SolutionDirectory: The current solution directory. Obtained by walking up the directory tree from SourceDirectory.
  • Context.TestException: Access to the exception if the current test has failed. See Test Failure.

using Xunit;
using Xunit.Abstractions;

public class ContextSample  :
    XunitContextBase
{
    [Fact]
    public void Usage()
    {
        Context.WriteLine("Some message");

        var currentLogMessages = Context.LogMessages;

        var testOutputHelper = Context.TestOutput;

        var currentTest = Context.Test;

        var sourceFile = Context.SourceFile;

        var sourceDirectory = Context.SourceDirectory;

        var solutionDirectory = Context.SolutionDirectory;

        var currentTestException = Context.TestException;
    }

    public ContextSample(ITestOutputHelper output) :
        base(output)
    {
    }
}

snippet source | anchor

Some members are pushed down to the be accessible directly from XunitContextBase:

using Xunit;
using Xunit.Abstractions;

public class ContextPushedDownSample  :
    XunitContextBase
{
    [Fact]
    public void Usage()
    {
        WriteLine("Some message");

        var currentLogMessages = Logs;

        var testOutputHelper = Output;

        var sourceFile = SourceFile;

        var sourceDirectory = SourceDirectory;

        var solutionDirectory = SolutionDirectory;

        var currentTestException = TestException;
    }

    public ContextPushedDownSample(ITestOutputHelper output) :
        base(output)
    {
    }
}

snippet source | anchor

Context can accessed via a static API:

using Xunit;
using Xunit.Abstractions;

public class ContextStaticSample :
    XunitContextBase
{
    [Fact]
    public void StaticUsage()
    {
        XunitContext.Context.WriteLine("Some message");

        var currentLogMessages = XunitContext.Context.LogMessages;

        var testOutputHelper = XunitContext.Context.TestOutput;

        var currentTest = XunitContext.Context.Test;

        var sourceFile = XunitContext.Context.SourceFile;

        var sourceDirectory = XunitContext.Context.SourceDirectory;

        var solutionDirectory = XunitContext.Context.SolutionDirectory;

        var currentTestException = XunitContext.Context.TestException;
    }

    public ContextStaticSample(ITestOutputHelper output) :
        base(output)
    {
    }
}

snippet source | anchor

Current Test

There is currently no API in xUnit to retrieve information on the current test. See issues #1359, #416, and #398.

To work around this, this project exposes the current instance of ITest via reflection.

Usage:

using Xunit;
using Xunit.Abstractions;

public class CurrentTestSample :
    XunitContextBase
{
    [Fact]
    public void Usage()
    {
        var currentTest = Context.Test;
        // DisplayName will be 'TestNameSample.Usage'
        var displayName = currentTest.DisplayName;
    }

    [Fact]
    public void StaticUsage()
    {
        var currentTest = XunitContext.Context.Test;
        // DisplayName will be 'TestNameSample.StaticUsage'
        var displayName = currentTest.DisplayName;
    }

    public CurrentTestSample(ITestOutputHelper output) :
        base(output)
    {
    }
}

snippet source | anchor

Implementation:

using System;
using System.Reflection;
using Xunit.Abstractions;
using Xunit.Sdk;

namespace Xunit
{
    public partial class Context
    {
        ITest? test;

        static FieldInfo? cachedTestMember;

        public ITest Test
        {
            get
            {
                InitTest();

                return test!;
            }
        }

        MethodInfo? methodInfo;
        public MethodInfo MethodInfo
        {
            get
            {
                InitTest();
                return methodInfo!;
            }
        }

        Type? testType;
        public Type TestType
        {
            get
            {
                InitTest();
                return testType!;
            }
        }

        void InitTest()
        {
            if (test != null)
            {
                return;
            }
            test = (ITest) GetTestMethod().GetValue(TestOutput);
            var method = (ReflectionMethodInfo) test.TestCase.TestMethod.Method;
            var type = (ReflectionTypeInfo) test.TestCase.TestMethod.TestClass.Class;
            methodInfo = method.MethodInfo;
            testType = type.Type;
        }

        public static string MissingTestOutput = "ITestOutputHelper has not been set. It is possible that the call to `XunitContext.Register()` is missing, or the current test does not inherit from `XunitContextBase`.";

        FieldInfo GetTestMethod()
        {
            if (TestOutput == null)
            {
                throw new Exception(MissingTestOutput);
            }

            if (cachedTestMember != null)
            {
                return cachedTestMember;
            }

            var testOutputType = TestOutput.GetType();
            cachedTestMember = testOutputType.GetField("test", BindingFlags.Instance | BindingFlags.NonPublic);
            if (cachedTestMember == null)
            {
                throw new Exception($"Unable to find 'test' field on {testOutputType.FullName}");
            }

            return cachedTestMember;
        }
    }
}

snippet source | anchor

Test Failure

When a test fails it is expressed as an exception. The exception can be viewed by enabling exception capture, and then accessing Context.TestException. The TestException will be null if the test has passed.

One common case is to perform some logic, based on the existence of the exception, in the Dispose of a test.

public static class GlobalSetup
{
    [System.Runtime.CompilerServices.ModuleInitializer]
    public static void Setup()
    {
        XunitContext.EnableExceptionCapture();
    }
}

[Trait("Category", "Integration")]
public class TestExceptionSample :
    XunitContextBase
{
    [Fact]
    public void Usage()
    {
        //This tests will fail
        Assert.False(true);
    }

    public TestExceptionSample(ITestOutputHelper output) :
        base(output)
    {
    }

    public override void Dispose()
    {
        var theExceptionThrownByTest = Context.TestException;
        var testDisplayName = Context.Test.DisplayName;
        var testCase = Context.Test.TestCase;
        base.Dispose();
    }
}

snippet source | anchor

Base Class

When creating a custom base class for other tests, it is necessary to pass through the source file path to XunitContextBase via the constructor.

public class CustomBase :
    XunitContextBase
{
    public CustomBase(
        ITestOutputHelper testOutput,
        [CallerFilePath] string sourceFile = "") :
        base(testOutput, sourceFile)
    {
    }
}

snippet source | anchor

Parameters

Provided the parameters passed to the current test when using a [Theory].

Use cases:

Usage:

using System.Collections.Generic;
using System.Linq;
using Xunit;
using Xunit.Abstractions;

public class ParametersSample :
    XunitContextBase
{
    [Theory]
    [MemberData(nameof(GetData))]
    public void Usage(string arg)
    {
        var parameter = Context.Parameters.Single();
        var parameterInfo = parameter.Info;
        Assert.Equal("arg", parameterInfo.Name);
        Assert.Equal(arg, parameter.Value);
    }

    public static IEnumerable<object[]> GetData()
    {
        yield return new object[] {"Value1"};
        yield return new object[] {"Value2"};
    }

    public ParametersSample(ITestOutputHelper output) :
        base(output)
    {
    }
}

snippet source | anchor

Implementation:

static List<Parameter> GetParameters(ITestCase testCase)
{
    return GetParameters(testCase, testCase.TestMethodArguments);
}

static List<Parameter> GetParameters(ITestCase testCase, object[] arguments)
{
    var method = testCase.TestMethod;
    var infos = method.Method.GetParameters().ToList();
    if (arguments == null || !arguments.Any())
    {
        if (infos.Count == 0)
        {
            return empty;
        }

        throw NewNoArgumentsDetectedException();
    }

    var items = new List<Parameter>();

    for (var index = 0; index < infos.Count; index++)
    {
        items.Add(new Parameter(infos[index], arguments[index]));
    }

    return items;
}

snippet source | anchor

Complex parameters

Only core types (string, int, DateTime etc) can use the above automated approach. If a complex type is used the following exception will be thrown

No arguments detected for method with parameters. This is most likely caused by using a parameter that Xunit cannot serialize. Instead pass in a simple type as a parameter and construct the complex object inside the test. Alternatively; override the current parameters using UseParameters() via the current test base class, or via XunitContext.Current.UseParameters().

To use complex types override the parameter resolution using XunitContextBase.UseParameters:

using System.Collections.Generic;
using System.Linq;
using Xunit;
using Xunit.Abstractions;

public class ComplexParameterSample :
    XunitContextBase
{
    [Theory]
    [MemberData(nameof(GetData))]
    public void UseComplexMemberData(ComplexClass arg)
    {
        UseParameters(arg);
        var parameter = Context.Parameters.Single();
        var parameterInfo = parameter.Info;
        Assert.Equal("arg", parameterInfo.Name);
        Assert.Equal(arg, parameter.Value);
    }

    public static IEnumerable<object[]> GetData()
    {
        yield return new object[] {new ComplexClass("Value1")};
        yield return new object[] {new ComplexClass("Value2")};
    }

    public ComplexParameterSample(ITestOutputHelper output) :
        base(output)
    {
    }

    public class ComplexClass
    {
        public string Value { get; }

        public ComplexClass(string value)
        {
            Value = value;
        }
    }
}

snippet source | anchor

UniqueTestName

Provided a string that uniquely identifies a test case.

Usage:

using Xunit;
using Xunit.Abstractions;

public class UniqueTestNameSample :
    XunitContextBase
{
    [Fact]
    public void Usage()
    {
        var testName = Context.UniqueTestName;

        Context.WriteLine(testName);
    }

    public UniqueTestNameSample(ITestOutputHelper output) :
        base(output)
    {
    }
}

snippet source | anchor

Implementation:

string GetUniqueTestName(ITestCase testCase)
{
    var method = testCase.TestMethod;
    var name = $"{method.TestClass.Class.ClassName()}.{method.Method.Name}";
    if (!Parameters.Any())
    {
        return name;
    }

    var builder = new StringBuilder($"{name}_");
    foreach (var parameter in Parameters)
    {
        builder.Append($"{parameter.Info.Name}=");
        if (parameter.Value == null)
        {
            builder.Append("null_");
            continue;
        }

        builder.Append($"{parameter.Value}_");
    }

    builder.Length -= 1;

    return builder.ToString();
}

snippet source | anchor

Global Setup

Xunit has no way to run code once before any tests executing. So use one of the following:

Security contact information

To report a security vulnerability, use the Tidelift security contact. Tidelift will coordinate the fix and disclosure.

Icon

Wolverine designed by Mike Rowe from The Noun Project.

About

Extends xUnit to expose extra context and simplify logging

Resources

License

Code of conduct

Stars

Watchers

Forks

Packages

No packages published

Languages

  • C# 100.0%