Skip to content

Prashant-Jonny/BenchmarkDotNet

 
 

Repository files navigation

BenchmarkDotNet is a lightweight .NET library for benchmarking. Microbenchmarking is very hard, but BenchmarkDotNet helps you to create accurate benchmarks in an easy way.

Features

  • BenchmarkDotNet creates an isolated project for each benchmark method and automatically runs it in a separate runtime, in Release mode, without an attached debugger.
  • You can create benchmark tasks that run your benchmark with different CLR, JIT and platform versions.
  • BenchmarkDotNet performs warm-up executions of your code, then runs it several times in different CLR instances, calculates statistics and tries to eliminate some runtime side-effects.
  • BenchmarkDotNet runs with minimal overhead so as to give an accurate performance measurment.

Developing

Join the chat at: Join the chat at https://gitter.im/PerfDotNet/BenchmarkDotNet

Also see the developing page.

Getting started

Step 1 Install BenchmarkDotNet via the NuGet package: BenchmarkDotNet

Step 2 Write a class with methods that you want to measure and mark them with the Benchmark attribute. In the following example, we will compare MD5 and SHA256 cryptographic hash functions:

public class Md5VsSha256
{
    private const int N = 10000;
    private readonly byte[] data;

    private readonly SHA256 sha256 = SHA256.Create();
    private readonly MD5 md5 = MD5.Create();

    public Md5VsSha256()
    {
        data = new byte[N];
        new Random(42).NextBytes(data);
    }

    [Benchmark]
    public byte[] Sha256()
    {
        return sha256.ComputeHash(data);
    }

    [Benchmark]
    public byte[] Md5()
    {
        return md5.ComputeHash(data);
    }
}

Step 3 Run it:

new BenchmarkRunner().Run<Md5VsSha256>();

Step 4 View the results, here is an example of output from the above benchmark:

BenchmarkDotNet-Dev=v0.7.8.0
OS=Microsoft Windows NT 6.2.9200.0
Processor=Intel(R) Core(TM) i7-4702MQ CPU @ 2.20GHz, ProcessorCount=8
HostCLR=MS.NET 4.0.30319.42000, Arch=64-bit  [RyuJIT]
Type=Algo_Md5VsSha256  Mode=Throughput  Platform=HostPlatform  Jit=HostJit  .NET=HostFramework  toolchain=Classic  Runtime=Clr  Warmup=5  Target=10
Method AvrTime StdDev op/s
Md5 26.2220 us 0.2254 us 38,138.59
Sha256 139.7358 us 9.2690 us 7,183.93

Advanced Features

BenchmarkDotNet provides you with several features that let you write more complex and powerful benchmarks.

  • [BenchmarkTask] attribute let you specify different useful variables: jit version (Legacy/RyuJIT), cpu architecture (x86/x64), target runtime (CLR/Mono), and so on. You can use several attributes to compare different benchmark configurations.
  • [OperationsPerInvoke] attribute let you create a benchmark that contains specific amount of operation. It can be very useful for really quick operations that hard to measure: you can merge them into a single method.
  • [Setup] attribute let you specify a method that can be run before each benchmark batch or run
  • [Params(..)] makes it easy to run the same benchmark with different input values

The code below shows how these features can be used. In this example the benchmark will be run 4 times, with the value of MaxCounter automaticially initialised each time to the values [1, 5, 10, 100]. In addition before each run the SetupData() method will be called, so that initialValuesArray can be re-sized based on MaxCounter.

[BenchmarkTask(platform: BenchmarkPlatform.X86, jitVersion: BenchmarkJitVersion.LegacyJit)]
[BenchmarkTask(platform: BenchmarkPlatform.X64, jitVersion: BenchmarkJitVersion.LegacyJit)]
[BenchmarkTask(platform: BenchmarkPlatform.X64, jitVersion: BenchmarkJitVersion.RyuJit)]
public class IL_Loops
{
    [Params(1, 5, 10, 100)]
    int MaxCounter = 0;
    
    private int[] initialValuesArray;

    [Setup]
    public void SetupData()
    {
        initialValuesArray = Enumerable.Range(0, MaxCounter).ToArray();
    }

    [Benchmark]
    public int ForLoop()
    {
        var counter = 0;
        for (int i = 0; i < initialValuesArray.Length; i++)
            counter += initialValuesArray[i];
        return counter;
    }

    [Benchmark]
    public int ForEachArray()
    {
        var counter = 0;
        foreach (var i in initialValuesArray)
            counter += i;
        return counter;
    }
}

Alternative ways of executing Benchmarks

You can also run a benchmark directly from the internet:

string url = "https://raw.githubusercontent.com/PerfDotNet/BenchmarkDotNet/master/BenchmarkDotNet.Samples/CPU/Cpu_Ilp_Inc.cs";
new BenchmarkRunner().RunUrl(url);

Or you can create a set of benchmarks and choose one from command line:

var benchmarkSwitcher = new BenchmarkSwitcher(new[] {
    typeof(Intro_00_Basic),
    typeof(Intro_01_MethodTasks),
    typeof(Intro_02_ClassTasks),
    typeof(Intro_03_SingleRun),
    typeof(Intro_04_UniformReportingTest),
});
benchmarkSwitcher.Run(args);

Authors

Andrey Akinshin, Jon Skeet, Matt Warren

Packages

No packages published

Languages

  • C# 100.0%