If you like HPCsharp, give it a star and donate to help us keep more good stuff coming.
Give us feedback and let us know where else could use additional performance.

High performance C# generic algorithms. Runs on Windows and Linux. Community driven to raise C# performance. Familiar interfaces, similar to standard C# algorithms and Linq. Free, open source, on nuget.org

* Number of different algorithms
** Number of functions for this algorithm
*** Coming soon

Usage examples are provided in the HPCsharpExamples folder, which has a VisualStudio 2017 solution. Build and run it to see performance gains on your machine. To get the maximum performance make sure to target x64 processor architecture for the Release build in VisualStudio, increasing performance by as much as 50%.

The first time you call a function that is implemented using SIMD/SSE instructions, C# just-in-time (JIT) compiler takes the time to compile and optimize that function, which results in much slower performance. On the second use of the function and on subsequent uses, the SIMD/SSE function will run at its full performance. Keep this behavior of the C# JIT compiler in mind as you use or benchmark HPCsharp functions.

HPCsharp improves .Sum() of numeric arrays in the following ways:

• Adds support for the missing signed integer data types: sbyte and short
• Adds support for all unsigned integer data types: byte, ushort, uint, and ulong
• Simplified use: no arithmetic overflow exceptions to deal with, for all integer data types
• SIMD/SSE implementations for all integer and floating-point data types, to boost performance several times per processor core, as well as multi-core to use all the cores
• Adds support for BigInteger: single-core and multi-core
• New checked SIMD/SSE addition in C#, unsigned and signed, for much higher performance
• Extended precision ulong[] and long[] summation for a full precision to a Decimal and BigInteger result, using integer computation only: SIMD/SSE, single-core and multi-core
• Reduced error from O(eN) downto O(elgN) for float and double arrays by performing pair-wise summation
• Reduced error further down to O(e) by implementing Kahan summation for float and double arrays, with slight performance reduction, implemented in SIMD/SSE and multi-core
• GigaAdds/sec performance for all processor native data types

The table below compares performance (in GigaAdds/second) of Linq.AsParallel().Sum() and HPCsharp.SumSsePar() - both use multi-core (6 or 14 of them), with HPCsharp also using SIMD/SSE data parallel instructions on each core to gain additional performance:

* arithmetic overflow exception is possible
n/a not available

** Linq doesn't implement BigInteger.Sum(), used .Aggregate() instead, which doesn't speed-up with .AsParallel()
* HPCsharp implements pair-wise floating-point parallel (multi-core) by default, since it uses divide-and-conquer algorithm for multi-core implementation.

All HPCsharp integer summations (unsigned and signed) including long[] and ulong[] arrays, do not throw overflow exceptions, while producing a perfectly accurate result. This simplifies usage, while providing high performance.

HPCsharp ulong[] array summation implements a full accuracy algorithm using integer only arithmetic to provide maximum performance. It detects and deals with arithmetic overflow internally, without using exceptions, using integer only computation. HPCsharp also uses SIMD/SSE data parallel instructions to get maximum performance out of each core, and uses multi-core to run even faster.

For more details, see several blogs on various aspects:

• Better C# .Sum() in Many Ways
• Better C# .Sum() in More Ways
• Faster Checked Addition in C#
• Faster Checked Addition in C# (Part 2)
• Checked SIMD/SSE Addition in C#
• Video of Checked SIMD/SSE/multi-core Addition in C#

Accelerated and safer implementation of standard deviation for integer type arrays, float and double arrays. Accelerated by using multi-core and SSE data parallel instructions. Avoids arithmetic overflow exceptions for integer data types, using the same methods as HPCsharp's .Sum(). The following benchmarks ran on 6-core i7-9750H processor:

The above benchmarks of Linq code were implemented in the following way:

intArray.Average(v => (long)v);     // intToLong
or
longArray.Average(v => (decimal)v); // longToDecimal

to ensure that no arithmetic overflow exception is possible, to make a fair comparison to HPCsharp implementations.

The following benchmarks ran on 14-core Xeon W-2175 processor:

https://duvanenko.tech.blog/2020/03/22/parallel-standard-deviation/

Another useful measure of variability within a dataset is Mean Absolute Deviation. It is related to standard deviation, using absolute value of the difference between the average value of the data set and-Conquer each data value, eliminating warping of the data.

https://duvanenko.tech.blog/2020/03/22/how-standard-deviation-measures-warped-data/

Provides parallel and serial generic functions, which support multi-core and single-core divide-and-conquer algorithm. Two versions are provided: single data type and two types.

For more details, see blog:

• Divide-and-Conquer

Counting Sort above is a linear time O(N) algorithm, sorting an array of byte, sbyte, short or ushort data types. In-place and not-in-place version have been implementated. The above benchmark is on a single core! Multi-core sorts even faster, at GigaElements/second ludicrous speed.

LSD Radix Sort is a linear time O(N), stable sorting algorithm.

LSD Radix Sort runs on a single core, whereas Linq.AsParallel ran on all the cores. Only slower when sorting presorted Array or List, but faster for random and constant distributions, even faster than parallel Linq.OrderBy.AsParallel.

Parallel LSD Radix Sort, uses multiple cores for the first phase (the counting phase) of the algorithm, bringing performance up to 125 MegaInt32/sec, bringing performance higher than .Sort() for random, presorted and constant distributions.

Radix Sort has been extended to sort user defined classes based on a UInt32 or UInt64 key within the class. Radix Sort is currently using only a single core.

Only slightly slower than Array.Sort and List.Sort for presorted distribution, but faster for all other distributions. Uses a single core and is stable. Faster than Linq.OrderBy and Linq.OrderBy.AsParallel

Merge Sort provides a performance boost comparing with Linq.OrderBy when running on a single core, but is not competitive with Array.Sort(). On a single core on variety of machines, sorting an array of Int32's, performance in Millions of Int32's per second is:

Parallel Merge Sort uses multiple CPU cores to accelerate performance, which scales well with the number of cores and the number of memory channels. C# Array.Sort does not support parallel sorting. On variety of machines, sorting an array of Int32's, performance in Millions of Int32's per second is:

HPCsharp's Parallel Merge Sort is not stable, just like Array.Sort. The version benchmarked above is the not-in-place one. Faster than Array.Sort and List.Sort across all distributions, and substantially faster than Linq.OrderBy and Linq.OrderBy.AsParallel, which doesn't scale well as the number of cores increases. HPCsharp's Parallel Merge Sort scales very well with the number of cores, for all distributions providing higher performance than Array.Sort() and Linq.OrderBy and Linq.OrderBy.AsParallel.

28-core (56-threads) AWS c5.18xlarge

Merge Sort is O(NlgN), never O(N²), generic, stable, and runs on a single CPU core. Faster than Linq.OrderBy and Linq.OrderBy.AsParallel.

O(N) linear-time generic merge algorithms for arrays and list containers. Merges two pre-sorted arrays or lists, of any data type that defines IComparer. Two not-in-place algorithms: comparison at the heads, and divide-and-conquer. Parallel Merge algorithm, using divide-and-conquer, merges two presorted collections using multiple cores. Used by Parallel Merge Sort. See example solution for working code samples.

Insertion Sort, which is O(N²), and useful for fast in-place sorting of very small collections, due to its cache-friendliness. Generic implemenation for Array and List containers. Used by Parallel Merge Sort and MSD Radix Sort for the base case.

Two algorithms for adding two arrays together:

• c[] = a[] + b[]
• a[] += b[]

The second algorithm is about 70% faster than the first. So far, both algorithms are implemented for int[] only, but other data types can be easily added. Both algorithms are implemented in scalar, data-parallel SIMD/SSE on a single core, and multi-core. Both run up to the memory bandwidth limit.

Generic implementation of the binary search algorithm, for Array and List containers. Used by the scalar and parallel divide-and-conquer Merge algorithms.

.Min() is implemented using SIMD/SSE instructions to run at 4 GigaInts/sec on a single core, and over 5 GigaInts/sec on quad-core.

Three scalar algorithms for in-place swapping two neighboring sub-regions of an array, which do not have to be of equal size:

• Reversal
• Gries and Mills
• Juggle Bentley

See an article for more details (http://www.drdobbs.com/parallel/benchmarking-block-swapping-algorithms/232900395)

Also, several generic version of two element swap.

Detects whether a byte array is zero in every byte. Runs at 17 GBytes/sec on a quad-core laptop, with two memory channels, using a single core. Provides short-circuit, early exit when a non-zero value is detected while scanning the array. Provides scalar, SSE, scalar-unrolled, SSE-unrolled, scalar unrolled multi-core, and SSE unrolled multi-core implementations. Unrolled refers to the loop being unrolled a few times to gain additional performance.

On dual memory channel CPUs, SSE-unrolled is the fastest, using a single core, saturating system memory bandwidth. For systems with more memory channels, SSE unrolled multi-core will most likely have the highest performance.

Converting a List to an Array is a common operation:

var listSource = new List<int> { 5, 7, 16, 3 };

int[] arrayDestination1 = listSource.ToArray();	    // C# standard conversion
int[] arrayDestination2 = listSource.ToArrayPar();  // HPCsharp parallel/multi-core/faster conversion

The following table shows performance (in Billion Int32's per second) for copy functions:

var listSource = new List<int> { 5, 7, 16, 3 };
int[] arrayDestination = new int[4];

listSource.CopyTo(arrayDestination);	 // C# standard List to Array copy
listSource.CopyToPar(arrayDestination);  // HPCsharp parallel/multi-core/faster copy

The following table shows performance (in GigaInt32/sec) for copy functions:

var arraySource = new int[4] { 5, 7, 16, 3 };
int[] arrayDestination = new int[4];

arraySource.CopyTo(arrayDestination);	 // C# standard List to Array copy
arraySource.CopyToPar(arrayDestination);  // HPCsharp parallel/multi-core/faster copy

HPCsharp provides parallel (multi-core) versions of List.ToArray() and List.CopyTo() functions, with exactly the same interfaces. Parallel Array.ToArray() and Array.CopyTo() are also available. These parallel functions are 3 times faster when the destination is a new array - i.e. allocated but never touched - a common use case shown in the first source code case above. When a destination array has been used before and has been paged into system memory, these parallel functions are 10-20% faster. These parallel copy functions provide a generic interface, handling any data type.

For more details, seee blog https://duvanenko.tech.blog/2019/08/19/faster-copying-in-c/

HPCsharp follows a few simple naming conventions:

• SSE functions append "Sse" to the function name
• multi-core functions append "Par" to the function name
• if the function name clashes with C# Linq name, then "Hpc" is appended to the function name

For details on the motivation see blog: https://duvanenko.tech.blog/2018/03/03/high-performance-c/

For more performance discussion see blog: https://duvanenko.tech.blog/2018/05/23/faster-sorting-in-c/

HPCsharp presentation at the Indianapolis .NET Consortium, March 2019 on https://youtu.be/IRNW4VGevvQ

HPCsharp lighning talk at the Indianapolis .NET Consortium, October 2019 on - https://www.youtube.com/watch?v=hNqE1Ghwbv4

Visit us at https://foostate.com/ and let us know what other high performance algorithms are important to you, and you'd like to see in this NuGet package.

If you like it, then help us keep more good stuff like this coming. Let us know other algorithms that could use acceleration.