Aurio is a .NET library that focuses on audio processing, analysis, media synchronization and media retrieval and implements various audio fingerprinting methods. It has been developed for research purposes and is a by-product of the media synchronization application AudioAlign.

• 32-bit floating point audio processing engine
• File I/O through NAudio and FFmpeg
• Audio playback through NAudio
• FFT/iFFT through PFFFT, FFTW (optional) and Exocortex.DSP (optional)
• Resampling through Soxr and SecretRabbitCode/libsamplerate (optional)
• Stream windowing and overlap-adding
• STFT, inverse STFT
• Chroma
• Dynamic Time Warping
• On-line Time Warping (Dixon, Simon. "Live tracking of musical performances using on-line time warping." Proceedings of the 8th International Conference on Digital Audio Effects. 2005.)
• Fingerprinting
• Haitsma, Jaap, and Ton Kalker. "A highly robust audio fingerprinting system." ISMIR. 2002.
• Wang, Avery. "An Industrial Strength Audio Search Algorithm." ISMIR. 2003.
• Echoprint (Ellis, Daniel PW, Brian Whitman, and Alastair Porter. "Echoprint: An open music identification service." ISMIR. 2011.)
• AcoustID Chromaprint

All audio processing (incl. fingerprinting) is stream-based and supports processing of arbitrarily long streams at constant memory usage. All fingerprinting methods are implemented from scratch, not ports from existing libraries, while keeping compatibility where possible.

Aurio.WaveControls provides WPF widgets for user interfaces:

• Spectrogram / Chromagram View
• Spectrum / Graph View
• VU Meter
• Correlometer
• Time Scale
• Wave View

• Complex type
• Various StreamWindower OutputFormats: Raw, Magnitudes, MagnitudesSquared, MagnitudesAndPhases, and Decibel
• Overlap-adder OLA
• Inverse STFT InverseSTFT
• Writable audio streams
  • IAudioWriterStream interface
  • MemoryWriterStream that writes audio to memory
• Aurio.WaveControls:
  • CaretOverlay, MultiplicationConverter migrated from AudioAlign
  • TrackMarkerOverlay to add markers with labels onto audio tracks
• Inverse FFT transformation in Aurio.PFFFT
• Concatenated AudioTracks (audio tracks consisting of multiple files)
• Seek indizes for FFmpeg input
  • Seek index support in Aurio.FFmpeg.Proxy and Aurio.FFmpeg's FFmpegReader
  • Automatic seek index generation in FFmpegSourceStream for streams that do not natively support seeking
• Windowing moved from STFT to StreamWindower
• NAudio updated to fix Windows 10 high CPU utilization during playback
• Various unit tests
• Many bug fixes, optimizations, and enhancements

• Support for Visual Studio 2015
• FFmpeg setup helper script
• FFmpeg updated to 2.8.2
• Decoding of various files formats through FFmpeg
  • Direct processing of audio from compressed formats and (video) containers
  • Video decoding (see Aurio.FFmpeg.Test program)
  • Automatic creation of wav proxy files for audio formats that don't allow seeking (shn, ape)
• Stream input support in FFmpeg decoder (required a file input before)
• Close-method and IDisposable implementation in audio streams to free resources
• Close all audio streams after use
• MultiStream to concatenate files/streams for decoding (e.g. vob files)
• Support additional video files (ts, mkv, mov) for automatic EDL export wav proxy replacement
• Fix release config postbuild event commands
• Fix FFmpeg proxy compile error in VS2013

For questions and issues, please open an issue on the issue tracker. Commercial support, development and consultation is available through Protyposis Multimedia Solutions.

• Windows
• Visual Studio 2013 or 2015
• .NET Framework 4.0

Aurio has been developed for Windows but since only a small part depends on the OS, it should be portable to the Mono platform and other OS'.

Aurio comes with all required dependencies except for FFmpeg (which would blow up the repo size too much). See libs\ffmpeg\ffmpeg-prepare.txt for instructions on how to download and where to put FFmpeg.

Building works as easy as building any other Visual Studio solution and should work without problems if all dependencies have been set up correctly. Open the Visual Studio solution Aurio\Aurio.sln and build the Aurio project and optionally the Aurio.WaveControls project. The compiled DLLs will be built in each project's bin folder.

Not available yet. If you have any questions, feel free to open an issue!

Mario Guggenberger. 2015. Aurio: Audio Processing, Analysis and Retrieval. In Proceedings of the 23rd ACM international conference on Multimedia (MM '15). ACM, New York, NY, USA, 705-708. DOI=http://dx.doi.org/10.1145/2733373.2807408

/* Read a high definition MKV video file with FFmpeg,
 * convert it to telephone sound quality,
 * and write it to a WAV file with NAudio. */
var sourceStream = new FFmpegSourceStream(new FileInfo("high-definition-video.mkv"));
var ieee32BitStream = new IeeeStream(sourceStream);
var monoStream = new MonoStream(ieee32BitStream);
var resamplingStream = new ResamplingStream(monoStream, ResamplingQuality.Low, 8000);
var sinkStream = new NAudioSinkStream(resamplingStream);
WaveFileWriter.CreateWaveFile("telephone-audio.wav", sinkStream);

// Setup STFT with a window size of 100ms and an overlap of 50ms
var source = AudioStreamFactory.FromFileInfoIeee32(new FileInfo("somefilecontainingaudio.ext"));
var windowSize = source.Properties.SampleRate/10;
var hopSize = windowSize/2;
var stft = new STFT(source, windowSize, hopSize, WindowType.Hann);
var spectrum = new float[windowSize/2];

// Read all frames and get their spectrum
while (stft.HasNext()) {
    stft.ReadFrame(spectrum);
    // do something with the spectrum (e.g. build spectrogram)
}

// Setup the source (AudioTrack is Aurio's internal representation of an audio file)
var audioTrack = new AudioTrack(new FileInfo("somefilecontainingaudio.ext"));

// Setup the fingerprint generator (each fingerprinting algorithms has its own namespace but works the same)
var defaultProfile = FingerprintGenerator.GetProfiles()[0]; // the first one is always the default profile
var generator = new FingerprintGenerator(defaultProfile);

// Setup the generator event listener
generator.SubFingerprintsGenerated += (sender, e) => {
    // Print the hashes
    e.SubFingerprints.ForEach(sfp => Console.WriteLine("{0,10}: {1}", sfp.Index, sfp.Hash));
};

// Generate fingerprints for the whole track
generator.Generate(audioTrack);

// Setup the sources
var audioTrack1 = new AudioTrack(new FileInfo("somefilecontainingaudio1.ext"));
var audioTrack2 = new AudioTrack(new FileInfo("somefilecontainingaudio2.ext"));

// Setup the fingerprint generator
var defaultProfile = FingerprintGenerator.GetProfiles()[0];
var generator = new FingerprintGenerator(defaultProfile);

// Create a fingerprint store
var store = new FingerprintStore(defaultProfile);

// Setup the generator event listener (a subfingerprint is a hash with its temporal index)
generator.SubFingerprintsGenerated += (sender, e) => {
    var progress = (double)e.Index / e.Indices;
    var hashes = e.SubFingerprints.Select(sfp => sfp.Hash);
    store.Add(e);
};

// Generate fingerprints for both tracks
generator.Generate(audioTrack1);
generator.Generate(audioTrack2);

// Check if tracks match
if (store.FindAllMatches().Count > 0) {
   Console.WriteLine("overlap detected!");
}

var drumTrack = new AudioTrack(new FileInfo("drums.wav"));
var guitarTrack = new AudioTrack(new FileInfo("guitar.wav"));
var vocalTrack = new AudioTrack(new FileInfo("vocals.wav"));
var band = new TrackList<AudioTrack>(new[] {drumTrack, guitarTrack, vocalTrack});

new MultitrackPlayer(band).Play();

Aurio comes with a few tools and test applications that can be taken as a reference:

• Aurio.FFmpeg.Test decodes audio to wav files or video to frame images
• Aurio.Test.FingerprintingBenchmark runs a file through all fingerprinting algorithms and measures the required time.
• Aurio.Test.FingerprintingHaitsmaKalker fingerprints files and builds a hash store to match the fingerprinted files. The matches can be browsed and fingerprints inspected.
• Aurio.Test.FingerprintingWang2003 display the spectrogram and constellation map while fingerprinting a file.
• Aurio.Test.WaveViewControl is a test bed for the WaveView WPF control.
• Aurio.Test.MultitrackPlayback is a multitrack audio player with a simple user interface.
• BatchResampler resamples wave files according to a configuration file.
• MusicDetector analyzes audio files for music content.
• WaveCutter cuts a number of concatenated files into slices of random length.

Aurio has originally been developed for AudioAlign, a tool to automatically synchronize overlapping audio and video recordings, which uses almost all functionality of Aurio. Its sources are also available and can be used as a implementation reference.

The fingerprinting methods by Haitsma&Kalker and Wang are protected worldwide, and Echoprint is protected in the US by patents. Their usage is therefore severely limited. In Europe, patented methods can be used privately or for research purposes.

Copyright (C) 2010-2017 Mario Guggenberger mg@protyposis.net. This project is released under the terms of the GNU Affero General Public License. See LICENSE for details. The library can be built to be free of any copyleft requirements; get in touch if the AGPL does not suit your needs.