Streamiz Kafka .NET is .NET stream processing library for Apache Kafka.

It's allowed to develop .NET applications that transform input Kafka topics into output Kafka topics. It's supported .NET Standard 2.1.

It's a rewriting inspired by Kafka Streams.

Finally it will provide the same functionality as Kafka Streams.

This project is being written. Thanks for you contribution !

• End May 2020 - Beta 0.1.0 - All stateless processors, Exactly Once Semantic, InMemory store
• End August 2020 - Beta 0.2.0 - All statefull processors, Global Store
• End October 2020 - Beta 0.3.0 - Persistent state store (eg: RocksDB Store)
• End 2020 / Begin 2021 - 1.0.0 RC1 - Processor API, Metrics, Interactive Queries

Read the full documentation on https://lgouellec.github.io/kafka-streams-dotnet/

Nuget packages are listed to nuget.org

Install the last version with :

dotnet add package Streamiz.Kafka.Net --version 0.1.0-beta

There, a sample streamiz application :

static void Main(string[] args)
{
    CancellationTokenSource source = new CancellationTokenSource();
    
    var config = new StreamConfig<StringSerDes, StringSerDes>();
    config.ApplicationId = "test-app";
    config.BootstrapServers = "192.168.56.1:9092";
    config.SaslMechanism = SaslMechanism.Plain;
    config.SaslUsername = "admin";
    config.SaslPassword = "admin";
    config.SecurityProtocol = SecurityProtocol.SaslPlaintext;
    config.AutoOffsetReset = AutoOffsetReset.Earliest;
    config.NumStreamThreads = 2;
    
    StreamBuilder builder = new StreamBuilder();

    builder.Stream<string, string>("test")
        .FilterNot((k, v) => v.Contains("test"))
        .To("test-output");

    builder.Table("topic", InMemory<string, string>.As("test-ktable-store"));

    Topology t = builder.Build();
    KafkaStream stream = new KafkaStream(t, config);

    Console.CancelKeyPress += (o, e) => {
        source.Cancel();
        stream.Close();
    };

    stream.Start(source.Token);
}

• Topology description
• Refactor topology node processor builder
• Subtopology impl
• Unit tests (TestTopologyDriver, ...)
• [WIP] Statefull processors impl
• [WIP] Global state store
• Task restoring
• Processor API
• Repartition impl
• Rocks DB state implementation
• Optimizing Kafka Streams Topologies
• Interactive Queries
• Metrics

Owners:

• lgouellec

Maintainers:

• lgouellec
• mmoron

Streamiz Kafka .Net is a community project. We invite your participation through issues and pull requests! You can peruse the contributing guidelines.

When adding or changing a service please add tests and documentations.

You can found support here