Engineering notes
Writing on streaming, data platforms and architecture.
Backpressure and Resilience: Cortex Pipelines That Don't Fall Over
How to keep Cortex.Streams pipelines alive under load: bounded buffers, backpressure strategies, concurrency and batching trade-offs, observability, graceful shutdown, and poison-message handling.
Change Data Capture in .NET: Stream Postgres, SQL Server, and MongoDB Changes as They Happen
A practitioner's guide to Change Data Capture with the Cortex Data Framework: stream every INSERT, UPDATE, and DELETE from PostgreSQL, SQL Server, and MongoDB into the same pipeline API, with initial snapshots, checkpointing, deduplication, and per-database prerequisites.
Choosing a Cortex State Store: RocksDB, SQL, Cassandra, DuckDB, and More
A decision guide for Cortex.States: why stateful streaming needs a store, how Cortex makes the backend pluggable behind one interface, and how to pick between in-memory, RocksDB, SQL Server, PostgreSQL, MongoDB, Cassandra, ClickHouse, DuckDB, and SQLite — with the same aggregation swapped across three backends.
CQRS in .NET with Cortex.Mediator
A practitioner's guide to building clean, testable .NET applications with Cortex.Mediator: the Mediator pattern, CQRS commands and queries, notifications, pipeline behaviors for logging, validation and transactions, and how it all fits Vertical Slice Architecture.
Windowing in Cortex: Tumbling, Sliding, and Session Windows in .NET
A practical guide to windowing in the Cortex Data Framework: why streams need windows, the difference between tumbling, sliding, and session windows, runnable C# for each, plus triggers, late-data handling, and the pitfalls that bite in production.
Introducing Cortex 3: Build a Real-Time Pipeline in .NET in 15 Minutes
Real-time stream processing in .NET no longer means reaching for the JVM. Cortex 3 is an open-source, modular .NET SDK with a fluent pipeline API, pluggable state, telemetry, and broker connectors. This hands-on quickstart walks from your first in-app stream to a stateful aggregation backed by Kafka.
One Pipeline, Any Broker: Kafka, Pulsar, RabbitMQ, and Cloud Queues Without Lock-In
How Cortex's source and sink operator abstraction lets you keep identical transformation logic while swapping Kafka, Pulsar, RabbitMQ, SQS, Azure Service Bus, and more — and what the abstraction does (and doesn't) unify.
Stream-Stream Joins: Matching Orders to Shipments in Real Time
A hands-on guide to windowed stream-stream joins in the Cortex Data Framework: correlate orders and shipments by key inside a time window, with the dual-source pattern, join semantics, late-event handling, and production memory notes.
Coupling Consumer-driven Services: Understanding Patterns and Anti-patterns
Microservices are socio-technical tools that help organizations scale by aligning software boundaries with team boundaries. When consumer-driven coupling creeps in, the benefits of loose coupling erode. This article explores common anti-patterns and the design principles that counteract them.