Open Source Pluggable 9 Backends

Cortex.States

Pluggable state stores for the Cortex Data Framework. Persist stream state in-memory, on disk, or across distributed databases—swap backends with a single line of code.

Documentation View on GitHub

9

Store Backends

1 Interface

IDataStore<K,V>

100%

Swappable
// Add a state store to any stream
var stream = StreamBuilder<SensorReading>
  .CreateNewStream("SensorPipeline")
  .Stream()
  .Filter(r => r.Value > 0)
  .Aggregate<string, double>(
    r => r.SensorId,
    (key, val, agg) => agg + val.Value,
    stateStoreName: "sensor-agg",
    // Swap backend here ↓
    stateStoreType: StateStoreType.RocksDb)
  .Sink(r => Console.WriteLine(r))
  .Build();
Architecture

One Interface, Many Backends

Every state store implements IDataStore<TKey, TValue>. Your stream code stays the same—just swap the store type and Cortex handles the rest.

In-Memory
RocksDB
IDataStore
<K, V>
SQL Server
PostgreSQL

Why State Stores Matter

Stateful operators—aggregations, windows, joins—need somewhere to keep running totals, session data, and lookup tables. State Stores give them a pluggable, fault-tolerant home.

  • Fault Tolerance
    Persistent stores survive crashes and restarts—no data loss.
  • Performance
    In-memory stores deliver microsecond reads; RocksDB balances speed with durability.
  • Scalability
    Distributed backends like Cassandra and MongoDB scale horizontally with your workload.
  • Seamless Integration
    Operators consume state stores automatically—no manual wiring required.
// Same pipeline, different store backends
// ① In-Memory (development)
stateStoreType: StateStoreType.InMemory
// ② RocksDB (embedded production)
stateStoreType: StateStoreType.RocksDb
// ③ SQL Server (enterprise)
stateStoreType: StateStoreType.SqlServer
// ④ PostgreSQL (cloud-native)
stateStoreType: StateStoreType.Postgres
// Swap one line—zero other changes ✓
State Store Catalog

Choose the Right Backend for Your Workload

From lightning-fast in-memory stores to distributed cloud databases—Cortex ships 9 production-ready backends.

In-Memory Store

Fastest possible reads and writes. Ideal for development, testing, and transient state that doesn't need to survive restarts.

Fastest Non-Persistent
RocksDB Store

Embedded, high-performance key-value store. Persistent, LSM-based, perfect for single-node production workloads.

Embedded Persistent
SQL Server Store

Enterprise-grade state management with full transactional support. Both key-value and structured store variants.

Enterprise Transactional
PostgreSQL Store

Reliable, ACID-compliant storage with advanced querying capabilities. Ideal for cloud-native deployments.

Cloud-Native ACID
SQLite Store

Lightweight, file-based persistent store. Zero configuration—perfect for edge, IoT, and single-process applications.

Lightweight Zero-Config
ClickHouse Store

Columnar analytics engine. Designed for high-performance analytical processing over massive state datasets.

Analytics Columnar
MongoDB Store

Document-oriented, schema-flexible storage. Great for complex state structures and rapid schema evolution.

Flexible Schema Distributed
Cassandra Store

Distributed, fault-tolerant, and highly available. Write-optimized for geo-distributed, always-on applications.

Distributed High Availability
DuckDB Store

In-process OLAP database. Blazing-fast analytical queries directly on state data without external infrastructure.

OLAP Embedded
Quick Reference

Choosing the Right Store

Store Persistence Speed Distributed Best For
In-Memory No ⚡ Fastest No Dev / test, transient state
RocksDB Yes ⚡ Very Fast No Single-node production
SQLite Yes Fast No Edge / IoT, single-process
SQL Server Yes Fast Optional Enterprise, transactional
PostgreSQL Yes Fast Optional Cloud-native, ACID
ClickHouse Yes ⚡ Very Fast Yes Analytics, columnar queries
MongoDB Yes Fast Yes Flexible schema, documents
Cassandra Yes Fast Yes Geo-distributed, always-on
DuckDB Yes ⚡ Very Fast No Embedded OLAP analytics
Why Cortex.States?

Built for Stateful
Stream Processing

State stores integrate directly with Cortex operators. Aggregations, windowing, and joins consume them automatically—no manual plumbing.

Explore State Stores
Aggregations

Running counters, sums, and averages persisted across stream restarts.

Windowed State

Tumbling, sliding, and session windows backed by durable storage.

Stream Joins

Lookup tables and co-partitioned state for stream-table and stream-stream joins.

Fault Recovery

Persistent stores resume exactly where they left off after a crash or deployment.

Use Cases

When You Need State

Financial Transactions

Running balances, fraud scores, and transaction histories that must survive restarts.

IoT & Sensor Data

Aggregate sensor readings over time and alert on threshold violations.

Session Tracking

Maintain user session state for real-time personalization and analytics.

Real-Time Analytics

Windowed aggregations over click-streams, logs, and event data at scale.

Add State to Your Streams

Pick the backend that fits your workload, plug it in with one line, and let Cortex manage the rest—persistence, recovery, and performance included.

Read the Docs View on GitHub Join Discord