RisingWave: Event Streaming for Agentic AI — Replace Kafka + Flink + Database
文章目录
- Single-system architecture: RisingWave replaces the traditional streaming stack with one composable engine. It natively reads CDC (Change Data Capture) from PostgreSQL, MySQL, and other databases; consumes from Kafka, Kinesis, and webhooks; and exposes all data through a PostgreSQL-compatible interface. No Flink jobs, no separate serving database needed. Incremental materialized views: Unlike batch systems that recompute from scratch, RisingWave maintains streaming materialized views that update incrementally as new data arrives. You can join streams with tables, run windowed aggregations, and query the latest results with sub-second latency — all in SQL. Rust-powered performance: Built entirely in Rust, RisingWave achieves high throughput with predictable memory usage. Its compactor and hummock storage layer manage S3-compatible object stores efficiently, compacting and vacuuming stale data automatically. Benchmark results show competitive performance against Flink on Nexmark queries.
- RisingWave's GitHub has 1,479 open issues with active, technically detailed discussions. Here are highlights from recent conversations:
- A production user reported that S3 storage grows indefinitely even under moderate load. Developer hzxa21 responded asking for meta node and compactor logs: "We have built-in compaction and vacuum process in RisingWave to compact and delete stale data on object store. Can you share the meta node and compactor node logs?" After investigation, the user found Not enough core parallelism warnings in the compactor — the system was under-provisioned for compaction tasks relative to streaming parallelism. This sparked a discussion about the balance between throughput and storage hygiene in production deployments.
- A major refactor is underway to unify all streaming parallelism configuration options into a coherent system. Developer shanicky broke it into a 6-PR stack: "I think we need slt tests to reveal the parallelism for the combination of different options: streaming_parallelism (5 modes) × streaming_parallelism_for_ (5 modes) × parallelism types — giving comprehensive coverage before merging." This reflects RisingWave's growing maturity: moving from "it works" to "it's correct across all configurations."
- A contributor proposed decoupling synced log store progress from downstream backpressure. Developer kwannoel asked for structured benchmarks, prompting EdwinaZhu to post detailed methodology covering barrier completion times, core counts, and release-mode benchmarks. The PR splits the consumer-side into two stages (ReadingChunk and processing), reducing contention under high parallelism.
- RisingWave is a compelling choice for teams building real-time AI applications — whether you need streaming RAG context, autonomous agent memory, or real-time analytics pipelines. Its PostgreSQL compatibility means you can query live streams with tools you already know. With 1,400+ open issues and active contributions from companies like ByteDance (the original creators), it's one of the most actively developed streaming platforms in the Rust ecosystem. 📦 @risingwavelabs · GitHub Repository · ⭐ 8,981 stars · Apache 2.0 License
RisingWave is an open-source event streaming platform purpose-built for agentic AI workloads. Instead of stitching together Debezium, Kafka, Flink, and a serving database — each adding latency and operational complexity — RisingWave handles the entire pipeline: ingest from databases, webhooks, and event streams; process data incrementally in real time; and serve fresh results via PostgreSQL-compatible SQL. It recently rebranded from "streaming database" to "event streaming for AI," signaling a shift toward agent-first use cases like RAG context enrichment, autonomous workflow state management, and real-time decision support.
- Single-system architecture: RisingWave replaces the traditional streaming stack with one composable engine. It natively reads CDC (Change Data Capture) from PostgreSQL, MySQL, and other databases; consumes from Kafka, Kinesis, and webhooks; and exposes all data through a PostgreSQL-compatible interface. No Flink jobs, no separate serving database needed.
- Incremental materialized views: Unlike batch systems that recompute from scratch, RisingWave maintains streaming materialized views that update incrementally as new data arrives. You can join streams with tables, run windowed aggregations, and query the latest results with sub-second latency — all in SQL.
- Rust-powered performance: Built entirely in Rust, RisingWave achieves high throughput with predictable memory usage. Its compactor and hummock storage layer manage S3-compatible object stores efficiently, compacting and vacuuming stale data automatically. Benchmark results show competitive performance against Flink on Nexmark queries.
RisingWave's GitHub has 1,479 open issues with active, technically detailed discussions. Here are highlights from recent conversations:
A production user reported that S3 storage grows indefinitely even under moderate load. Developer hzxa21 responded asking for meta node and compactor logs:
"We have built-in compaction and vacuum process in RisingWave to compact and delete stale data on object store. Can you share the meta node and compactor node logs?"
After investigation, the user found Not enough core parallelism warnings in the compactor — the system was under-provisioned for compaction tasks relative to streaming parallelism. This sparked a discussion about the balance between throughput and storage hygiene in production deployments.
A major refactor is underway to unify all streaming parallelism configuration options into a coherent system. Developer shanicky broke it into a 6-PR stack:
"I think we need slt tests to reveal the parallelism for the combination of different options:
streaming_parallelism(5 modes) ×streaming_parallelism_for_(5 modes) × parallelism types — giving comprehensive coverage before merging."
This reflects RisingWave's growing maturity: moving from "it works" to "it's correct across all configurations."
A contributor proposed decoupling synced log store progress from downstream backpressure. Developer kwannoel asked for structured benchmarks, prompting EdwinaZhu to post detailed methodology covering barrier completion times, core counts, and release-mode benchmarks. The PR splits the consumer-side into two stages (ReadingChunk and processing), reducing contention under high parallelism.
RisingWave is a compelling choice for teams building real-time AI applications — whether you need streaming RAG context, autonomous agent memory, or real-time analytics pipelines. Its PostgreSQL compatibility means you can query live streams with tools you already know. With 1,400+ open issues and active contributions from companies like ByteDance (the original creators), it's one of the most actively developed streaming platforms in the Rust ecosystem.
📦 @risingwavelabs · GitHub Repository · ⭐ 8,981 stars · Apache 2.0 License