GenDB is a Generative Query Engine that uses LLM agents to generate instance-optimized query execution code, tailored to your specific data, workloads, and hardware.
☝ Interactive guided tour • Step-by-step visualization • Try your own data
3.2x
faster than DuckDB on TPC-H
6.8x
faster than DuckDB on SEC-EDGAR
462x
faster than PostgreSQL on TPC-H
280x
faster than PostgreSQL on SEC-EDGAR
What is GenDB?
Five specialized LLM agents collaborate through a structured pipeline to generate optimized storage, indexes, and standalone native executables — all tailored to the specific data, workload, and hardware.
Agent 1
Profiles hardware, samples data, extracts workload characteristics
Agent 2
Designs layouts with encoding, compression, indexes, and zone maps
Agent 3
Generates resource-aware execution plans adapted to data and hardware
Agent 4
Implements plans as optimized native code with SIMD and parallelism
Agent 5
Iteratively refines code using runtime profiling feedback
Why GenDB?
Today, every new use case demands either a painful extension or an entirely new system:
Option 2 — Build a new system
DuckDB, Umbra, ClickHouse, Milvus, Pinecone, InfluxDB, Neo4j …
Each requires years of engineering and huge monetary costs.
Option 3 — Generate
Use LLMs to generate per-query execution code. No extension wrestling, no multi-year engineering. New techniques become reachable through prompt updates.
⚡
Instance-optimized code exploits exact data distributions, join selectivities, group cardinalities, and hardware characteristics. No general-purpose engine can match this.
⚙
Integrating new techniques requires prompting, not re-engineering. Semantic queries, GPU-native code — all reachable through prompt updates.
Leaderboard
Total query execution time across all queries. GenDB variants use different LLM backbone models. All systems run on identical hardware with full parallelism enabled.
TPC-H (SF10, ~10GB)
SEC-EDGAR (3yr, ~5GB)
| # | System | Total Time | vs. Best GenDB | Relative |
|---|
| # | System | Total Time | vs. Best GenDB | Relative |
|---|
Model Comparison
Different LLM backbone models offer different trade-offs between generated code quality, generation time, and cost. Ranked by average query execution time.
Language Comparison
We select the best-performing C++ binary for each TPC-H query from a GenDB run, then give Claude Code (Opus 4.6) 5 iterations to analyze, profile, and improve — first for optimized C++, then for a full Rust rewrite.
GenDB-generated code with standard compilation.
241 ms
total (5 queries)
Aggressive flags, madvise tuning, parallelized joins, thread optimization.
185 ms
total — 1.30x faster
Full rewrite with rayon, memmap2, unsafe bounds-check elimination.
283 ms
total — competitive main_scan
| Query | Original C++ | Optimized C++ | Rust | Best |
|---|---|---|---|---|
| Q1 | 49.8 ms | 39.2 ms | 71.7 ms | Opt. C++ |
| Q3 | 25.0 ms | 26.0 ms | 52.5 ms | Orig. C++ |
| Q6 | 31.8 ms | 35.5 ms | 23.7 ms | Rust |
| Q9 | 85.4 ms | 64.4 ms | 101.9 ms | Opt. C++ |
| Q18 | 49.2 ms | 20.1 ms | 32.8 ms | Opt. C++ |
| Total | 241.2 ms | 185.2 ms | 282.6 ms | Opt. C++ (1.30x) |
Key findings: Optimized C++ achieves a 1.30x overall speedup, with Q18 showing the largest gain (2.44x) from parallelized join building.
Rust wins on Q6 (zone-map scan with get_unchecked) but carries ~30ms per-query overhead from mmap page table setup, penalizing short queries.
The Rust main_scan compute times are competitive with C++, suggesting the overhead is structural rather than algorithmic.
We plan to introduce a dedicated Code Refiner agent to the pipeline, responsible for low-level, implementation-level optimizations — to automatically achieve these gains as part of the standard GenDB workflow.
Roadmap
GenDB is under active development. Every step follows three principles:
Completed
Multi-agent pipeline for analytical queries. Evaluated on TPC-H and SEC-EDGAR, outperforming DuckDB, Umbra, ClickHouse, MonetDB, and PostgreSQL.
In Progress
Agents learn from past runs, accumulate optimization experience, and improve generation quality over time — without retraining the underlying LLMs.
Planned
Generate CUDA and GPU-accelerated code targeting libcudf for cost-efficient GPU analytics, not just CPU.
Planned
Generate code for multimodal data — images, audio, text — with AI-powered operators, moving beyond SQL’s relational model.
Planned
Reusable operators across queries, query template generation, hybrid execution with traditional DBMS, and further cost reduction as LLMs become faster and cheaper.
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。