Zilliz, the innovator behind the widely integrated open-source vector database Milvus, has announced the public preview of Zilliz Vector Lakebase. This major expansion of Zilliz Cloud integrates its established production vector database with a shared, lake-native structural foundation. The platform allows a unified collection of vectors to handle production queries, drive discovery sessions, and sustain multi-petabyte machine learning data pipelines simultaneously eliminating the need for data duplication, manual migration, or independent infrastructure stacks.
The launch preserves Zilliz Cloud’s high-velocity vector search core a technology utilized by over 10,000 enterprises and artificial intelligence engineering teams, including Zillow, OpenEvidence, Exa, Filevine, and MiniMax. Vector Lakebase extends this capability by adding interactive discovery, massive batch analytics, and federated search across external data lakes. Consequently, organizations can execute diverse workloads against a single logical copy of their data, with billing for on-demand and batch computational processes applied only when compute resources are actively running.
“Production vector search is and will remain at the heart of what Zilliz does it’s why thousands of teams choose Milvus and Zilliz Cloud, and it’s getting faster and more cost-efficient every release,” said Charles Xie, Founder and CEO of Zilliz. “Vector Lakebase is what we believe comes next: one data foundation where the same vectors can serve a production query, anchor a discovery session, and power a multi-petabyte training-data pipeline without copies, migration, or a parallel stack.”
The Strategic Value of a Centralized AI Data Foundation
Modern artificial intelligence operations have outgrown basic single-query retrieval frameworks. They operate as continuous feedback loops that serve data, analyze user feedback, refine datasets, and return to serving. Historically, this workflow has demanded separate siloed infrastructures for data serving, exploratory analysis, and heavy batch processing. Migrating billions of high-dimensional vectors between these disparate systems often requires days of engineering overhead. This operational barrier causes many teams to abandon iterative data refinement altogether, rendering critical datasets stagnant.
Vector Lakebase solves this logistical hurdle by establishing a zero-copy semantic data plane layered over shared lake-native storage. This architecture permits real-time production serving, exploratory data manipulation, and broad batch processing to execute seamlessly against one master data copy across gigabyte to petabyte scales.
“Teams asked for a way to keep their data in one place and run very different workloads against it from real-time agent memory to overnight semantic deduplication,” said Robert Guo, VP of Product at Zilliz and one of the architects behind Milvus. “Vector Lakebase delivers that through a unified storage layer on Vortex, tiered serving for the production path, and on-demand compute for everything else.”
Also Read: IBM and ServiceNow Broaden Strategic Alliance to Revolutionize Enterprise Data for Scalable AI
Five Core Technical Capabilities
The newly unified platform introduces five distinct architectural features designed to simplify the AI data lifecycle:
- Tiered Real-Time Serving: There are three tiers available on the platform that are designed for distinct performance requirements: Performance-optimized (more than 1,000 QPS, single digit milliseconds latency, totally in-memory); Capacity-optimized (between 100 to 500 QPS, less than 100 ms latency, memory-NVME); and Tiered-storage (10 to 50 QPS, approximate 100 ms latency, memory, NVME, and low cost object storage). Each tier provides guaranteed minimum recall of 95-98%, customizable to 99%+, with a 99.99% uptime SLA and Global Cluster regional high availability.
- On-Demand Search: Designed for highly variable workloads where infrastructure frequently sits idle, this pay-as-you-go pricing model bills directly for object storage and actual active compute rather than inflating serverless premiums. Internal benchmarks involving one billion 768-dimension vectors and 10 hours of monthly computational activity revealed a total cost of $318, compared to $4,937 under standard serverless models amounting to roughly 1/15th of the expense.
- External Data Lake Search: Featuring a zero-copy External Collection mode, this capability embeds advanced indexing and exhaustive search protocols directly into existing Iceberg, Lance, Parquet, and Vortex tables, executing incremental synchronization upon data refreshes without moving source data from its original repository.
- Full-Spectrum AI Search: This feature enables unified queries across dense and sparse vectors, unstructured text, JSON payloads, and geospatial coordinates. It supports hybrid retrieval systems, BM25 text ranking, regular expressions, multi-vector processing, and iterative queries. Output data can be re-ranked using integrations like Cohere, Voyage AI, Reciprocal Rank Fusion (RRF), and custom weighted boosting models.
- Unified Lake-Native Storage: Unified lake-native storage relies on Vortex, which is an open columnar storage format that has been optimized for quick and cheap random reads in contrast to Lance and Parquet storage formats. This storage system utilizes object-storage-friendly vector, JSON, and BM25 index to achieve reduction in read amplification by more than 90%. Thus, backfilling 100 million rows schema takes just a few minutes.
Through the integration of these work processes, teams working on the development of AIs will be able to integrate their parallel-serving cluster systems and batch systems into one system that features uniform indexing, automatic data versioning, and compute that scales down to zero.
