Lumi Loglake has been introduced by Imply as an innovative feature incorporated into Imply Lumi that allows companies to be able to search for unstructured log data from within object storage.
Since the amount of telemetry data being generated through artificial intelligence systems, cloud architecture, and security operations has reached new heights, businesses have shifted toward lakehouse architectures and open-source storage systems. This trend can largely be attributed to the growing necessity for cost control and data retention. Nonetheless, because it is a routine job for observability and security analysts to deal with unstructured machine data, early explorations are often undertaken without even knowing which attributes or fields may turn out important.
Lumi Loglake is a solution that has been featured at Databricks’ Data + AI Summit and helps overcome this challenge by allowing companies to analyze and query unstructured logs directly from where they are stored, such as AWS S3, Delta Lake, or Apache Iceberg. This approach entirely obviates the necessity for using data catalogs, schemas, or data rehydration processes.
Engineered for the Economics of AI-Driven Telemetry
“AI is generating more telemetry than organizations can afford to index,” said Eric Tschetter, Chief Architect at Imply. “Lumi Loglake gives teams a new way to retain, search, and investigate that data without the cost and complexity of always-on indexing architecture. By bringing interactive log search directly to open storage, Loglake applies the economics and flexibility to lakehouse architectures to operationalize log data.”
Conventional observability frameworks typically force data teams to catalog, structure, and index logs before they can be searched. While this methodology was sustainable when enterprise data volumes were manageable, the sheer scale of modern AI outputs forces IT teams to make premature decisions regarding which data to store and index before knowing what will actually matter. This outdated paradigm introduces heavy operational friction and makes budget management incredibly difficult.
Also Read: IBM and ServiceNow Broaden Strategic Alliance to Revolutionize Enterprise Data for Scalable AI
By separating compute resources from underlying object storage, Lumi Loglake enables businesses to scale their telemetry retention independently of their infrastructure budget. Compute capacities dynamically spin up or down based on real-time query activity, which removes the financial burden of maintaining always-on indexing infrastructure while ensuring historical telemetry remains instantly accessible.
Consequently, engineering teams can house vastly larger volumes of information inside object storage, perform immediate inquiries, and target indexing efforts exclusively where they yield actionable value.
Minimizing Infrastructure Costs and Simplifying the Evolution of Observability
Enterprises adopting Lumi Loglake stand to achieve software cost reductions of 70% or higher, alongside hardware savings of up to 40%. These metrics are realized by migrating data out of expensive, always-on indexed tiers and into highly economical open storage setups like Amazon S3.
For organizations currently utilizing Splunk, this new deployment models a seamless and practical migration strategy. Instead of funneling massive telemetry datasets directly into Splunk’s proprietary indexing tier, enterprises can store historical logs in S3 while executing queries using SPL to generate native Splunk events. This allows security and IT analysts to preserve their familiar user workflows while giving the broader organization the economic advantages of open storage.
Furthermore, teams can query identical data repositories across multiple analytical tools without creating redundant storage copies. Supported query methods include accessing data via Splunk using SPL, Databricks using Spark SQL, Grafana using LogQL, and various business intelligence or AI platforms through standard ANSI SQL/JDBC connections.
Meeting the Industry Shift in Observability Architectures
As enterprise teams fundamentally re-evaluate how telemetry information is captured, preserved, and operationalized at scale, technology analysts point to an accelerating market demand for adaptable observability frameworks.
“Traditional observability pricing models are forcing teams into visibility tradeoffs at the exact moment AI systems are driving unprecedented telemetry growth,” said Stephen Catanzano, Principal Analyst at Omdia. “As infrastructure complexity and data volumes continue to rise, organizations are looking for more scalable approaches that improve operational flexibility without significantly increasing costs.”
“Observability and SIEM require analyzing mountains of data without prohibitive cost or complexity,” said Kevin Petrie, Vice President of Research at BARC. “Lumi reduces this tradeoff by searching logs in object storage without the overhead of indexing, cataloging, or schemas. Collecting all these logs in a cloud lakehouse also creates opportunities beyond observability, because telemetry data can enrich other analytics and AI initiatives.”
Loglake directly answers these industry-wide challenges, transforming data stored within cloud object storage into highly accessible assets for security information and event management (SIEM) and observability investigations, all without requiring technical teams to build or maintain supplementary indexing setups.
Lumi Loglake is available immediately to enterprise customers.
