Quobyte, the innovator in high-performance storage solutions, has unveiled its new distributed File Query Engine at ISC High Performance 2024, enhancing its storage platform for enterprises and researchers. This engine allows users to query file system metadata at exceptional speeds, making it ideal for environments with massive data sets.
The File Query Engine offers a range of capabilities, including the ability to query user-defined metadata for AI/ML training, enabling users to label files with data directly instead of managing small “metadata files.” Additionally, administrators can quickly answer questions such as identifying space-consuming cold files or locating files owned by specific users. The File Query Engine replaces slow file system tree walks (“find”), offering a faster and more efficient alternative for large volumes.
Key to the File Query Engine’s performance is its integration with Quobyte’s distributed and replicated key-value store, which stores metadata. Unlike other products, Quobyte’s File Query Engine does not require an additional database layer, resulting in faster queries and significant resource savings. Queries are executed in parallel across all metadata servers for fast scans across the entire cluster or select volumes. This engine never operates on outdated data.
“The File Query Engine is a game-changer for our customers,” said Bjorn Kolbeck, CEO of Quobyte. “It streamlines the process of querying file system metadata, offering fast and efficient results even for large datasets, AI, and machine-learning workloads.”
The File Query Engine is part of Quobyte release 3.22 and is automatically available without any configuration. Users can run file metadata queries using the command-line tool “qmgmt,” which supports output in CSV or JSON formats. Additionally, queries can be initiated via the Quobyte API, providing flexibility and ease of use.
Source: PRNewsWire
