Databricks, the Data and AI company, announced an expanded collaboration with NVIDIA during the Data + AI Summit to optimize data and AI workloads by bringing NVIDIA CUDA accelerated computing to the core of Databricks’ Data Intelligence Platform. As data prep, curation and processing workloads are essential to using enterprise data for generative AI applications, the companies’ work together is designed to boost the efficiency, accuracy and performance of AI development pipelines for modern AI factories. Through this broadened alliance, Databricks is adding native support for NVIDIA GPU acceleration on the Databricks Data Intelligence Platform. Announcement builds on Databricks and NVIDIA’s collaboration to offer enriched experiences for enterprises, whether via training classical ML models, building and deploying generative AI applications, or optimizing digital twins.
“We’re thrilled to continue growing our partnership with NVIDIA to deliver on the promise of data intelligence for our customers from analytics use cases to AI,” said Ali Ghodsi, Co-founder and CEO at Databricks. “Together with NVIDIA, we’re excited to help every organization build their own AI factories on their own private data.”
“Data is the fuel for the generative AI industrial revolution, so reducing data processing energy demands with accelerated computing is essential to sustainable AI platforms,” said Jensen Huang, founder and CEO of NVIDIA. “Databricks is the pioneer of large-scale data processing. By bringing NVIDIA CUDA acceleration to Databricks’ core computing stack, we’re laying the foundation for customers everywhere to use their data to power enterprise generative AI.”
Driving Price-Performance with Photon and NVIDIA
Databricks plans to develop native support for NVIDIA-accelerated computing in Databricks’ next-generation vectorized query engine, Photon, to deliver improved speed and efficiency for customers’ data warehousing and analytics workloads.
Photon powers Databricks SQL, Databricks’ serverless data warehouse with industry-leading price-performance and total cost of ownership (TCO). Databricks and NVIDIA believe this collaboration will lead to the next frontier of price-performance.
Also Read: Alphawave Semi Collaborates with Arm on High-Performance Compute Chiplet
Creating Generative AI Factories with NVIDIA NIM
At COMPUTEX, Databricks’ open source model DBRX became available as an NVIDIA NIM microservice. NVIDIA NIM inference microservices provide models as fully optimized, pre-built containers for deployment anywhere in order to dramatically increase enterprise developer productivity by providing a simple, standardized way to add generative AI models to their applications.
Launched in March 2024, DBRX was built entirely on top of Databricks, leveraging all the tools and techniques available to Databricks customers and partners, and was trained with NVIDIA DGX Cloud, a scalable end-to-end AI platform for developers. Organizations can customize DBRX with enterprise data in order to create organization-specific, high-quality models, or as a reference architecture to build custom DBRX-style mixture of expert (MoE) models from the ground up.
Databricks’ Data Intelligence Platform offers the most comprehensive platform for building, evaluating, deploying, securing, and monitoring end-to-end generative AI applications. With Databricks Mosaic AI’s data-centric approach to generative AI, customers benefit from an open, flexible platform to easily scale generative AI applications on their unique data that are safe, accurate and governed.
Building on Recent Databricks Momentum
Today’s announcement comes on the heels of Databricks’ strategic acquisition of Tabular, a data management startup founded by Ryan Blue, Daniel Weeks, and Jason Reid. By bringing together the original creators of Apache Iceberg™ and Linux Foundation Delta Lake, the two leading open source lakehouse formats, Databricks will lead the way with data compatibility so that organizations are no longer limited by which of these formats their data is in. Databricks also recently announced it is doubling down on its Delta Sharing open ecosystem with product innovations and strategic partnerships, enabling customers to efficiently break down data silos and unlock AI innovation.
Source: PRNewswire
