Baseten, the leader in high-performance AI inference, is excited to announce the beta release of Chains. Built on the foundations of Baseten’s open-source Truss framework, Chains enhances the performance of products using multiple AI models for compound AI systems, offering unmatched efficiency and scalability for businesses deploying complex AI inference workflows.
The landscape of artificial intelligence is expanding at an unprecedented rate, with companies increasingly reliant on multiple AI models running on heterogeneous hardware. Chains addresses the complexities of managing and optimizing these models at scale, ensuring businesses can maximize their AI investments without the usual performance bottlenecks.
“Chains represents a leap forward in our mission to help companies ship great AI products,” said Tuhin Srivastava, CEO of Baseten. “The next generation of products will use many different AI models combined with developer code, and Chains provides the framework and infrastructure needed to make those products excellent. With Chains, you can select the right GPU or CPU for every model or code component in your Chains workflow and scale them independently for the highest performance at the lowest cost.”
Also Read: Verus Introducing Llama 3 VerusGPT – Open-Source Training Data and Domain-Expert LLM for Verus & Other Uses
Key Benefits of Chains:
- Heterogeneous GPU Resourcing: Allows customers to select GPU and CPU resources for each component of a Chains workflow and auto-scale them independently.
- Business Logic Integration: Enables developers to orchestrate business logic with their ML models within a single Python program.
- Improved Developer Experience: Provides code-checking and type-checking by default to eliminate typical mistakes within complex workflows.
- Comprehensive Monitoring: Offers real-time insights into performance, resource utilization, and operational metrics, empowering businesses with actionable intelligence.
- Reduced Latency: Chains removes boilerplate code and unnecessary network hops to increase throughput and low latency at each step of a multi-model workflow.
Evolving the Truss Framework for Multiple Models
Building on the principles of the open-source Truss framework, Chains is its natural evolution, specifically engineered to address the heightened demands of multi-model, compound AI system deployments. Chains improves on Truss to provide a unified framework for orchestrating complex AI workflows spanning multiple component models, types of hardware, and arbitrary code in a way that prioritizes performance and efficiency at scale.
“It’s a fact that the more moving pieces you have in a product, the more critical it is that all of their interactions are fast and reliable. For companies like our customers that are using multiple AI models to power their core products, it becomes existential,” noted Amir Haghighat, CTO of Baseten. “We built Chains to help our customers power their multi-model use cases cost-efficiently, with low latency and high throughput. Chains builds on all the great work we’ve done for customers to improve inference performance and applies it to multi-modal use cases.”
Source: PRNewswire