New Delta Lake Support Simplifies Data Management, Allows Enterprises to Quickly Access and Integrate Data to Build LLMs
Fivetran, the global leader in data integration, announced support for Delta Lake on Amazon Simple Storage Service (Amazon S3), further broadening its support for Amazon S3 as a data lake destination. Hundreds of thousands of data lakes run on top of Amazon S3, an object storage service from Amazon Web Services (AWS) that offers industry-leading scalability, data availability, security and performance. Today’s news means Fivetran customers can land data in Amazon S3 and easily access their Delta Lake tables.
Data lakes are uniquely suited for handling the massive amounts of unstructured and semi-structured data due to their flexibility and scalability. Fivetran’s automation transforms data lakes from traditionally ungoverned repositories of data into organized, governed, user-friendly data stores, enabling organizations to quickly access and leverage data to build a variety of use cases, including predictive analytics, generative artificial intelligence (AI) applications, machine learning (ML) models and large language models (LLMs).
In April Fivetran announced support for Amazon S3 with Apache Iceberg, another leading high-performance format, with availability also announced today at AWS re:Invent 2023.
“We are thrilled to enable our customers to seamlessly leverage Delta Lake on Amazon S3,” said Fraser Harris, VP of Product at Fivetran. “Data lakes have proven to be the ideal foundation for machine learning, AI and generative AI projects. This enhancement represents a significant step forward in simplifying data management for such initiatives.”
Also Read: Reltio Accelerates Cloud-native Master Data Management (MDM) with AI Innovations
Fivetran brings data governance capabilities, industry-leading security and compliance, cost efficiency, and ease of use to data destinations. Fivetran’s no-code platform offers enterprises a simple, flexible way to move data from nearly any data source to any destination. Whether customer data is found in on-premises databases or data warehouses, data lakes, Software-as-a-Service (SaaS) apps, files, or events, Fivetran can move and replicate it from one source to another with 99.9% uptime. Common use cases range from migrating strategic data workloads from a cloud data warehouse to data lake for use in AI/ML modeling or generative AI application building, to replicating entire production on-premises databases in the cloud – using change data capture to keep cloud data in sync at all times.
By choosing Fivetran, customers can leverage these capabilities and avoid the complexity and lack of governability that may hinder data lake adoption at scale. Fivetran’s data platform automatically converts customer data to Delta Lake format and ensures data quality by anonymizing personally identifiable information (PII), cleansing and normalizing the data.
Fivetran’s 400+ pre-built connectors and fully managed data pipeline platform supports on-premises and cloud databases, data warehouses, SaaS applications, events and files. Fivetran can also create custom connectors which means that no matter the data source, Fivetran can support it – so customers don’t have to spend precious engineering resources on connector development. This broad portfolio of source compatibility enables customers to unify their data in the data lake, regardless of where it currently resides.
SOURCE: BusinessWire
