Airbyte, the leading open-source data movement platform, announced new capabilities for moving data at scale for artificial intelligence and analytics workloads while ensuring governance so that organizations spend less time managing data pipelines while unlocking value from data.
“We make it possible for organizations to protect their data, ensuring that it doesn’t accidentally become accessible outside the organization by consumers of AI models,” said Michel Tricot, co-founder and CEO, Airbyte. “What we’re delivering today helps eliminate data silos – and improves data accessibility while still ensuring security and compliance to maintain data sovereignty without adding operational overhead.”
At its fourth annual move(data) conference, Airbyte announced new and updated products providing organizations with additional support for data movement while retaining sovereignty over their own first-party data, plus enhanced security, speed, and enhanced resource management.
Also Read: SecPod Unveils Saner Cloud: A Next-Generation CNAPP for Proactive Cybersecurity
Support for Unstructured Data and Portable Data Lake Formats
- Support for the Iceberg open standard for moving data into modern lakehouse architectures – the backbone for AI workloads with Large Language Models (LLMs), as well as modern analytics at scale.
- File transfer support for Google Drive, SharePoint, and OneDrive for movement of unstructured data such as PDF, video, image files – along with their metadata and permissions – making all of this data accessible for AI.
- An Enterprise Connector Bundle as a complement to Airbyte’s Cloud Teams and Self-Managed Enterprise. The bundle includes connectors for NetSuite, Oracle database with Change Data Capture (CDC), SAP HANA, ServiceNow, and Workday. This bundle of connectors streamlines how the world’s largest organizations access their most valuable financial, operational and human resource data. The Airbyte Enterprise products ensure that organizations can easily and securely extract critical data from complex and sensitive sources with governance controls for data privacy and compliance.
Sovereignty and Security
- New Mappers feature enables users to perform lightweight data transformations directly within the Airbyte interface with capabilities that include hashing, encrypting, renaming fields, and filtering rows helping organizations maintain compliance with data privacy regulations like GDPR and HIPAA.
- Support for AWS PrivateLink that reduces exposure to public internet traffic with secure, private cloud-to-cloud data transfers to ensure sensitive information is controlled.
- Support for OAuth 2.0 assures secure authentication while simplifying integrations by reducing manual work.
Simplifying Operations
- Resource management so that data syncs can be prioritized for critical data pipelines.
- Support for OpenTelemetry (OTEL) improves pipeline observability and monitoring with metrics for visibility into sync performance, API activity, and data volume movement.
- Updated Python Connector Developer Kit (CDK) that enables faster connector development.
- Performance improvements that increase speed of data syncs and reduce latency.
Airbyte makes moving data easy and affordable across nearly any source and destination, ensuring enterprises have accurate, timely data for analysis and decision-making. With over 900 contributors and a community of more than 230,000 members, Airbyte supports the largest data engineering community and is the industry’s only open data movement platform.
Source: BusinessWire