Deepening interoperability with Google Cloud through open standards
by John Spencer and Jason Reid
Today, we're excited to announce that customers can now access the same copy of data from either Databricks Unity Catalog or BigQuery without duplication.This marks an important milestone for open standards and the broader data community.
As open table formats like Delta Lake and Apache Iceberg gained adoption, the next challenge was connecting the catalogs that manage them. First, Unity Catalog Open APIs enabled any external engine to access Unity Catalog data. Then, we introduced catalog federation, which lets customers register external catalogs, access foreign tables in Databricks without data copy, and unify governance across their entire data estate with Unity Catalog.
Catalog federation has become the industry standard for interoperability and Unity Catalog already supports bidirectional interoperability with data platforms including AWS, Microsoft Azure, and Snowflake. Today, we deepen our collaboration with Google Cloud, adding BigQuery and Google Cloud’s Lakehouse to the growing list of platforms that interoperate with Unity Catalog.
If you're a Databricks customer with data in BigQuery, or a BigQuery user who needs to access Unity Catalog tables, here's what's now possible and how to get started:
Read tables managed by Unity Catalog using Google BigQuery
To help you easily discover and analyze all your enterprise data stored in Unity Catalog, Google Cloud is announcing catalog federation in preview (learn more). Customers using engines such as BigQuery can now read Unity Catalog managed tables without data copy.
Read tables managed by Google Cloud Lakehouse using Databricks
We are excited to announce the private preview of Google Cloud’s Lakehouse federation (sign up for the preview). This new capability enables Databricks customers using Google Cloud to govern and read foreign Iceberg tables managed by Cloud Lakehouse. Customers can create a connection to their Lakehouse and then seamlessly mount foreign Iceberg tables in Unity Catalog.
Unified governance for federated data
Unity Catalog's policies, fine-grained access controls and lineage tracking will apply consistently whether an Iceberg table is created in Unity Catalog or federated from Google Cloud’s Lakehouse. Your business users can now use Genie to query data using natural language, as your entire data estate is secured and contextualized to business logic with Unity Catalog.
Today's announcement reflects how far the industry has come toward an open, interoperable data ecosystem. As we deepen our collaboration with Google Cloud, our vision is that customers no longer need to choose between innovation and interoperability, but instead can have both. Stay tuned for more ecosystem updates, including new features such as catalog metadata exchange via Iceberg REST catalog which allows you to define governance policies once and enforce them across platforms.
Get started
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。