The Era of Open Governance
A year after we open-sourced Unity Catalog (UC), the results are clear: openness isn’t just a principle, it’s working in practice.
Since then, hundreds of enterprises have adopted Unity Catalog as their foundation for open, interoperable governance across Delta Lake, Apache Iceberg, and every major engine in the modern data stack. What began as a commitment to open standards has evolved into a thriving ecosystem of open APIs, partner integrations, and customer impact at scale.
Today, Unity Catalog stands as the most widely adopted open catalog for data and AI. Data teams no longer have to make trade-offs across performance, interoperability, and governance; they can have it all.
From “why open?” to “open at scale”
In 2024, we open-sourced UC and launched UC Open APIs to enhance interoperability with external tools. These APIs make it simple for any engine to securely connect to Unity Catalog, read or write Delta and Iceberg tables, and apply governance automatically through credential vending and centralized access policies.
A year later, the ecosystem speaks for itself:
- Over 700 companies now use UC to centralize governance across multiple engines and tools.
- UC client SDKs see more than 1 million downloads per month, fueling cross-platform adoption.
- Partners like Starburst, ClickHouse, and Confluent have built deep integrations on top of UC Open APIs, extending governance well beyond Databricks.
This momentum proves that interoperability scales best when openness and governance work together.
The best catalog for Delta Lake and Apache Iceberg
Unity Catalog provides first-class support for Delta and Iceberg across governance, access, and performance. Through UC Open APIs and the Iceberg REST Catalog API, organizations can securely connect any engine through reading, writing, and creating tables while adhering to unified access policies.
Unity Catalog makes external access simple:
- Define access controls once, and UC automatically enforces them across clouds, engines, and formats.
- Credential vending issues temporary, scoped credentials behind the scenes, removing the need to configure cloud storage permissions or replicate policies manually.
- Extend governance to AI through a unified control plane that enables you to manage datasets, features, and model versions.
- Connect any tool or engine, from Spark and Trino to custom ML pipelines, via Unity REST APIs and the Iceberg REST Catalog API.
Take governance one step further by leveraging UC Managed Tables, where openness meets performance. These Databricks-optimized tables use Predictive Optimization and Liquid Clustering to deliver up to 20× faster queries and 50% lower storage costs while staying fully open and accessible through standard APIs. Managed Tables represent the new standard: centralized governance, open formats, and intelligent performance—all in one.
Industry and ecosystem momentum behind UC Open APIs
Over the past year, UC Open APIs have helped hundreds of organizations break format silos, unify governance, and extend interoperability across every part of their stack.
PepsiCo: Unified data governance across multi-engine analytics
PepsiCo runs a diverse analytics ecosystem with multiple compute engines, such as Spark on Kubernetes. Historically, these engines had to bypass UC and connect to external tables via path-based access. With UC Open APIs, PepsiCo can now adopt managed tables and have external engines access data through a single, centralized governance without requiring storage-level workarounds.
With Unity Catalog’s Open APIs, we’ve empowered our teams to use their preferred tools while maintaining governance and data consistency. We can leverage the benefits of managed tables within a truly interoperable data and AI platform that works across multiple compute engines.— Sudipta Das, Director Enterprise Data Operations
Coinbase: Graph queries at scale with credential vending
Coinbase relies on PuppyGraph to process terabytes of data daily. UC Open APIs and credential vending eliminate the need for ETL pipelines, letting Coinbase query Delta and Iceberg tables directly while enforcing policies and capturing audit logs.
Using Unity Catalog’s Open APIs, PuppyGraph can query 2TB+ of data daily with temporary credentials, analyzing service dependencies at scale – all while keeping governance centralized in UC.— Eric Sun, Head of Data Platform at Coinbase
Ecosystem partnerships
Unity Catalog sits at the heart of a growing ecosystem of partners, extending governance beyond Databricks:
- Confluent Tableflow integrates with UC to convert Kafka event streams into Delta tables. These tables are accessible via UC Open APIs and have governance policies automatically applied.
- ClickHouse enables its users to leverage UC Open APIs to discover and query Delta and Iceberg tables for real-time analytics and observability. With this integration, users can access Delta and Iceberg tables governed by UC directly from ClickHouse, while keeping UC at the center of governance.
- Starburst Trino has developed a Delta Lake connector that supports reading managed tables governed by Unity Catalog. To support writes to managed tables, Starburst has also integrated with the private preview of external writes via UC Open APIs.
Starburst shares Databricks’ vision of openness and interoperability across the data ecosystem. By integrating with Databricks Unity Catalog, we are enabling customers to create a single source of truth for all their data, with centralized governance and the flexibility to leverage the tools of their choice.— Justin Borgman, CEO, Starburst Data
What’s Next for UC Open Connectivity?
Unity Catalog continues to evolve as the most open and interoperable governance layer for the lakehouse. Here’s what’s coming next:
- External writes and table creation for UC-managed tables: Today, external engines can read UC-managed tables. Coming soon to Public Preview, we will enable external writes via Unity REST APIs and table creation directly from external clients. Interested in testing this out? Join our Private Preview.
- Note: We already offer full support for the Iceberg REST Catalog API, allowing external engines to read (Generally Available) and write (Public Preview) to Unity Catalog–managed Iceberg tables
- Secure access beyond tables: Credential vending is being extended to volumes, so unstructured data can be securely accessed from tools like Daft and Ray for AI/ML workflows.
- Easily migrate to UC Managed Tables: With a few simple commands, you’ll be able to convert UC external or catalog-federated foreign tables to fully managed tables—preserving Delta history, settings, permissions, and views.
- Unity Catalog 0.4 release: in the next release of UC (v0.4), we’re adding managed tables support and implementing credential renewal in the UC Spark client for long running jobs.
Get Started Today
Unity Catalog Open APIs are available for both Delta and Iceberg clients. You can start by:
Start building with UC Open APIs today and see how easy interoperability and unified governance can be. To get started with Unity Catalog, follow the guides for AWS, Azure, and GCP.
