Open Data Lakehouse Governance: Breakthrough for Apache Iceberg v3 and Polaris
🔄 Update — 02. July 2026: Iceberg v3 GA and the Escalating Catalog War (Polaris vs. Unity Catalog)
With the general availability of the Iceberg v3 specification, the open table format war has been decisively won by Apache Iceberg. Major platforms including Snowflake, Databricks, and AWS have rolled out native or public preview support for Iceberg v3 features such as deletion vectors, row lineage, and the Variant data type. As a result, the competitive frontier has shifted from physical storage layouts to the metadata and catalog governance layers, led by Apache Polaris and Databricks Unity Catalog.
Was ist neu? / What’s new?
- Iceberg v3 GA & Platform Support: Apache Iceberg v3 introduces advanced deletion vectors and the Variant data type; Snowflake has made v3 generally available, Databricks provides public preview support, and AWS has launched S3 Tables.
- REST-Based Catalog Battle: The focus has shifted to REST-compliant catalogs, primarily Apache Polaris (which graduated to a top-level Apache project in Feb 2026) and Databricks’ open-source Unity Catalog.
- Server-Side Scan Planning: REST catalogs introducing server-side scan planning present new control points and governance mechanisms for enterprise cloud platforms.
Warum es den Artikel ergänzt / Why this adds to the article
This update marks the definitive victory of Apache Iceberg in the format war and demonstrates how the front line of open data lakehouse governance has moved from table formats to metadata catalogs.
🔄 Update — 23. June 2026: SQL-based GenAI Integrations for Apache Iceberg Lakehouses
Red Hat has introduced new integration guides for connecting Apache Iceberg lakehouses directly to LLMs using SQL on OpenShift AI. This advancement enables developers to run AI queries natively from their open data lakehouse. Additionally, EnterpriseDB released sovereign blueprints for secure database-lakehouse configurations built on Iceberg.
Was ist neu? / What’s new?
- SQL & GenAI Integration: Red Hat OpenShift AI provides guides to query LLMs directly on Apache Iceberg tables using SQL.
- Sovereign Lakehouse Blueprints: EnterpriseDB introduced architectures to build secure, sovereign database-lakehouse environments.
Warum es den Artikel ergänzt / Why this adds to the article
This integration proves that Apache Iceberg is not only the standard for data governance, but is also rapidly becoming the foundational platform for secure, SQL-driven enterprise Generative AI.
🔄 Update — 09. June 2026: Data Lakehouse Emerges as Default Enterprise Analytics & AI Backbone
Analyst reports and a recent CIO feature confirm that the data lakehouse has become the default architectural choice for enterprise analytics and AI workloads. This shift is driven by the maturation of lakehouse platforms from Databricks, Microsoft Fabric, and Snowflake, alongside the critical need for data consolidation for GenAI. Open table standards reduce vendor lock-in and enable seamless cross-platform interoperability.
Was ist neu? / What’s new?
- Standard Infrastructure: The data lakehouse is now central to enterprise data strategy, moving from a trend to the default architecture.
- AI Acceleration: Generative AI data consolidation requirements are heavily driving the adoption of lakehouse architectures.
- Cross-Platform Interoperability: Open standards like Apache Iceberg v3 and Delta Lake are proving crucial in enabling multi-engine access.
Warum es den Artikel ergänzt / Why this adds to the article
This update reinforces the thesis of the original article: the open governance standards (Apache Iceberg and Polaris) have successfully paved the way for the data lakehouse to become the enterprise standard.
Summary
The open data lakehouse governance landscape consolidated this week around Apache Polaris and Apache Iceberg v3. Cloudera announced the adoption of Apache Polaris as its open catalog for Iceberg-based lakehouse architectures. Snowflake integrated Polaris into Horizon Catalog with bidirectional Iceberg interoperability. Simultaneously, Databricks made Iceberg v3 generally available. This convergence signals that Apache Iceberg (plus Polaris) is becoming the de facto standard for open, governed data access across multi-engine, multi-cloud environments, drastically reducing vendor lock-in for enterprise data platforms.
What happened?
- Cloudera Announcement (June 4): Official adoption of Apache Polaris as the open-source catalog built around the Iceberg REST Catalog spec to improve interoperability across hybrid/multi-cloud ecosystems.
- Snowflake Horizon Integration (June 2): Integration of Apache Polaris into Horizon Catalog, allowing external engines to read/write Snowflake-managed Iceberg tables and vice versa.
- Databricks Iceberg v3 GA: Databricks made Iceberg v3 generally available during the same period.
- Iceberg v3 Features: The new version brings Deletion Vectors, VARIANT data type, row lineage, and deeper Unity Catalog integration.
Why it matters
The years-long “format war” between Delta Lake and Apache Iceberg seems to be decided in favor of Iceberg, as all major platform providers now offer native support. For enterprises, this means significantly higher flexibility: data can be stored in an open format while different engines (Snowflake, Databricks, Cloudera, Starburst) can access it simultaneously without compromising governance. This lowers switching costs and enables “best-of-breed” architectures.
Evidence
- Cloudera Press Release: Official confirmation of Polaris adoption on June 4, 2026.
- Snowflake Summit News: Announcement of the new framework for interoperable enterprise data.
- LinkedIn Trends: Leading data infrastructure experts note that Iceberg now dominates the market.
- Technical Documentation: Iceberg v3 release notes confirm the introduction of Deletion Vectors and bidirectional access.
Analysis
The convergence on Apache Polaris as a catalog standard is arguably even more significant than the table format itself. An open catalog like Polaris acts as a “source of truth” for metadata across different clouds and engines. The fact that Snowflake – traditionally a more closed system – open-sourced Polaris and Cloudera is now adopting it shows the immense market pressure toward open governance. The advantage lies in decoupling storage, metadata (catalog), and compute.
Practical Takeaways
- Standardization: New data lakehouse projects should primarily use Apache Iceberg as the table format.
- Catalog Strategy: Evaluate Apache Polaris as a central, vendor-neutral metadata catalog for multi-cloud scenarios.
- Assess Migration: Existing Delta Lake installations should be reviewed for long-term interoperability, especially if multi-engine access is required.
Open Questions
- How quickly will the community adopt Polaris compared to established proprietary solutions (like AWS Glue)?
- Will Databricks further open its Unity Catalog strategy to keep up with the Polaris momentum?
- How performant is bidirectional interoperability in extremely large production systems?