Almost every data organization now says it runs a lakehouse. In practice, the label covers everything from a folder of Parquet files on object storage queried by one Spark cluster to a governed, multi-engine platform serving dashboards, machine learning and AI agents from the same open tables. The difference between those two ends of the spectrum is not the technology logo on the architecture diagram; it is maturity across formats, catalogs, governance, operations and the ability to deliver trustworthy data to AI.
This article offers a practical five-stage maturity model for the lakehouse, a self-assessment checklist you can run with your team in an afternoon, and a forward-looking view of what “mature” is likely to mean between 2027 and 2030. It is designed for data leaders who need to decide where to invest next, and for architects who need a shared vocabulary to explain those choices.
Why a Maturity Model Matters Now
Three forces make an honest maturity assessment more urgent in late 2026 than it was two years ago.
- AI raised the bar for data. In a February 2025 press release, Gartner reported that 63% of organizations either do not have or are unsure whether they have the right data management practices for AI, based on a survey of 1,203 data management leaders, and predicted that through 2026 organizations will abandon 60% of AI projects unsupported by AI-ready data. A lakehouse that serves monthly reports well may still be far from ready for agents and retrieval-augmented generation.
- The format wars are cooling into interoperability. Apache Iceberg and Delta Lake now interoperate more than ever. Delta Lake’s UniForm keeps Iceberg metadata in sync with Delta commits so Iceberg readers can query Delta tables, and recent Delta releases introduced catalog-managed tables and an open Unity Catalog Delta API specification that any catalog provider can adopt. The strategic question has shifted from “which format” to “which catalog and governance model.”
- Platforms are absorbing operations. Managed table services and automatic optimization now handle compaction, snapshot expiry and cleanup for managed tables. Mature teams spend less time running jobs and more time setting policy and measuring outcomes.
The Five Dimensions of Lakehouse Maturity
Assess maturity across five dimensions rather than one overall score. Teams are often advanced in one and early in another, and the weakest dimension usually limits what the platform can deliver.
- Storage and formats: how consistently data is stored in open table formats with appropriate layout, partitioning and maintenance.
- Catalog and interoperability: whether a central catalog governs tables, and whether multiple engines can safely read and write through it.
- Governance and trust: access control, lineage, data quality, contracts, classification of sensitive data and auditability.
- Operations and economics: automation, observability, reliability engineering and cost transparency.
- Consumption and AI readiness: how well the platform serves BI, data science, applications and AI agents with fit-for-purpose, well-described data.
The Five Stages
Stage 1: Data Lake With Ambitions
Data lands in object storage as raw files, often Parquet or CSV, organized by folder conventions. One or two engines query it. There is no transactional table format, or it is used inconsistently. Access is controlled through storage permissions. Quality issues are discovered by users. Typical symptoms include duplicate “final” datasets, slow queries on small files and fear of changing anything upstream.
Stage 2: Table-Format Foundation
Core datasets have moved into Iceberg or Delta tables with ACID transactions and schema enforcement. A medallion or similar layered design exists. A catalog is in place, but it is tied to one engine or one team. Maintenance runs as hand-written jobs, sometimes inconsistently. Governance is mostly table-level grants. The platform reliably supports BI and batch data science, but cross-team reuse is limited.
Stage 3: Governed, Shared Platform
A central catalog governs most tables, with fine-grained access control such as row filters and column masks, end-to-end lineage and documented ownership. Data quality checks and data contracts protect critical pipelines. Maintenance is automated, either through managed services or standardized jobs. Cost is visible by domain. Multiple teams publish and consume data products. This is where many organizations that have invested seriously for a few years are heading.
Stage 4: Open, Multi-Engine and Real Time
The catalog exposes open interfaces such as the Iceberg REST catalog specification, so different engines (Spark, Flink, Trino, DuckDB, warehouse engines and even Postgres through extensions) read and write the same governed tables with credential vending rather than shared storage keys. Streaming ingestion lands data in minutes. Interoperability between formats is handled deliberately. Semantic definitions and metrics are managed centrally. Platform teams operate with SLOs for freshness and availability. Changing engines for a workload is a configuration decision, not a migration.
Stage 5: AI-Native Data Platform
The lakehouse serves AI agents and applications as first-class consumers. Structured, semi-structured and unstructured data are governed in the same catalog, with active metadata that describes meaning, quality, sensitivity and permitted use. Agents access data through governed interfaces with identity-aware policies and full audit trails. Operational stores for agent memory and features are integrated with analytical tables. Data observability covers inputs to AI as rigorously as inputs to finance reports. Maintenance and optimization are largely autonomous, with humans setting policy. Few organizations are fully here yet; it is the target state for 2027–2030.
The weakest dimension sets your real maturity. A lakehouse with excellent formats and engines but weak governance is not AI-ready, no matter how modern the stack looks.
Self-Assessment Checklist
Score each statement from 0 (not true) to 2 (consistently true). Total by dimension. Low scores show where to invest first. Run the exercise with platform engineers, governance leads and at least one major data consumer, because each group sees different gaps.
Storage and formats
- All curated data is stored in an open table format with ACID transactions.
- Partitioning, clustering and file sizes are designed for actual query patterns.
- Snapshot retention is defined per table class with business and compliance input.
- Compaction and cleanup run automatically and their outcomes are monitored.
Catalog and interoperability
- A single catalog (or a deliberate federation) governs all production tables.
- At least two engines can read and write governed tables without bypassing the catalog.
- Engines receive scoped, short-lived credentials rather than broad storage keys.
- Format interoperability (for example, Iceberg readers on Delta tables) is tested, not assumed.
Governance and trust
- Every production table has a named owner and documented purpose.
- Sensitive data is classified automatically and protected by policy.
- Column-level lineage is available from source to consumption.
- Critical pipelines have data contracts and automated quality checks.
Operations and economics
- Freshness and availability SLOs exist for key data products and are reported.
- Pipeline and data incidents follow an on-call and post-incident process.
- Platform cost is attributed to domains or products and reviewed regularly.
- Infrastructure and pipelines are deployed through version-controlled automation.
Consumption and AI readiness
- Business metrics are defined once in a semantic layer used by BI and AI tools.
- Unstructured content used for AI is governed and lineage-tracked like tables.
- AI agents access data through governed, audited interfaces under their own identities.
- Data quality and drift are monitored for datasets that feed models and agents.
A rough reading of the totals: if any dimension scores below half of its maximum, treat that dimension as your current stage ceiling regardless of how well the others score.
From Assessment to Roadmap: Moving Up One Stage at a Time
Skipping stages rarely works. A multi-engine, AI-native platform built on weak governance simply spreads untrusted data faster. Use these transitions as a guide.
- Stage 1 to 2: migrate the most-used datasets into an open table format, establish a layered design, introduce a catalog and schedule maintenance. Measure query performance and storage before and after to build momentum.
- Stage 2 to 3: consolidate on a central catalog, implement fine-grained access control and lineage, introduce data contracts for critical feeds, adopt managed maintenance, and start cost attribution by domain.
- Stage 3 to 4: expose open catalog interfaces, onboard a second and third engine for real workloads, move high-value sources to streaming ingestion, centralize metrics in a semantic layer and define platform SLOs.
- Stage 4 to 5: bring unstructured data and AI assets under the same governance, enrich active metadata, provide governed agent access with identity and audit, integrate operational stores for agent memory and features, and extend observability to AI inputs.
The Operating Model Behind Each Stage
Architecture alone does not move an organization up the model. Each stage also implies a different way of working, and mismatches between platform ambition and operating model are a common reason maturity programs stall.
- Stages 1 and 2 are usually run by a central data engineering team that builds and operates everything. That works while the number of sources and consumers is small, but the team becomes a bottleneck as demand grows.
- Stage 3 requires a platform team that provides self-service capabilities, plus domain teams that own their data products, with clear service levels between them. Governance moves from approval queues to policies enforced by the catalog.
- Stage 4 adds site reliability practices for data: SLOs, error budgets, incident reviews and capacity planning across several engines. FinOps becomes a shared responsibility between platform and domain teams.
- Stage 5 brings AI and product teams into the data operating model as consumers with their own identities, budgets and accountability, and adds governance roles focused on AI-specific risks such as permitted use, retention of agent interactions and evaluation data.
When you plan a stage transition, budget for these organizational changes alongside the technical work: new roles, updated responsibilities, training and the metrics each team will be measured on.
Worked Example (Hypothetical)
A hypothetical regional insurer runs the checklist with its platform, governance and analytics leads. Scores are strong for storage and formats: claims and policy data sit in Delta tables with automated optimization. Catalog and interoperability are moderate: everything is in one catalog, but only one engine is used. Governance is the weakest dimension: ownership is unclear for a third of tables, there is no column-level lineage, and sensitive fields are protected by convention rather than policy. AI readiness is low because the claims documents the business wants to use for a claims-assistant agent live outside the catalog entirely.
Although the business is pushing for the agent, the team concludes it is at Stage 2 overall, limited by governance. Its twelve-month roadmap therefore prioritizes ownership, automated classification and masking of sensitive data, lineage and contracts for claims feeds, and bringing claims documents under catalog governance. The agent is piloted in parallel on a narrow, well-governed slice of data, with the broader rollout tied to governance milestones. This sequencing keeps the AI ambition visible while avoiding an agent that exposes data it should not.
What “Mature” Will Mean Between 2027 and 2030
Maturity is a moving target. Based on the direction of open standards and vendor roadmaps visible in 2026, expect these shifts:
- Catalogs become the control plane. With commits increasingly ratified by catalogs and open catalog APIs spreading, the catalog will be where access, lineage, quality signals and optimization policy converge.
- Engines become interchangeable. Organizations will choose engines per workload, including lightweight single-node engines for small data, and expect them all to respect the same governance.
- Agents become primary consumers. In some domains, more queries will come from AI agents than from humans, making semantic clarity, identity-aware access and audit essential.
- Operational and analytical storage converge further. Managed Postgres and other operational engines integrated with lakehouse tables will reduce the number of pipelines between apps and analytics.
- Regulation keeps raising the floor. Requirements around data governance for AI systems, data portability and operational resilience will make lineage, documentation and access logs baseline expectations rather than differentiators.
Common Pitfalls in Maturity Programs
- Treating a platform purchase as a maturity jump; tools enable stages but do not deliver them.
- Averaging scores across dimensions and hiding a critical weakness.
- Running the assessment once; re-assess every six to twelve months as platforms and standards evolve.
- Ignoring people and process; ownership, on-call and product thinking matter as much as architecture.
- Chasing Stage 5 for every domain; some domains only need a solid Stage 3, and that is a perfectly good outcome.
Conclusion
A lakehouse is a journey, not a product. Assessing maturity across storage, catalog, governance, operations and AI readiness gives you an honest picture of where you stand and a defensible sequence for what to do next. Many organizations will find their biggest gains in governance and catalog interoperability rather than in yet another engine. Move one stage at a time, re-assess regularly, and aim for an open, catalog-centric platform that can serve both people and AI agents with data they can trust.
How Hanoi Technology Can Help
Hanoi Technology runs lakehouse maturity assessments and delivers the roadmap through projects, dedicated teams or managed services.
Further Reading
- Lack of AI-Ready Data Puts AI Projects at Risk – Gartner
- Simplifying Your Open Lakehouse with the Delta Kernel and the UC Delta APIs – Delta Lake
- Predictive optimization for Unity Catalog managed tables – Databricks Documentation
- pg_lake 3.5: Universal Iceberg REST Catalog Support for Postgres – Snowflake