Skip to content

DRILL-8543: Add Support for Materialized Views#3036

Open
cgivre wants to merge 8 commits intoapache:masterfrom
cgivre:views
Open

DRILL-8543: Add Support for Materialized Views#3036
cgivre wants to merge 8 commits intoapache:masterfrom
cgivre:views

Conversation

@cgivre
Copy link
Contributor

@cgivre cgivre commented Feb 2, 2026

DRILL-8543: Add Support for Materialized Views

Description

This PR adds materialized view support to Apache Drill, enabling users to store pre-computed query results for improved query performance.

Features

  • SQL Commands: CREATE [OR REPLACE] MATERIALIZED VIEW, DROP MATERIALIZED VIEW, and REFRESH MATERIALIZED VIEW
  • Query Rewriting: Automatic query optimization using Calcite's SubstitutionVisitor to transparently rewrite queries to use materialized views when beneficial
  • Parquet Storage: MV data stored as Parquet files for efficient columnar access
  • Metastore Integration: Optional synchronization of MV metadata to Drill Metastore (Iceberg, RDBMS, MongoDB backends)

Implementation

  • New SQL parser classes for MV statements
  • MaterializedView data model with JSON serialization (.materialized_view.drill files)
  • MaterializedViewHandler for CREATE/DROP/REFRESH operations
  • MaterializedViewRewriter for query plan substitution
  • DrillMaterializedViewTable implementing Calcite's TranslatableTable
  • Metastore API extensions: MaterializedViews interface and MaterializedViewMetadataUnit
  • Iceberg metastore backend implementation for MV metadata

Configuration

  • planner.enable_materialized_view_rewrite (default: true) - Controls automatic query rewriting

Documentation

Added docs/dev/MaterializedViews.md with complete feature documentation

Testing

Added additional unit tests.

@cgivre cgivre self-assigned this Feb 2, 2026
@cgivre cgivre added enhancement PRs that add a new functionality to Drill doc-impacting PRs that affect the documentation performance PRs that Improve Performance major-update labels Feb 2, 2026
Copy link
Contributor

@letian-jiang letian-jiang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Materialized view is a powerful feature for analytic engine. 🥳

Copy link
Contributor

@letian-jiang letian-jiang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also add a plan-asserting test to ensure the query is correctly rewrite using MV.

@cgivre
Copy link
Contributor Author

cgivre commented Feb 8, 2026

@letian-jiang I believe I addressed your review comments. Could you please mark the review as complete so we can merge?
Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc-impacting PRs that affect the documentation enhancement PRs that add a new functionality to Drill major-update performance PRs that Improve Performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants