Apache Iceberg vs Apache Hudi

Apache Iceberg vs Apache Hudi

The short answer: choose Iceberg when several engines need to read and write the same tables through an open catalog, and consider Hudi when your main workload is high-volume upserts by record key with incremental pulls in Spark or Flink. As of October 2026 the current releases are Iceberg 1.12.0 and Hudi 1.2.1.

Apache Iceberg and Apache Hudi (Hadoop Upserts Deletes and Incrementals) are both Apache Software Foundation-governed open table formats that solve the “mutable data in object storage” problem for data lakehouses. They approach the problem from different angles, reflecting their different origins and primary use cases.

Origins and Design Philosophy

Hudi was born at Uber to solve a specific operational problem: efficiently updating ride-sharing data in HDFS (later S3) without full partition rewrites. Iceberg was born at Netflix to solve a different problem: table format fragility, hidden partitioning complexity, and lack of atomic semantics in the Hive ecosystem.

Architecture Comparison

Table Types

Hudi has two native table types that correspond to write optimization strategies:

• Copy-on-Write (CoW): On each write, affected files are rewritten with updates applied. Reads are always clean (no merge needed), but writes are expensive.
• Merge-on-Read (MoR): Updates are stored in delta log files. Reads merge base files with deltas. Fast writes, more complex reads. Requires periodic compaction.

Iceberg also supports CoW and MoR, but these are per-operation write modes (write.delete.mode, write.update.mode, write.merge.mode), not different table types. Iceberg’s abstractions are more unified.

Transaction Log / Metadata

Hudi: Uses a timeline stored in a .hoodie/ directory. Each commit, clean, compaction, and rollback is an action on the timeline. The timeline is Hudi-specific and requires the Hudi library to interpret.

Iceberg: Uses a snapshot-based metadata tree (metadata JSON → manifest list → manifests → data files). The metadata is self-describing and structured: any client that can read the spec can navigate it.

Indexing

Hudi has built-in, native indexing support:

• Simple index (the default on Spark; joins incoming keys against keys in existing files).
• Bloom filter index (per-file bloom filters on record keys).
• Bucket index (hash-based, deterministic file placement, including consistent hashing).
• Record-level index (key-to-file mapping stored in Hudi’s metadata table, partitioned or global).
• Secondary and expression indexes (Hudi 1.x metadata-table indexes for reader-side pruning).
• HBase index (external HBase lookup for global key tracking).

Hudi’s indexing makes it extremely efficient for key-based upserts: given a set of record keys, Hudi can determine which files contain those keys without a full scan.

Iceberg: File-level statistics in manifests plus optional Parquet bloom filters. Iceberg relies on query engines, sorting, and compaction for clustering rather than native record-key indexing.

Feature Comparison

Incremental Processing: Hudi’s Strength

Hudi’s native incremental query mode is more powerful than Iceberg’s snapshot-diff approach for certain streaming use cases:

Hudi incremental query: Query only records changed since a specific commit, including which records were inserted, updated, or deleted with their exact keys.

# Hudi: native incremental read
spark.read.format("hudi") \
    .option("hoodie.datasource.query.type", "incremental") \
    .option("hoodie.datasource.read.begin.instanttime", "20260514000000") \
    .load("s3://bucket/orders/")

Iceberg incremental read: Spark incremental reads return rows appended between snapshots. For updates and deletes, Iceberg’s create_changelog_view Spark procedure computes row-level change records (with optional pre and post images by identifier columns), and format v3 row lineage adds stable row IDs. Hudi still tracks changed keys more directly.

For streaming CDC pipelines where you need to know precisely which keys changed (not just which files), Hudi’s native incremental semantics can be more precise.

Multi-Engine: Iceberg’s Strength

Iceberg’s REST Catalog specification enables any engine to read and write Iceberg tables with full catalog services (discovery, access control, credential vending). Hudi’s ecosystem is more Spark-centric:

• Dremio: Full Iceberg support (native). Hudi: external table support.
• Trino: Full Iceberg read and write support. Hudi: read-focused connector (now maintained in the Hudi repo as of 1.2.1).
• DuckDB: Iceberg extension reads tables and writes through a REST catalog. Hudi: limited.
• PyIceberg: Full Python client for Iceberg. Hudi: the hudi-rs project offers Python and Rust readers, with a smaller feature set.
• Flink: Both supported natively.

When to Choose Apache Iceberg

• Multi-engine architecture: Any team using more than one query engine.
• Open governance priority: Teams valuing Apache Foundation neutrality.
• AI/agent analytics: Dremio’s Agentic Lakehouse, MCP, semantic layer.
• Cloud-native deployment: AWS (S3 Tables, Glue, Athena), GCP (BigLake), Azure (Fabric).
• General lakehouse use case: Batch ETL, BI, streaming, ML.

When Apache Hudi May Be Preferred

• Record-key-centric upsert pipelines: Frequent updates to specific record keys by primary key.
• Spark or Flink streaming environments: Mature Hudi integrations for both engines.
• Existing Hudi investment: Organizations with significant existing Hudi tables and pipelines.
• Incremental CDC pipelines: Where Hudi’s native incremental semantics provide cleaner change tracking.

The Current Industry Landscape

As of 2026, Apache Iceberg has the broadest multi-engine support and the most active cloud vendor adoption. Hudi remains strong in Spark and Flink streaming upsert workloads, particularly in organizations where it was the original adoption, and it keeps adding features (Hudi 1.2 added VECTOR, VARIANT, and BLOB types and Lance file support). Teams that need both can use Apache XTable (incubating) to expose Hudi tables as Iceberg metadata.

For new lakehouse deployments, Apache Iceberg is the safer default for maximum future optionality.