Tag: Apache Iceberg
All the articles with the tag "Apache Iceberg".
- 27 MIN READ•May 22, 2026
Apache Iceberg SCD Type 2 and CDC Patterns: Building Historical Lakehouse Tables
A deep dive into implementing Slowly Changing Dimension Type 2 (SCD Type 2) patterns and Change Data Capture (CDC) pipelines on Apache Iceberg, using PySpark and Dremio.
apache icebergcdcscd type 2 - 25 MIN READ•May 22, 2026
Setting Up an AWS-Native Open Lakehouse: Querying Apache Iceberg with AWS Athena and AWS Glue Catalog
A comprehensive guide to building an open, high-performance lakehouse on AWS using Apache Iceberg, AWS Glue Catalog, Amazon S3, and S3 Tables, with query acceleration via the Dremio engine.
Apache IcebergAWS AthenaAWS Glue Catalog - 24 MIN READ•May 22, 2026
Apache Iceberg Catalogs Explained: REST, Glue, Hive Metastore, Polaris, Nessie, and Snowflake
A deep dive into Apache Iceberg catalog architecture, comparing REST catalogs, AWS Glue, Project Nessie, Polaris, and Snowflake. Learn catalog role, credential vending, and cross-engine configurations.
apache icebergcatalogsNessie - 24 MIN READ•May 22, 2026
Maintaining Apache Iceberg Tables: Compaction, Snapshot Expiration, and Orphan File Cleanup
An in-depth guide to orchestrating maintenance operations on Apache Iceberg tables, covering bin-packing, sort-based, Z-Order compaction, snapshot expiration, and orphan file removal, with query acceleration details for the Dremio engine.
Apache IcebergCompactionData Engineering - 24 MIN READ•May 22, 2026
Apache Iceberg with Spark: Create, MERGE, Upsert, and Evolve Tables End to End
A comprehensive developer guide to configuring Apache Spark with Apache Iceberg, executing transactional writes, and managing schema evolution.
apache sparkapache icebergdata engineering - 21 MIN READ•May 22, 2026
Common Misconceptions About Data Lakehouse and Apache Iceberg
Addressing common search queries and reader confusion about Data Lakehouse architectures, Apache Iceberg catalogs, partitions, and lock-in.
data lakehouseapache icebergdata engineering - 9 MIN READ•Apr 29, 2026
What Are Table Formats and Why Were They Needed?
Table formats like Apache Iceberg solved the ACID, schema, and performance problems that turned data lakes into data swamps. Here is how each one works.
data lake table formatsApache IcebergDelta Lake - 4 MIN READ•Mar 7, 2026
Agentic Analytics on the Apache Lakehouse
If you grant a Large Language Model direct access to a raw Amazon S3 bucket filled with Parquet files, it will fail to answer your business questions....
data lakehouseapache icebergapache parquet - 4 MIN READ•Mar 7, 2026
What is Apache Arrow? Erasing the Serialization Tax
If you pull a million records from a database into a Python notebook, the query runs instantly, but the transfer feels endlessly slow. Your compute en...
data lakehouseapache icebergapache parquet - 5 MIN READ•Mar 7, 2026
What is Apache Iceberg? The Table Format Revolution
If you drop ten thousand Parquet files into an S3 bucket, you have a data swamp. You do not have a database. To run SQL queries against those files sa...
data lakehouseapache icebergapache parquet - 4 MIN READ•Mar 7, 2026
What is Apache Parquet? Columns, Encoding, and Performance
If you ask a data analyst to calculate the average transaction amount for the month of July using a massive CSV file, the compute engine must read eve...
data lakehouseapache icebergapache parquet - 21 MIN READ•Mar 7, 2026
What is Apache Polaris? Unifying the Iceberg Ecosystem
Treating thousands of Parquet files as a unified database table requires a brain. Apache Iceberg provides the metadata structure to do this, but the Iceberg specification alone does not manage security roles, handle network requests, or broker credentials. You need an open catalog service to orchestrate those root metadata pointers. Apache Polaris serves as that open-source, vendor-neutral brain. This comprehensive guide explains the catalog fragmentation war, open governance under the Apache Software Foundation, role-based access control hierarchies, credential vending vs IAM sprawl, and how Polaris powers Dremio's agentic query acceleration.
data lakehouseapache icebergapache parquet