Tag:data engineering
Articles tagged "data engineering", page 3.
- 38 MIN READ•Aug 19, 2026
Zero-Copy Warehouse Modernization: Moving to Apache Iceberg Without Downtime
A four-phase plan for moving a warehouse to Apache Iceberg without downtime: federate first, redirect new data, materialize selectively, and cut over gradually.
Apache Icebergwarehouse migrationzero-copy - 31 MIN READ•Aug 2, 2026
Apache Polaris 1.7.0 and the Quiet Work of Making a Catalog Trustworthy
Apache Polaris 1.7.0 deep dive: idempotent writes, semantic models, stricter credential vending, orphan cleanup, and what the upgrade asks of you.
Apache PolarisApache IcebergCatalog - 31 MIN READ•Aug 2, 2026
Designing Batch Pipelines That Write Well Into Apache Iceberg
How to design batch pipelines that write well into Apache Iceberg: commit strategy, partitioning, sort order, write-audit-publish, and maintenance done right.
Apache IcebergData EngineeringBatch Pipelines - 31 MIN READ•Jul 28, 2026
Running an Apache Iceberg Lakehouse With No Internet Connection
A practical guide to deploying an Iceberg lakehouse in air-gapped environments: component choices, artifact pipelines, identity without a cloud.
Apache IcebergAir-GappedData Engineering - 31 MIN READ•Jul 28, 2026
Why Iceberg V4 Wants to Retire Equality Deletes, and What Streaming Teams Should Do About It
Why equality deletes slow Iceberg reads for streaming upserts, what the V4 proposal changes, and how to convert them to deletion vectors with Flink and verify.
Apache IcebergStreamingData Engineering - 31 MIN READ•Jul 28, 2026
Surviving Commit Conflicts When Dozens of Writers Hit the Same Iceberg Table
Why Iceberg commits fail on stale metadata as writer counts grow, how isolation levels and retry settings change the outcome, and how to budget for contention.
Apache IcebergConcurrencyData Engineering - 31 MIN READ•Jul 28, 2026
The Parquet Versioning Problem, and Why Iceberg Cares About It
Parquet files have a version field that doesn't reliably signal feature requirements. A new versioning discipline is coming, borrowing from Iceberg's.
ParquetApache IcebergData Engineering - 31 MIN READ•Jul 25, 2026
The Apache Iceberg Market in the Middle of 2026
A survey of the Apache Iceberg market in July 2026: the state of the specification, platform support, the acquisition wave, the catalog contest, and how.
apache iceberglakehousedata engineering - 31 MIN READ•Jul 25, 2026
Three Vendors Are Rebuilding the Path From Transaction to Agent
Databricks, Snowflake, and SAP are closing the gap between operational databases and analytical platforms through acquisition, betting on different layers.
data engineeringapache iceberglakehouse - 20 MIN READ•Jul 13, 2026
Active Analytics Loops for AI Action Agents
The seven steps of an active analytics loop where agents watch signals, validate causes, and act with approval, plus tool design, guardrails, and case studies.
AI AgentsAnalyticsData Engineering - 18 MIN READ•Jul 13, 2026
Hybrid Lakehouse Design for Regulated Markets
How regulated firms keep data on-prem in Iceberg tables on private object storage while sharing approved metadata via Nessie or Polaris and read-only gateways.
Hybrid LakehouseRegulated MarketsData Engineering - 18 MIN READ•Jul 13, 2026
Iceberg Concurrency for AI Agent Writes
How Iceberg optimistic concurrency behaves when fleets of agents commit, and how partition isolation, commit queues, and ingestion gateways prevent conflicts.
Apache IcebergConcurrencyAI Agents