Tag:data engineering
Articles tagged "data engineering", page 4.
- 18 MIN READ•Jul 13, 2026
REST Catalog V2: Fixing Iceberg Protocol Debt
Where the Iceberg REST catalog v1 protocol strains under BI and agent planning load, and how batch table loading, caching, and stateless servers address it.
REST Catalog v2IcebergProtocol - 30 MIN READ•Jul 6, 2026
The State of Apache Iceberg v4 in July 2026: What the Dev List Tells Us About the Format's Next Chapter
What the Iceberg v4 dev list tells us about adaptive metadata trees, single-file commits, column updates, and the format's next chapter in mid-2026.
Apache Icebergdata engineeringlakehouse architecture - 28 MIN READ•Jul 6, 2026
The State of Apache Arrow in 2026: Ten Years In, the Invisible Standard Is Everywhere
Apache Arrow at 10, ADBC, Flight SQL, nanoarrow, the AI reinterpretation, and how an in-memory standard eliminated the copy tax across the data stack.
Apache Arrowdata engineeringlakehouse architecture - 28 MIN READ•Jul 6, 2026
The State of Apache Parquet in 2026: The Quiet Format Enters Its Loudest Decade
Apache Parquet in 2026, variant types, geospatial, ALP encoding, footer redesign, the versioning debate, and how the decade-old format is renovating.
Apache Parquetdata engineeringlakehouse architecture - 29 MIN READ•Jul 6, 2026
The State of Apache Polaris in July 2026: From Incubating Catalog to the Governance Layer of the Open Lakehouse
Apache Polaris as a TLP, federation, credential vending, semantic layers, lineage, and how the open catalog became the governance plane.
Apache Polarisdata engineeringlakehouse architecture - 30 MIN READ•Jul 6, 2026
The State of Streaming to Apache Iceberg in July 2026: Every Path, Its Latency, and What to Do When Seconds Are Not Fast Enough
Every path for streaming data into Iceberg in 2026, Flink, Spark, Kafka Connect, broker-native, managed pipelines, with honest latency numbers.
Apache Icebergstreamingdata engineering - 26 MIN READ•Jul 6, 2026
Deterministic Data Engineering With AI Harnesses: Using Claude Code, Codex, Antigravity, and OpenCode for Data Work You Can Actually Trust
How to use AI agent harnesses for data engineering without losing determinism, reproducibility, and trust in your data pipelines and analytics.
data engineeringAI agentsClaude Code - 14 MIN READ•Jun 8, 2026
Apache Iceberg v4 Roadmap: Adaptive Metadata Trees, Single-File Commits, and the Delta Convergence
A deep technical breakdown of Apache Iceberg v4's proposed architecture: adaptive metadata trees, one-file commits, relative paths, column families.
apache-icebergopen-table-formatsdata-engineering - 24 MIN READ•May 23, 2026
Single-Node Data Engineering: DuckDB, DataFusion, Polars, and LakeSail
Optimize single-node data engineering with DuckDB, DataFusion, Polars, and LakeSail. Compare architectures and learn when to transition to Dremio MPP.
DuckDBApache ArrowDataFusion - 27 MIN READ•May 22, 2026
Apache Iceberg SCD Type 2 and CDC Patterns: Building Historical Lakehouse Tables
A deep dive into implementing Slowly Changing Dimension Type 2 (SCD Type 2) patterns and Change Data Capture (CDC) pipelines on Apache Iceberg, using.
apache icebergcdcscd type 2 - 24 MIN READ•May 22, 2026
Maintaining Apache Iceberg Tables: Compaction, Snapshot Expiration, and Orphan File Cleanup
An in-depth guide to orchestrating maintenance operations on Apache Iceberg tables, covering bin-packing, sort-based, Z-Order compaction, snapshot.
Apache IcebergCompactionData Engineering - 21 MIN READ•May 22, 2026
Common Misconceptions About Data Lakehouse and Apache Iceberg
Common lakehouse and Apache Iceberg myths, corrected: why a lakehouse is more than a database on S3, what catalogs do, and why partitions don't lock you in.
data lakehouseapache icebergdata engineering