Posts - Page 27
All the articles I've posted, page 27.
- 7 MIN READ•Apr 29, 2026
What Are Lakehouse Catalogs? The Role of Catalogs in Apache Iceberg
Lakehouse catalogs store metadata pointers, manage namespaces, and enforce access control. Here is the complete catalog landscape from Polaris to Glue.
lakehouse catalogsIceberg REST catalogApache Polaris - 8 MIN READ•Apr 29, 2026
Buffer Pools, Caches, and the Memory Hierarchy
Databases use buffer pools, column caches, and result caches to keep hot data in RAM. Here is how each caching strategy works and what happens when data.
database cachingbuffer poolcache eviction - 8 MIN READ•Apr 29, 2026
Writing to an Apache Iceberg Table: How Commits and ACID Actually Work
Here is exactly how an engine writes to an Iceberg table, step by step, from data files through the atomic commit that makes ACID guarantees possible.
writing to Apache IcebergIceberg ACID transactionsatomic commit - 8 MIN READ•Apr 29, 2026
Volcano, Vectorized, Compiled: How Engines Execute Your Query
The Volcano model processes one row at a time. Vectorized execution processes batches with SIMD. Code generation fuses operators into compiled code.
query execution modelsvectorized executionvolcano iterator model - 8 MIN READ•Apr 29, 2026
Hidden Partitioning: How Iceberg Eliminates Accidental Full Table Scans
Iceberg's hidden partitioning separates physical layout from user queries using transform functions.
Iceberg hidden partitioningpartition transformsaccidental full table scan - 8 MIN READ•Apr 29, 2026
Inside the Query Optimizer: How Engines Pick a Plan
Query optimizers transform SQL into execution plans using rule-based rewrites, cost-based search, and adaptive runtime adjustments.
query optimizercost-based optimizationquery planning - 8 MIN READ•Apr 29, 2026
Partition Evolution: Change Your Partitioning Without Rewriting Data
Iceberg lets you change partition schemes without rewriting data. Here is how partition evolution works internally and why Hive-style partitioning could.
Iceberg partition evolutionpartition spechidden partitioning - 8 MIN READ•Apr 29, 2026
B-Trees, LSM Trees, and the Indexing Tradeoff Spectrum
B-trees balance reads and writes for OLTP. LSM trees maximize write throughput. Bitmap indexes accelerate OLAP filtering. Here is when to use each.
database indexing strategiesB-tree vs LSM treebitmap index - 8 MIN READ•Apr 29, 2026
Performance and Apache Iceberg's Metadata
Iceberg's three-layer metadata tree eliminates directory listing and enables multi-level data skipping. Here is how scan planning actually works.
Apache Iceberg metadata performanceIceberg scan planningmanifest pruning - 8 MIN READ•Apr 29, 2026
How Databases Organize Data on Disk: Pages, Blocks, and File Formats
Databases structure data on disk as heap files, sorted files, or LSM trees, then wrap it in formats like Parquet with metadata that lets engines skip.
data file formatsParquet file formatdatabase storage internals - 8 MIN READ•Apr 29, 2026
The Metadata Structure of Modern Table Formats
Iceberg uses a metadata tree, Delta Lake uses a transaction log, Hudi uses a timeline. Here is exactly how each format organizes metadata and why it matters.
table format metadataApache Iceberg metadataDelta Lake transaction log - 8 MIN READ•Apr 29, 2026
Row vs. Column: How Storage Layout Shapes Everything
Row stores keep records together for fast transactions. Column stores keep field values together for fast analytics.
columnar vs row storagecolumn-oriented databaserow store performance