Posts
All the articles I've posted.
- 15 MIN READ•Jul 6, 2026
Decoupled Catalogs vs. Managed Tables: Architectural Freedom in the Age of Table Format Convergence
Open table formats have changed buyer expectations. A few years ago, the question was whether an organization should put more analytical data into...
catalogstable formatsarchitecture - 26 MIN READ•Jul 6, 2026
Designing Your Own AI Harness: A Deep Dive Into the Architecture of Agent Loops, Tools, Context, and Control
A deep dive into custom AI harness architecture: model layers, tool design, context management, permissions, control budgets, persistence, orchestration, and evaluation systems.
AI agentsagent architectureMCP - 26 MIN READ•Jul 6, 2026
Deterministic Data Engineering With AI Harnesses: Using Claude Code, Codex, Antigravity, and OpenCode for Data Work You Can Actually Trust
How to use AI agent harnesses for data engineering without losing determinism, reproducibility, and trust in your data pipelines and analytics.
data engineeringAI agentsClaude Code - 17 MIN READ•Jul 6, 2026
Preparing Your Data Lakehouse for the EU AI Act: Auditable Lineage and Data Provenance
The EU AI Act changes the conversation around AI architecture because it makes trust operational. It is not enough to say that an AI system is usef...
eu ai actcompliancedata lakehouse - 28 MIN READ•Jul 6, 2026
Federation and the Lakehouse: Two Roads to Unified Data Access, and How to Know Which One to Take
Every data strategy document written this decade contains some version of the same sentence: we need a single place to access all our data. The sen...
data federationdata lakehouseunified access - 27 MIN READ•Jul 6, 2026
A Deep Dive Into File Compression: How Data Gets Smaller, Why Codecs Differ, and What to Actually Use in the Lakehouse
Somewhere in your data platform right now, a single configuration property is quietly deciding a meaningful percentage of your storage bill, your q...
file compressioncodecsparquet - 27 MIN READ•Jul 6, 2026
The File Format Renaissance: Parquet, Lance, Vortex, Nimble, BtrBlocks, and the New Physics of Columnar Storage
For a decade, the file format layer was the most settled real estate in data. Apache Parquet held the analytical world, ORC held the Hive legacy es...
parquetlancevortex - 16 MIN READ•Jul 6, 2026
Enforcing Fine-Grained Security at Machine Speed: Dynamic Access Control for High-Frequency AI Agents
AI agents change the security model for analytics. A human user may run a handful of queries, pause, interpret the answer, and ask a follow-up. An...
fine-grained securityai agentsaccess control - 16 MIN READ•Jul 6, 2026
Implementing Positional Deletes in Iceberg v3: Streamlining Merge-on-Read for Fast-Inbound Event Lakes
Event data has a way of humbling neat architecture diagrams. It arrives late. It arrives twice. It arrives with incorrect attributes. It needs priv...
iceberg v3positional deletesevent lakes - 16 MIN READ•Jul 6, 2026
Mapping the Variant Type in Iceberg v3: Standardizing Semi-Structured AI JSON Payloads
AI applications are messy data producers. They create prompts, completions, tool calls, retrieval traces, ranking signals, evaluation scores, safet...
iceberg v3variant typejson - 15 MIN READ•Jul 6, 2026
Designing Idempotent Pipelines in the Agentic Lakehouse: Eliminating Double-Write Anomalies
Agents retry. Networks fail. Jobs time out after doing some work. APIs return ambiguous responses. Schedulers run the same workflow twice. A human...
idempotent pipelinesagentic lakehousedouble-write - 27 MIN READ•Jul 6, 2026
File Encryption for the Lakehouse: The Terminology, the Machinery, and the Hard Problem of Interoperable Encrypted Tables
For years, the open lakehouse had an honest gap that practitioners whispered about and slide decks skipped: encryption. Not the checkbox kind, ever...
encryptiondata lakehousesecurity