Tag:data engineering
Articles tagged "data engineering", page 12.
- 4 MIN READ•Aug 30, 2024
Using the alexmerced/datanotebook Docker Image
How to use the alexmerced/datanotebook Docker image for quick Python data work: bundled libraries, saving your work, and connecting to Spark and MinIO.
data lakehousedata engineeringdata analytics - 5 MIN READ•Aug 29, 2024
Understanding Apache Iceberg Delete Files
Continuing the Understand Apache Iceberg series, this article delves into the Manifest, a critical component of Apache Iceberg's architecture.
data lakehousedata engineeringApache Iceberg - 8 MIN READ•Aug 27, 2024
Understanding the Apache Iceberg Manifest
Continuing the Understand Apache Iceberg series, this article delves into the Manifest, a critical component of Apache Iceberg's architecture.
data lakehousedata engineeringApache Iceberg - 7 MIN READ•Aug 25, 2024
Understanding the Apache Iceberg Manifest List (Snapshot)
Continuing the Understand Apache Iceberg series, this article delves into the Manifest List, a critical component of Apache Iceberg's architecture.
data lakehousedata engineeringApache Iceberg - 8 MIN READ•Aug 21, 2024
Understanding Apache Iceberg's Metadata.json
A field-by-field walkthrough of Iceberg's metadata.json, covering format version, schemas, partition specs, and snapshots, and how it enables time travel.
data lakehousedata engineeringapache iceberg - 3 MIN READ•Aug 18, 2024
What Apache Iceberg REST Catalog is and isn't
What the Iceberg REST catalog spec covers, a uniform API for table operations, and what it does not: it is not a catalog and does not standardize security.
data lakehousedata engineeringcatalog - 4 MIN READ•Aug 15, 2024
ACID Guarantees and Apache Iceberg - Turning Any Storage into a Data Warehouse
What atomicity, consistency, isolation, and durability mean, how warehouses provide them, and how Apache Iceberg brings ACID transactions to data lakes.
data lakehousedata engineering - 10 MIN READ•Aug 5, 2024
Data Lakehouse 101 - The Who, What and Why of Data Lakehouses
What a data lakehouse is, why it costs less than copying data into warehouses, a three-step migration path with Dremio, and seven signs it is time to switch.
data lakehousedata engineering - 6 MIN READ•Jul 26, 2024
Apache Iceberg Reliability
How Apache Iceberg avoids the consistency problems of Hive tables on S3 with a persistent metadata tree, atomic commits, and serializable isolation.
data lakehousedata engineeringApache Iceberg - 9 MIN READ•Jul 20, 2024
Understanding the Polaris Iceberg Catalog and Its Architecture
An overview of the Polaris Iceberg catalog: its entities, internal versus external catalogs, and how principals and roles control access to your tables.
data lakehousedata engineeringapache iceberg - 3 MIN READ•Jul 20, 2024
Upcoming Data Talks from Alex Merced (And how to follow)
Come see me talk live at these events
data lakehousedata engineering - 11 MIN READ•Jul 12, 2024
Databases Deconstructed - The Value of Data Lakehouses and Table Formats
How a lakehouse splits a database into layers (Parquet storage, an Iceberg table format, a catalog, and query engines) and why that modularity matters.
data lakehousedata engineering