Tag:data lakehouse
Articles tagged "data lakehouse", page 7.
- 26 MIN READ•Oct 31, 2024
Hands-on with Apache Iceberg & Dremio on Your Laptop within 10 Minutes
Run Dremio, Nessie, and MinIO with Docker Compose in about 10 minutes, create Iceberg tables, run transactions, and inspect the files and metadata.
data lakehousedata engineeringapache iceberg - 8 MIN READ•Oct 30, 2024
Data Modeling - Entities and Events
The difference between modeling data as entities and as events, when each approach fits, key design choices, and how to combine both for full analysis.
data lakehousedata engineeringdata modeling - 4 MIN READ•Oct 21, 2024
All About Parquet Part 01 - An Introduction
An introduction to Apache Parquet: how it differs from row formats like CSV and JSON, key features such as compression and schema evolution, and when to use it.
data lakehousedata engineeringapache parquet - 5 MIN READ•Oct 21, 2024
All About Parquet Part 02 - Parquet's Columnar Storage Model
How Parquet's columnar storage model works and why it speeds analytics through faster scans, better compression, and quick aggregation, plus when to avoid it.
data lakehousedata engineeringapache parquet - 6 MIN READ•Oct 21, 2024
All About Parquet Part 03 - Parquet File Structure | Pages, Row Groups, and Columns
Inside a Parquet file: how row groups, column chunks, and pages are laid out, what footer metadata stores, and how their sizes affect query performance.
data lakehousedata engineeringapache parquet - 6 MIN READ•Oct 21, 2024
All About Parquet Part 04 - Schema Evolution in Parquet
How Parquet handles schema evolution when you add, remove, retype, rename, or reorder columns, how it compares to other formats, and best practices.
data lakehousedata engineeringapache parquet - 6 MIN READ•Oct 21, 2024
All About Parquet Part 05 - Compression Techniques in Parquet
The compression codecs Parquet supports (Snappy, Gzip, Brotli, Zstandard, LZO), the speed and size tradeoffs of each, and how they pair with encoding.
data lakehousedata engineeringapache parquet - 7 MIN READ•Oct 21, 2024
All About Parquet Part 06 - Encoding in Parquet | Optimizing for Storage
How Parquet encodings work and when to use them: dictionary, run-length, bit-packing, delta, and plain, and how encoding pairs with compression.
data lakehousedata engineeringapache parquet - 7 MIN READ•Oct 21, 2024
All About Parquet Part 09 - Parquet in Data Lake Architectures
Why Parquet is the default file format for data lakes, how it underpins Iceberg, Delta Lake, and Hudi, and best practices for partitioning and pushdown.
data lakehousedata engineeringapache parquet - 6 MIN READ•Oct 18, 2024
A Guide to dbt Macros - Purpose, Benefits, and Usage
What dbt macros are and how they cut repeated SQL: defining them, calling them in models, generating SQL with variables, and keeping them maintainable.
data lakehousedata engineeringdbt - 6 MIN READ•Oct 16, 2024
Data Lakehouse Roundup 1 - News and Insights on the Lakehouse
What's Going on in the Data Lakehouse Space
data lakehousedata engineeringstreaming - 17 MIN READ•Oct 15, 2024
Getting Started with Data Analytics Using PyArrow in Python
A practical PyArrow guide: core objects like Tables and RecordBatches, reading and writing Parquet, running computations, and using Arrow Flight.
data lakehousedata engineeringPython