Tag:data lakehouse
Articles tagged "data lakehouse", page 8.
- 14 MIN READ•Oct 9, 2024
What is Three-Tier Data (Bronze, Silver, Gold) and How Dremio Simplifies It
What the bronze, silver, and gold data tiers are, the copy and sync costs of moving data between them, and how Dremio views and Reflections cut that work.
data lakehousedata engineering - 7 MIN READ•Oct 7, 2024
A Brief Guide to the Governance of Apache Iceberg Tables
How access control for Apache Iceberg tables works at the file, engine, and catalog level, and how Project Nessie and Apache Polaris governance compare.
data lakehousedata engineeringapache iceberg - 17 MIN READ•Oct 7, 2024
Exploring Data Operations with PySpark, Pandas, DuckDB, Polars, and DataFusion in a Python Notebook
Hands-on examples of loading and querying data with PySpark, Pandas, DuckDB, Polars, and DataFusion in one Docker notebook, plus querying Dremio.
data lakehousedata engineeringPython - 8 MIN READ•Oct 5, 2024
Ultimate Directory of Apache Iceberg Resources
A directory of Apache Iceberg learning material by topic: architecture, ingestion, migration, streaming, partitioning, maintenance, and catalogs.
data lakehousedata engineeringapache iceberg - 14 MIN READ•Oct 4, 2024
Change Data Capture (CDC) when there is no CDC
How to sync changing data when a source has no native CDC: timestamp and version columns, SQL to find changes, MERGE, and scheduled incremental batches.
data lakehousedata engineeringstreaming - 4 MIN READ•Sep 25, 2024
Virtualization + Lakehouse + Mesh = Data At Scale
How virtualization, a lakehouse, and data mesh ideas combine into one point of access with data curated by many domain teams, and where Dremio fits.
data lakehousedata engineering - 12 MIN READ•Sep 10, 2024
Why Data Analysts, Engineers, Architects and Scientists Should Care about Dremio and Apache Iceberg
How Dremio and Apache Iceberg address the daily problems of data engineers, architects, analysts, and data scientists, from data silos to governance.
data lakehousedata engineering - 43 MIN READ•Sep 10, 2024
Hands-on with Apache Iceberg on Your Laptop - Deep Dive with Apache Spark, Nessie, Minio, Dremio, Polars and Seaborn
Build an Iceberg lakehouse on your laptop: ingest with Spark into MinIO, track tables in Nessie, query in Dremio, then analyze with Polars and Seaborn.
data lakehousedata engineering - 7 MIN READ•Sep 1, 2024
5 Trends in the Data Lakehouse Space
Five lakehouse trends: storage vendors adding warehouse features, table format convergence, managed Iceberg upkeep, catalog wars, and unified platforms.
data lakehousedata engineering - 4 MIN READ•Aug 30, 2024
Using the alexmerced/datanotebook Docker Image
How to use the alexmerced/datanotebook Docker image for quick Python data work: bundled libraries, saving your work, and connecting to Spark and MinIO.
data lakehousedata engineeringdata analytics - 5 MIN READ•Aug 29, 2024
Understanding Apache Iceberg Delete Files
Continuing the Understand Apache Iceberg series, this article delves into the Manifest, a critical component of Apache Iceberg's architecture.
data lakehousedata engineeringApache Iceberg - 8 MIN READ•Aug 27, 2024
Understanding the Apache Iceberg Manifest
Continuing the Understand Apache Iceberg series, this article delves into the Manifest, a critical component of Apache Iceberg's architecture.
data lakehousedata engineeringApache Iceberg