Tag: Data Engineering

2025-12-05 • Alex Merced

dremioframe & iceberg - Pythonic interfaces for Dremio and Apache Iceberg

Discover DremioFrame and IceFrame, two new Python libraries that simplify working with Dremio and Apache Iceberg. Learn ...

2025-11-29 • Alex Merced

Introducing dremioframe - A Pythonic DataFrame Interface for Dremio

Discover dremioframe, a new Python library that offers a DataFrame-like experience for interacting with Dremio's data la...

2025-11-12 • Alex Merced

Comprehensive Hands-on Walk Through of Dremio Cloud Next Gen (Hands-on with Free Trial)

Walkthrough with the new trial of the Dremio Cloud Platform...

2025-10-23 • Alex Merced

2025-2026 Guide to Learning about Apache Iceberg, Data Lakehouse & Agentic AI

A curated guide to mastering Apache Iceberg, data lakehouse architectures, and the emerging field of Agentic AI for data...

2025-10-21 • Alex Merced

An Exploration of the Commercial Iceberg Catalog Ecosystem

Dive into the world of commercial Iceberg catalogs and discover how they enhance data lakehouse architectures for modern...

2025-10-17 • Alex Merced

Building a Universal Lakehouse Catalog - Beyond Iceberg Tables

Exploring paths to a universal lakehouse catalog that supports multiple data formats and engines, building on Apache Ice...

2025-10-16 • Alex Merced

Intro to Apache Iceberg with Apache Polaris and Apache Spark

Learn how to leverage Apache Iceberg with Apache Polaris and Apache Spark to build scalable and efficient data lakehouse...

2025-10-14 • Alex Merced

The State of Apache Iceberg v4 - October 2025 Edition

What's Coming in Apache Iceberg v4: A Deep Dive into the Future of Open Table Formats...

2025-09-24 • Alex Merced

The Ultimate Guide to Open Table Formats - Iceberg, Delta Lake, Hudi, Paimon, and DuckLake

Understanding Iceberg, Delta Lake, Hudi, Paimon, and DuckLake...

2025-09-23 • Alex Merced

The 2025 & 2026 Ultimate Guide to the Data Lakehouse and the Data Lakehouse Ecosystem

What is the Data Lakehouse and the Data Lakehouse Ecosystem? This comprehensive guide covers everything you need to know...

2025-09-16 • Alex Merced

The Endgame — Building an Autonomous Optimization Pipeline for Apache Iceberg

Learn how to automate compaction, snapshot expiration, and layout optimization in Apache Iceberg using metadata-driven t...

2025-09-09 • Alex Merced

Managing Large-Scale Optimizations — Parallelism, Checkpointing, and Fail Recovery

Learn how to scale Apache Iceberg table optimizations across large datasets using parallelism, checkpointing, and fail r...

2025-09-05 • Alex Merced

Unlocking the Power of Agentic AI with Apache Iceberg and Dremio

Unlocking the Power of Agentic AI with Apache Iceberg and Dremio...

2025-09-02 • Alex Merced

Hidden Pitfalls — Compaction and Partition Evolution in Apache Iceberg

Partition evolution in Apache Iceberg is a powerful feature, but if not managed carefully, it can introduce fragmentatio...

2025-08-19 • Alex Merced

Designing the Ideal Cadence for Compaction and Snapshot Expiration

Learn how to design an effective schedule for compaction and snapshot expiration in Apache Iceberg to balance cost, perf...

2025-07-03 • Alex Merced

How to Discover or Organize Lakehouse & Apache Iceberg Meetups

Guide on How to Be Part of the Lakehouse Community...

2025-05-02 • Alex Merced

Introduction to Data Engineering Concepts | What is Data Engineering?

Introduction to the terms in data engineering...

2025-05-02 • Alex Merced

Introduction to Data Engineering Concepts | Understanding Data Sources and Ingestion

Introduction to the terms in data engineering...

2025-05-02 • Alex Merced

Introduction to Data Engineering Concepts | ETL vs ELT – Understanding Data Pipelines

Introduction to the terms in data engineering...

2025-05-02 • Alex Merced

Introduction to Data Engineering Concepts | Batch Processing Fundamentals

Introduction to the terms in data engineering...