Tag: Apache Iceberg

2026-09-02 • Alex Merced

dbt on Iceberg: Incremental Models on Open Tables

How dbt incremental materializations map to Iceberg operations, and the configuration, predicates, and maintenance that ...

2026-09-02 • Alex Merced

Disaster Recovery for Iceberg Tables: Replication, Backup, and Restore

Disaster recovery for Iceberg across four tiers: snapshots, object versioning, catalog backup, and cross-region replicat...

2026-09-02 • Alex Merced

Deleting User Data From an Immutable Lakehouse: GDPR Hard Deletes on Iceberg

How to turn a logical delete on immutable Iceberg into a physical erasure across snapshots, versions, replicas, and down...

2026-09-02 • Alex Merced

Geospatial Data in Apache Iceberg: Geometry, Geography, and GeoParquet

How Iceberg v3 geometry and geography types, bounding boxes, and native Parquet types give spatial data first-class stan...

2026-09-02 • Alex Merced

Default Column Values and Field IDs: How Iceberg Schema Evolution Works at the Spec Level

How field IDs and initial and write defaults let Iceberg change schemas on large tables without rewriting data, at the s...

2026-09-02 • Alex Merced

The Iceberg Table Properties That Actually Matter

The Iceberg table properties that decide file count, pruning, write amplification, retention, and metadata growth, by wo...

2026-09-02 • Alex Merced

Inside the Puffin File Format

The Puffin file format inside out, byte by byte, covering Theta sketches for distinct values and deletion vectors....

2026-09-02 • Alex Merced

The Lakehouse Ingestion Tool Landscape: Fivetran, Airbyte, dlt, and CDC vs Batch

How Fivetran, Airbyte, dlt, and CDC and streaming tools land well-behaved Apache Iceberg tables, and how to choose and m...

2026-09-02 • Alex Merced

Local Iceberg Development Environments: Docker, MinIO, and In-Memory Catalogs for CI

Local Iceberg development environments: in-process catalogs, a Docker Compose stack with MinIO, and CI configurations th...

2026-09-02 • Alex Merced

Moving Iceberg Tables Between Catalogs Without Rewriting Data

Why moving Iceberg tables between catalogs is a pointer copy, and the protocol that makes a cutover safe for one table o...

2026-09-02 • Alex Merced

Logs, Traces, and Metrics as Tables: Building an OpenTelemetry Data Lake on Iceberg

Building an OpenTelemetry data lake on Iceberg: schemas for spans, logs, and metrics, ingestion, query patterns, and ret...

2026-09-02 • Alex Merced

Schema Registries and Event Schemas: Avro, Protobuf, and JSON Schema on the Way Into the Lakehouse

How Avro, Protobuf, and JSON Schema evolve through a registry, and how that maps to the schema evolution rules of Iceber...

2026-09-02 • Alex Merced

Storage-Partitioned Joins and the Bucket Transform

How the spec-defined bucket transform lets engines skip the shuffle in joins, and how to set it up and keep it engaged i...

2026-08-25 • Alex Merced

Agent-Driven Storage Tiering for Apache Iceberg: Moving Cold Data Without Breaking Queries

A background agent can move cold Iceberg partitions to cheaper tiers without breaking live queries. Heatmaps, path-safe ...

2026-08-25 • Alex Merced

DataFusion Comet 1.0 and What Native Rust Scans Change for Spark on Iceberg

DataFusion Comet 1.0 replaces Spark Iceberg scans with native Rust. What speeds up, what still falls back to the JVM, an...

2026-08-25 • Alex Merced

High-Throughput Branch Merging: Automating Concurrency and Conflict Resolution in Multi-Branch Iceberg Pipelines

High-throughput Iceberg branch merges need conflict detection and automation. How to reconcile concurrent writes without...

2026-08-25 • Alex Merced

Multi-Cloud REST Catalog Topologies: Running Apache Polaris Across AWS, Azure, and GCP

Polaris can catalog Iceberg tables across AWS, Azure, and GCP. Four topologies, credential vending, and the tradeoffs of...

2026-08-25 • Alex Merced

Parquet-Only Manifests in Iceberg v4: Why the Metadata Layer Is Going Columnar

Iceberg v4 is moving manifests from Avro to Parquet so planners can read only the stats they need. Why the metadata laye...

2026-08-25 • Alex Merced

Semantic Layer Federation: One Logical Model Over Data on Three Clouds

One logical model over Iceberg and databases on three clouds. Pushdown, egress, Reflections, and where semantic federati...

2026-08-25 • Alex Merced

Serverless Iceberg Ingestion with PyIceberg and DuckDB: Micro-Batches Without a Spark Cluster

Land small Iceberg micro-batches with PyIceberg and DuckDB in a serverless function. Commits, concurrency, and why Spark...

Events & Community

AI newsletter, Thursdays
Apache lakehouse newsletter, Fridays
Subscribe on Substack
Agentic Lakehouse Events
Data Lakehouse Hub Events
Data Lakehouse Hub Slack
Data Events Slack
Data & Tech Slack
r/datalakehouseandai
Data Lakehouse Hub on LinkedIn
Alex Merced Tech
Alex Merced Data & AI

The views, thoughts, and opinions expressed on this site belong solely to Alex Merced and do not represent the views of any organization or employer.