Watermarks

June 12, 2026

Featured image: Handling late-arriving events, four numbered steps: measure lateness, size the watermark, pick the output mode, and correct what is dropped with a MERGE

Handling Late-Arriving Events in Real-Time Data Pipelines

Why events arrive late, how Spark watermarks and Stream Analytics tolerances decide what counts, and six strategies to measure lateness, size watermarks and correct late data instead of losing it.

June 10, 2026

Featured image: Spark Structured Streaming in practice, a PySpark code card that applies a 10-minute watermark and a five-minute window, then writes to Delta with append output mode, a checkpoint and an availableNow trigger

Apache Spark Structured Streaming: A Practical Guide

A hands-on guide to Spark Structured Streaming: the micro-batch model, sources and sinks, output modes, triggers, watermarks, foreachBatch upserts and monitoring, with output from a local run.