Real-Time Data Archives - Ronak Makwana

June 12, 2026

Featured image: Handling late-arriving events, four numbered steps: measure lateness, size the watermark, pick the output mode, and correct what is dropped with a MERGE

Handling Late-Arriving Events in Real-Time Data Pipelines

Why events arrive late, how Spark watermarks and Stream Analytics tolerances decide what counts, and six strategies to measure lateness, size watermarks and correct late data instead of losing it.

June 10, 2026

Featured image: Spark Structured Streaming in practice, a PySpark code card that applies a 10-minute watermark and a five-minute window, then writes to Delta with append output mode, a checkpoint and an availableNow trigger

Apache Spark Structured Streaming: A Practical Guide

A hands-on guide to Spark Structured Streaming: the micro-batch model, sources and sinks, output modes, triggers, watermarks, foreachBatch upserts and monitoring, with output from a local run.

June 4, 2026

Featured image: Event Hubs partitions and consumer groups, a large Azure Event Hubs icon with a Kafka protocol badge

Azure Event Hubs Explained: Architecture, Partitions, and Consumer Groups

How Azure Event Hubs works under the hood: partitions and partition keys, sizing the partition count, consumer groups, checkpoints and tier limits, with Bicep and Python examples.

June 2, 2026

Featured image: Real-time data architectures on Azure, with Azure Event Hubs at the centre of an orbit of Databricks, Cosmos DB, Eventhouse, Stream Analytics, IoT Hub and Event Grid icons

Designing Real-Time Data Processing Architectures on Azure

How to design a real-time data platform on Azure: latency tiers, Event Hubs ingestion, choosing Stream Analytics, Databricks, Fabric or Functions, and the decisions on event time, partitions and replay.