Azure Data Engineering Archives

February 27, 2026

Featured image: 10 common Azure data engineering mistakes, a grid of ten numbered tiles, each with a product icon and a short name such as tiny files, keys and SAS tokens, unsafe reruns and portal-only monitoring

10 Common Azure Data Engineering Mistakes and How to Avoid Them

Ten mistakes that make Azure data platforms slow, insecure or unreliable, from tiny files and leaked keys to failure paths that report success, with the fix for each.

February 24, 2026

Featured image: Securing data pipelines with managed identity, with a large Azure Key Vault icon and a managed identity icon on a teal background

Securing Azure Data Pipelines with Managed Identity and Key Vault

Remove secrets from Azure data pipelines: a user-assigned managed identity with least-privilege roles in Bicep, Entra-only SQL access, Key Vault for what's left, Databricks storage credentials, and audit logging.

February 9, 2026

Featured image: ADLS Gen2 best practices for the enterprise, a white card with the title beside icon chips for containers and layout, private endpoints, Entra ID with RBAC and ACLs, and soft delete and lifecycle

Azure Data Lake Storage Gen2: Best Practices for Enterprise Data Platforms

Enterprise practices for Azure Data Lake Storage Gen2: account topology, redundancy, directory layout, file sizing, RBAC versus ACLs, private endpoints, HNS feature support and lifecycle tiering.

February 6, 2026

Featured image: Understanding the medallion architecture, with stacked landing, bronze, silver and gold slabs labelled with ADLS Gen2, Delta Lake, Databricks and Power BI

Understanding the Medallion Architecture: Bronze, Silver, and Gold Layers

What belongs in the bronze, silver and gold layers of a lakehouse, with a worked PySpark and Delta example, a placement guide for common tasks, and the trade-offs behind how strictly to follow the pattern.

February 2, 2026

Featured image: Building a modern data platform on Azure, with a left-to-right pipeline of icons for source systems, Azure Data Factory, ADLS Gen2, Azure Databricks and Power BI on a blueprint grid

Building a Modern Data Engineering Platform on Microsoft Azure

A guided build of a lakehouse-style data platform on Azure: ADLS Gen2, Data Factory, Databricks with Unity Catalog, Key Vault and Azure Monitor, deployed with Bicep and wired together with managed identities.