Episode Details

Fabric Dataflows Gen2: The Future of ETL in Microsoft Fabric

If you’ve been juggling separate Power Query scripts, Data Factory pipelines and custom SQL jobs just to keep your reports alive, Fabric Dataflows Gen2 is your chance to simplify everything. In this episode, I walk through why Microsoft rebuilt dataflows as a first‑class Fabric resource—so your transformations live once, feed Lakehouse, Warehouse and Power BI together, and stop breaking every time an upstream schema changes

We start with the pain of today’s fragmented ETL. You’ll recognize the pattern: the same logic duplicated across tools, parallel pipelines for different teams, and endless arguments about why numbers don’t match between reports that supposedly use the same source. I trace how this grew out of disconnected services and why it capped the scalability of even well‑run data teams.

Then we dive into the core architecture of Dataflows Gen2. You’ll learn how compute separation lets you scale transformation power without inflating storage, how managed staging kills off zombie temp tables, and how the new authoring and deployment model finally treats ETL like a governed, versioned asset instead of a side script. With concrete examples, we look at how a single dataflow can support both detailed analytics and summarized reporting without cloning logic in three places.

Finally, we explore what “seamless across Fabric” really means in practice. Instead of exporting and re‑importing data between tools, your dataflows feed Lakehouse, Warehouse and semantic models directly, so each part of Fabric sees the same, consistent transformation output. By the end, you’ll know when to move ETL into Dataflows Gen2, which legacy jobs to retire first, and how this shift can free up a significant chunk of the time your data team currently spends just keeping fragile pipelines alive.

WHAT YOU’LL LEARN

• Why fragmented ETL across Power Query, Data Factory and SQL has become a scaling bottleneck.
• How Fabric Dataflows Gen2 changes the architecture with compute separation and managed staging.
• How a single, governed dataflow can feed Lakehouse, Warehouse and Power BI without duplicated logic.
• When and how to start migrating existing ETL into Dataflows Gen2 for a more maintainable Fabric environment.

THE CORE INSIGHT

The core insight of this episode is that ETL is no longer a side process glued onto your analytics stack—it’s a first‑class Fabric asset. Once your transformations live in Dataflows Gen2 instead of being scattered across scripts and pipelines, you get cleaner governance, easier scaling and far fewer surprises every time your data model changes.

Listen Now