Episode Details

If you've ever wondered why your data suddenly disappears from a report, or who exactly changed the source file feeding your monthly dashboard, you're not alone. Most teams are flying blind when it comes to seeing the full journey of their data.Today, we're going to trace that journey inside Microsoft Fabric — from ingestion, through transformation, into analytics — and uncover how lineage, permissions, and the catalog work together to keep you in control. By the end, you'll see every hop your data makes, and exactly who can touch it.Seeing the Invisible: The Path Data Actually TakesMost people picture data traveling like a straight road: it leaves the source, passes through a few hands, and ends up neatly in a report. In reality, it’s closer to navigating an old building that’s been renovated a dozen times. You’ve got hallways that suddenly lead to locked doors, side passages you didn’t even know existed, and shortcuts that bypass major rooms entirely. That’s the challenge inside any modern analytics platform—your data’s path isn’t just a single pipeline, it’s a web of steps, connections, and transformations. Microsoft Fabric’s Lakehouse model gives the impression of a single, unified home for your data. And it is unified—but under the hood, it’s a mix of specialized services working together. There’s a storage layer, an analytics layer, orchestration tools, and processing engines. They talk to each other constantly, passing data back and forth. Without the right tools to record those interactions, what you actually have is a maze with no map. You might know how records entered the system and which report they eventually landed in, but the middle remains a black box. When that black box gets in the way, it’s usually during troubleshooting. Maybe a number is wrong in last month’s sales report. You check the report logic, it looks fine. The dataset it’s built on seems fine too. But somewhere upstream, a transformation changed the values, and no one documented it. That invisible hop—where the number stopped being accurate—becomes the needle in the haystack. And the longer a platform has been in use, the more invisible hops it tends to collect. This is where Fabric’s approach to lineage takes the maze and lays down a breadcrumb trail. Take a simple example: data comes in through Data Factory. The moment the pipeline runs, lineage capture starts—without you having to configure anything special. Fabric logs not just the target table in the Lakehouse but also every source dataset, transformation step, and subsequent table or view created from it. It doesn’t matter if those downstream objects live in the same workspace or feed into another Fabric service—those links get recorded automatically in the background. In practice, that means if you open the lineage view for a dataset, you’re not just seeing what it feeds—you’re seeing everything feeding it, all the way back to the ingestion point. It’s like tracking a shipment and seeing its path from the supplier’s warehouse, through every distribution center, truck, and sorting facility, instead of just getting a “delivered” notification. You get visibility over the entire chain, not just the start and finish. Now, there’s a big difference between choosing to document lineage and having the system do it for you. With user-driven documentation, it’s only as current as the last time someone updated it—assuming they remembered to update it at all. With Fabric, this happens as a side effect of using the platform. The metadata is generated as you create, move, and transform data, so it’s both current and accurate. This reduces the human factor almost entirely, which is the only way lineage maps ever stay trustworthy in a large, active environment. It’s worth noting that what Fabric stores isn’t just a static diagram. That automatically generated metadata becomes the basis for other controls—controls that don’t just visualize the flow but actually enforce governance. It’s the foundation for con