Skip to content
All workCase study · Healthcare services · Fabric platform

One Fabric pipeline for every source.

Metadata-Driven Lakehouse: a medallion lakehouse where adding a source is a row of configuration, not a new pipeline, and drifting data fails loudly before it reaches a report.

15+ hrs
a week of manual reporting retired, worth $35K+ a year
6+ sources
loaded by one pipeline, from APIs to AWS S3
50+ tables
onboarded with a config row each
01The problem

Reporting ran on exports. Each source had its own extract, its own schedule and its own way of breaking, and someone spent hours every week pulling files together by hand before any report could refresh.

02What we built
  1. 1One metadata-driven pipeline. Every source and table is a row of configuration, so a new table onboards without new pipeline code.
  2. 2Bronze, silver and gold layers. Raw data lands as it arrives, gets cleaned and conformed, then shaped into business-ready tables.
  3. 3Schema contracts. Silver checks each source against its expected shape and fails the run when something drifts, instead of passing bad data along.
  4. 4Credentials kept safe. Connection secrets live in Azure Key Vault, never in notebooks or pipeline settings.
03The result

Millions of rows land on schedule from every source, a new table onboards with a config row, and more than fifteen hours a week of manual reporting is gone.

04 · The shape of it

Every source in, one model out.

APIsS3SFTPCRMGoldSilverBronze6+ sources · 50+ tables · 1 pipeline