Projects
Selected work
Iberia validation frameworkShowHide
Problem
A regional rollout for Iberia could not rely on ad-hoc QA; downstream consumers needed repeatable proof that pipelines met bar before scale-up.
Built
Implemented a validation framework aligned to regional requirements: automated checks, clear pass/fail signals, and integration points so teams could not “skip” quality silently.
Impact
Reduced release risk for the geography and made quality a shared contract instead of a last-mile scramble.
Microsoft Graph API → S3 automationShowHide
Problem
Manual pulls and brittle scripts were wasting time and hiding failures; object-landed data needed a dependable path with operational visibility.
Built
Automated Graph-to-S3 ingestion with structured monitoring hooks, retries where appropriate, and failure surfaces that pointed to root cause classes.
Impact
Roughly ~90% improvement in bulk processing time for the targeted workloads and fewer surprise breaks in the landing zone.
Tidal-orchestrated ingestion — Germany LauertaxeShowHide
Problem
Germany Lauertaxe data had to land on schedule with orchestration that could survive dependencies and retries without corrupting downstream state.
Built
Orchestrated pipeline in Tidal with clear stage boundaries, dependency handling, and validation aligned to how finance consumed the feeds.
Impact
Reliable on-time loads and less firefighting during close periods when timing pressure peaks.
Unity Catalog migration & validation automationShowHide
Problem
UC migration without automation invites partial cutovers, orphaned assets, and endless reconciliation threads.
Built
Defined migration steps with automated validation: ownership mapping, table lineage sanity checks, and gates before promoting workloads.
Impact
Faster confidence in cutover readiness and less manual reconciliation; UC became the system of record, not a parallel shadow.
ConcurrentAppendException & pipeline latencyShowHide
Problem
Write contention and exception storms were inflating pipeline wait times and eroding trust in scheduled workloads.
Built
Profiled write patterns, adjusted concurrency assumptions, and aligned commit semantics with storage constraints so pipelines stopped tripping over each other.
Impact
About ~95% reduction in pipeline wait time for the affected flows; ops stopped treating “wait it out” as the default playbook.