Senior Data Engineer — Microsoft Fabric & Databricks

XCentium · Bangalore, India

Senior Data Engineer — Microsoft Fabric & Databricks

XCentium — India (Remote)

This is a Full-Time Remote position with a flexible work environment and great benefits! Apply today!

About Us

XCentium is an award-winning digital consultancy focused on DX, UX, Content, Commerce, CRM, Service and Cloud. We primarily work on Sitecore, Salesforce, Optimizely and Microsoft platforms.

XCentium is a leader in Salesforce Commerce, Salesforce B2B Commerce, OMS, CRM, Service, Marketing, Experience and MuleSoft and first company to achieve Salesforce Master Navigator for B2B Commerce globally.

We are a team of highly dedicated individuals, successful at delivering creative, new ideas that drive change for us and our clients. We build with heart, lead with excellence, and guide with expertise. Our team is growing, and we would love to hear from you!

Job Description

XCentium is seeking a Senior Data Engineer with deep hands-on expertise in Microsoft Fabric and/or Databricks to join our data practice. In this role, you will own the end-to-end implementation of complex data pipelines and lakehouse components, translate architectural designs into production-grade solutions, and provide technical guidance to junior engineers.

You will work closely with data architects and client stakeholders to build ingestion, transformation, data quality, and semantic modeling layers, and take the lead on performance optimization and engineering best practices across the medallion tiers. You'll play a key role in proof-of-concepts, accelerators, and enterprise-scale implementations, often serving as the senior technical presence on client engagements.

Core Qualifications

• 6+ years in data engineering or enterprise data platform roles
• Strong hands-on experience with Microsoft Fabric (OneLake, Lakehouse, Data Factory, Eventstream, KQL Database) and/or Databricks (Workflows, Unity Catalog, Delta Live Tables)
• Strong proficiency with Delta Lake format and its role across Lakehouse tiers
• Experience supporting Power BI / DirectLake connectivity or other BI tools consuming Gold-layer data

Medallion Architecture Implementation

• Ability to implement layer-specific data handling: raw ingestion at Bronze, cleansing/validation at Silver, and business-ready modeling at Gold
• Experience building star schemas in the Gold layer for BI consumption
• Hands-on experience implementing CDC (Change Data Capture) patterns within the medallion framework

Data Engineering & Integration

• Strong proficiency with PySpark / Spark Structured Streaming (within Fabric notebooks and/or Databricks notebooks)
• Experience building ingestion pipelines using Azure Data Factory, Dataflows Gen2, or Databricks Workflows/Autoloader
• Familiarity with source systems such as Salesforce, Oracle Service Cloud, Profisee (or similar MDM platforms)
• Understanding of real-time vs. batch ingestion path design (Eventstream, Spark Structured Streaming vs. scheduled pipelines)
• Experience with performance tuning, partitioning, and optimization of Spark/Delta workloads
• Exposure to Azure AI Foundry or AI/ML integration into the lakehouse is a plus

Governance & Quality

• Experience implementing and designing data quality frameworks and testing standards across medallion tiers
• Familiarity with Microsoft Purview or Unity Catalog for data catalog, lineage, and governance
• Experience with master data management (MDM) integration patterns

Leadership & Client-Facing Skills

• Experience mentoring or providing technical guidance to junior engineers
• Ability to work directly with client stakeholders and architects to clarify requirements and design decisions
• Comfortable operating as the senior technical resource on client engagements
• Ability to troubleshoot complex pipeline and performance issues independently

Nice to Have

• Microsoft Fabric, Azure Data Engineer, or Databricks certification (e.g., Databricks Certified Data Engineer Professional)
• Experience working across both Fabric and Databricks platforms
• Experience with CI/CD for data pipelines (Azure DevOps, GitHub Actions)
• Prior consulting experience across multiple client engagements