Senior Data Engineer-(Immediate Joiners with LWD (Sept Month) Only Apply!

Spectral Consultants · Bangalore district, India

Spectral Consultants is currently hiring for a Senior Data Engineer – Azure Databricks for a leading Professional Services & Accounting organization.

This is an opportunity to work on modern Lakehouse platforms and next-generation AI & Analytics initiatives, building scalable, governed, and AI-ready data solutions using Azure Databricks and the broader Azure ecosystem.

Location: Bangalore

Experience: 5–9 Years

Work Mode: Hybrid

About the Role:-

As a Senior Data Engineer, you will design and develop scalable ETL/ELT pipelines and curated Lakehouse data models using Azure Databricks, PySpark, SQL, Delta Lake, Unity Catalog, and ADLS Gen2. You will collaborate with Analytics, AI/ML, Product, and Business teams to deliver reliable, governed, and AI-ready datasets.

Key Responsibilities:-

• Design and maintain scalable ETL/ELT pipelines using Azure Databricks.
• Build data pipelines using Medallion Architecture (Bronze, Silver, Gold).
• Develop and optimize Delta Lake data models for Analytics and AI/ML use cases.
• Implement data ingestion using Spark Structured Streaming, Auto Loader, connectors, and Delta Sharing.
• Manage governed data using Unity Catalog and integrate with ADLS Gen2.
• Implement data quality, lineage, audit, reconciliation, and governance frameworks.
• Orchestrate workflows using Databricks Jobs/Workflows and DLT/Lakeflow.
• Implement CI/CD using Git and Databricks Asset Bundles.
• Troubleshoot production issues and collaborate with cross-functional teams to deliver reliable, consumption-ready datasets.

Must-Have Skills:-

• 5–8 years of hands-on experience in Data Engineering/Data Integration/Analytics Engineering.
• Strong expertise in Azure Databricks, PySpark/Spark, and SQL.
• Hands-on experience with Delta Lake and Medallion Architecture.
• Strong experience with Unity Catalog and ADLS Gen2.
• Experience building scalable ETL/ELT pipelines and Databricks Jobs/Workflows.
• Strong understanding of Data Governance, Data Quality, Lineage, Access Control, and PII handling.
• Experience with Git, CI/CD, and modern data engineering practices.
• Strong analytical, problem-solving, and stakeholder management skills.

Important Eligibility: Only Immediate Joiners or candidates whose Last Working Day (LWD) falls in September are eligible to apply.