Lead Azure Data Engineer_ Exp: 8+ Years

Atyeti Inc · India

Job Overview : We are seeking an experienced Senior Data Engineer to design, build, optimize, and support enterprise-scale data platforms on Azure. The ideal candidate will possess strong expertise in data ingestion, transformation, orchestration, and data warehousing using technologies such as Azure Databricks, Azure Data Factory (ADF), Delta Lake, PySpark, and Azure SQL. The candidate will work closely with business stakeholders, data architects, analysts, and application development teams to deliver scalable, high-performance, and reliable data solutions.

Responsibilities:

• Design, develop, and maintain scalable data ingestion, transformation, and delivery pipelines.

• Build ETL/ELT solutions using Azure Databricks, PySpark, and Azure Data Factory (ADF).

• Develop and optimize Databricks notebooks, workflows, and scheduled jobs.

• Implement and maintain Delta Lake-based Bronze, Silver, and Gold data layers.

• Integrate data from databases, APIs, files, and cloud storage platforms.

• Ensure data quality, performance, reliability, and operational support for production pipelines.

• Collaborate with architects, analysts, and business stakeholders to deliver data solutions.

• Bachelor's degree in Computer Science, Information Technology, or a related field.

•7+ years of experience in Data Engineering and Data Warehousing.

• 4+ years of hands-on experience with Azure Databricks and PySpark.

• Experience working in Agile/Scrum environments.

• Excellent communication and problem-solving skills.

Skills: Azure Databricks (PySpark, Spark SQL, Delta Lake), Azure Data Factory (ADF) ,Databricks Jobs & Workflows, Azure Data Lake, Storage (ADLS Gen2), Azure SQL Database /SQL Server, ETL/ELT Design & Development, Data Ingestion &Data Transformation, Data Modeling & Data Warehousing, Performance Tuning & Optimization, CI/CD, Git, Data Quality, Monitoring, and Production Support