Sr Azure Data Engineer - Hyderabad

Job Title: Senior Data Engineer Employment Type: Full Time, Permanent Position Location: Hyderabad Qualifications: BE/B.Tech/MCA Degree in Computer Science, Engineering, or similar relevant field Total Experience: 8 to 14 years Working Model: Work from office Data Pipeline Development • Design and implement end-to-end batch and streaming data pipelines using PySpark and Apache Spark. • Build and manage data workflows on Databricks, including Delta Lake architecture and medallion (Bronze/Silver/Gold) patterns. • Develop and maintain ETL/ELT processes to ingest, transform, and load data from diverse sources. • Optimize Spark jobs for performance, cost-efficiency, and reliability at scale. Cloud & Azure Services • Manage data solutions using Azure services including Azure Data Lake Storage (ADLS Gen2), Azure Event Hubs, and Azure Key Vault. • Ensure data security, access control, and compliance within Azure environments. • Monitor and troubleshoot cloud infrastructure, pipelines, and data flows. Data Modeling & SQL • Design and implement dimensional and normalized data models for analytical and operational use cases. • Write complex SQL queries for data transformation, validation, and reporting. • Work with structured, semi-structured, and unstructured data across various storage formats (Parquet, Delta, JSON, CSV). Required Skill • 6 years of hands-on experience in data engineering roles. • Strong expertise in Apache Spark and PySpark for large-scale data processing. • Deep experience with Databricks platform — workspace setup, clusters, jobs, Delta Live Tables. • Strong experience in building Azure Data Factory Pipeline. • Proficiency with Azure cloud services for data engineering workloads. • Advanced SQL skills including window functions, CTEs, and query optimization. • Solid Python programming skills for data engineering, scripting, and automation. • Experience with version control (Git) and CI/CD pipelines for data code deployments. • Familiarity with data governance, data lineage, and metadata management. Good to have • Azure Synapse & Azure EventHub are good to have.