Senior Azure Databricks Engineer

Job Title: Senior Azure Databricks Engineer
Please read the following job description thoroughly to ensure you are the right fit for this role before applying.
Duration: 12 monthsSchedule: 5-40 scheduleLocation: Glendale, AZThis opportunity is focused on designing and implementing scalable enterprise data platform solutions in a modern Azure environment. The role will support enterprise data engineering frameworks across batch, streaming, ELT/ETL, and real-time analytics workloads, with strong emphasis on Azure Databricks, Delta Lake, Unity Catalog, and Azure Data Lake Storage Gen2. This position offers involvement in platform architecture, governance, security, performance optimisation, and migration to a modern Lakehouse architecture.Required ExperienceAt least 8-10 years of experience designing and implementing enterprise data platformsAt least 5 years of hands-on experience with Azure Databricks in enterprise environmentsExperience architecting modern Lakehouse solutions using Azure Databricks and Delta LakeExperience administering Azure Databricks workspaces, clusters, Unity Catalog, compute policies, job orchestration, and workspace securityExperience implementing enterprise data governance including metadata management, data lineage, and fine-grained access controlsExperience designing enterprise security architectures including RBAC, managed identities, private endpoints, Key Vault integration, encryption, and network securityExperience implementing enterprise data quality frameworks, monitoring, observability, and operational dashboardsExperience supporting migration of legacy enterprise data warehouses, ETL solutions, and reporting platforms to Azure Databricks and modern Lakehouse architectureExperience providing technical support and troubleshooting for production incidents related to Databricks, Spark workloads, data pipelines, and platform infrastructureRequired SkillsStrong capability in designing and developing enterprise data engineering frameworks for batch, streaming, ELT/ETL, and real-time analytics workloadsAdvanced knowledge of Azure Databricks, Delta Lake, Unity Catalog, and Azure Data Lake Storage Gen2Strong experience with Apache Spark, including PySpark and Spark SQL developmentStrong SQL programming skills with proficiency in Python and PySparkStrong understanding of data modelling techniques including dimensional modelling, Data Vault, medallion architecture, and enterprise data warehouse conceptsKnowledge of Azure Key Vault, Azure Monitor, Microsoft Entra ID, and Azure networkingAbility to optimise Databricks workloads through Spark tuning, cluster sizing, workload isolation, caching, partitioning, Delta optimisation, Photon, and cost optimisation practicesAbility to design enterprise monitoring, logging, dashboards, alerting, and platform health monitoring using Azure Monitor, Log Analytics, Application Insights, and Databricks system tablesExperience implementing CI/CD pipelines using Azure DevOps, automated testing, release management, and deployment automationAbility to develop reusable architecture patterns, reference implementations, development standards, naming conventions, coding standards, operational procedures, and platform documentationAbout AirswiftAirswift is an international workforce solutions provider within the energy, process and infrastructure industries. Airswift serves as a strategic partner to clients, offering a turnkey workforce solution to capture and deliver the top talent needed to complete successful projects by aligning with unique project needs. xzbcwsv With over 1000 employees and 9,000 contractors operating in over 60 countries, Airswift's geographical reach and pool of talent available is unmatched in the industry.