Databricks data engineer

Total Experience: 6+yrs (min 3+yrs in Databricks)
Notice: Immediate only
Location: Bangalore (Hybrid)/Rest of India- Remote)
If interested, pls share your resume at
ABOUT THE ROLE
We are looking for experienced Data Engineers to join a high-impact engagement building an AI-driven data platform from scratch. You will design accelerators, define self-service frameworks for a growing user base, and own end-to-end pipeline delivery from enterprise source systems. Requirements will evolve you are expected to drive clarity, not wait for it.
WHAT YOU WILL DO
•   Design and build reusable data engineering accelerators and self-service frameworks on Databricks
•   Build end-to-end pipelines ingesting from enterprise systems including Bullhorn, Beeline, and SAP Fieldglass
•   Lead Unity Catalog adoption data discovery, lineage, access control, and sharing standards
•   Build API/MCP integrations enabling programmatic access to the data platform
•   Develop dbt models and semantic layer components for commercial and operational reporting
•   Embed AI tooling into development workflows and data pipeline design
TECHNICAL REQUIREMENTS
Databricks — Non-Negotiable
•   Production experience with Spark, Delta Lake, Workflows, DLT, and Repos
•   Unity Catalog implementation and governance
•   Performance tuning: partitioning, Z-ordering, caching, cluster configuration, AQE
Python & SQL — Mandatory
•   Strong Python: OOP, testing (pytest), modular design, packaging
•   Advanced SQL: window functions, CTEs, query optimisation at scale
•   Py Spark: Data Frame API, UDFs, Structured Streaming
Data Modelling & dbt
•   dbt Core or Cloud: models, macros, testing, documentation, semantic layer
•   Medallion architecture (bronze/silver/gold) and lake house design principles
Cloud & Infrastructure
•   Hands-on with AWS, Azure, or GCP — IAM, storage, secrets management, networking
•   CI/CD pipelines with Git-based workflows (Git Hub Actions, Azure Dev Ops, or equivalent)
Data Security
•   Row/column-level security, dynamic data masking, audit logging
•   Secrets and credential management in cloud-native environments
AI & Emerging Tooling
•   Practical experience with LLM APIs, prompt engineering, or AI-assisted development
SOURCE SYSTEM EXPERIENCE (ADVANTAGE)
•   Bullhorn (ATS/CRM) — API-based extraction and data model familiarity
•   Beeline — VMS platform, reporting exports, integration patterns
•   SAP Fieldglass — contingent workforce data, API or file-based ingestion
WHO YOU ARE
•   Independent — you deliver with incomplete requirements and drive clarity yourself
•   Proactive — you identify gaps, raise concerns early, and propose solutions without being asked
•   Fast finisher — you ship working solutions and iterate; you don't over-engineer
•   Communicator — clear with technical and non-technical stakeholders alike
•   Integrity & work ethic — non-negotiable, no exceptions
•   Curious learner — you follow the field and adapt quickly as AI tooling evolves
NICE TO HAVE
•   Kafka or event-driven architecture experience
•   Terraform or Pulumi for infrastructure management
•   Prior consulting or client-facing delivery experience
•   Staffing, HR tech, or workforce management industry background
Great attitude is a must. Integrity and work ethic are non-negotiable.