Senior Data Platform Engineer

SoTalent · Bangalore district, India

Senior Data Platform Engineer

Location: Bangalore, India

Work Arrangement: Hybrid

Employment Type: Full-time

A leading global consumer health organisation is seeking a Senior Data Platform Engineer to build, automate, and operate enterprise-scale data platforms using modern cloud, data, and container technologies.

This role focuses on platform engineering, DevOps automation, reliability, observability, security, and self-service enablement for data engineering and analytics teams. The ideal candidate will have strong experience working with modern data platforms, Infrastructure as Code, automated deployments, and cloud-based environments.

Key Responsibilities

Data Platform Engineering

• Design, implement, and support enterprise data platforms using Databricks, Snowflake, and Microsoft Fabric.
• Build reusable platform capabilities for self-service onboarding, provisioning, monitoring, and governance.
• Standardise platform architecture, deployment patterns, and operational processes.
• Support Development, QA, and Production environments.

Kubernetes & Cloud Infrastructure

• Build and manage Kubernetes-based services supporting data processing workloads.
• Deploy and operate containerised applications and platform services.
• Implement auto-scaling, high availability, disaster recovery, and resiliency solutions.
• Manage infrastructure using Infrastructure as Code tools such as Terraform, Bicep, and Helm.

DevOps & Automation

• Develop CI/CD pipelines for platform deployment and configuration management.
• Automate provisioning of Databricks workspaces, Snowflake objects, Microsoft Fabric resources, and Kubernetes infrastructure.
• Implement GitOps and automated release management practices.
• Develop automation frameworks to reduce manual operational effort.

Reliability & Operations

• Establish monitoring, observability, and alerting across platform components.
• Improve platform reliability, stability, and performance.
• Conduct root-cause analysis and remediation of platform incidents.
• Support production operations and critical issue resolution.

Security & Governance

• Implement security controls, RBAC, secrets management, and network security.
• Ensure compliance with enterprise governance standards.
• Support metadata management, auditing, lineage, and monitoring.
• Partner with cybersecurity teams to address vulnerabilities and risks.

Cost & Capacity Management

• Optimise Databricks clusters, Snowflake warehouses, Microsoft Fabric capacities, and Kubernetes resources.
• Monitor platform utilisation and identify cost-saving opportunities.
• Implement FinOps practices and governance guardrails.

Requirements

• Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field.
• 5–7 years of experience in Platform Engineering, DevOps, Cloud Engineering, or Data Platform Operations.
• Strong experience supporting Databricks, Snowflake, or Microsoft Fabric.
• Production experience managing Kubernetes-based workloads.
• Hands-on experience with CI/CD, Infrastructure as Code, and cloud-native architectures.
• Strong scripting and automation skills using Python, PowerShell, Bash, or similar languages.
• Experience working within Agile and DevOps delivery environments.

Technical Skills

Data & Cloud Platforms

• Databricks
• Snowflake
• Microsoft Fabric
• Delta Lake
• Data Lakehouse Architecture
• Microsoft Azure
• AWS or GCP exposure

Kubernetes & Containers

• Kubernetes
• Docker
• Helm
• Kustomize
• Container lifecycle management

DevOps & Infrastructure

• Azure DevOps
• GitHub Actions
• Jenkins
• Git
• CI/CD
• Terraform
• Bicep
• ARM Templates
• Infrastructure automation

Monitoring & Observability

• Prometheus
• Grafana
• Azure Monitor
• Log Analytics
• OpenTelemetry

Security

• RBAC
• Azure AD / Entra ID
• Key Vault
• Secrets management
• Security automation

Programming

• Python
• SQL
• PowerShell
• Bash

Preferred Qualifications

• Databricks Data Engineer or Platform Administrator certification.
• SnowPro certification.
• Microsoft Fabric certification.
• Certified Kubernetes Administrator (CKA).
• Terraform Associate certification.
• Experience with Apache Airflow, Kafka, Spark on Kubernetes, Kyuubi, GitOps, ArgoCD, service mesh technologies, or Platform SRE practices.

This is an opportunity to work on large-scale, cloud-based data infrastructure , with a strong focus on automation, reliability, security, and modern data platform engineering.