Gcp senior data engineer

GCP Senior Data Engineer | 6-10 Years of experience |

Chennai/ Hyderabad / Kolkata /Bangalore

Required Technical Skills

Python SQL Py Spark GCP (Preferred) or Azure Snowflake Airflow Jenkins / Jira / Git CI/CD Kubernetes / Docker

Must-Have

Strong development experience in Python, Py Spark, and SQL. Experience with cloud platforms (GCP preferred). Experience working in an Agile delivery environment. Experience with orchestration tools such as Airflow and ADF. Experience with GCP services: GCS Bucket Cloud Functions Dataproc Dataflow Pub/Sub Big Query Hands-on experience in Snowflake. Experience building automated data pipelines. Experience in data analysis and data exploration. Experience working in a multi-developer environment using version control tools such as Git. Strong critical thinking, communication, and problem-solving skills.

Good-to-Have

Understanding of Dev Ops best practices and CI/CD. Understanding of containerization technologies such as Kubernetes and Docker. Healthcare domain knowledge.

Key Responsibilities

Perform data analytics on large-scale datasets to solve complex business problems and recommend solutions. Develop and enhance end-to-end data pipelines using big data technologies such as Hadoop, Cloud platforms, and the Spark ecosystem. Troubleshoot complex data structures and perform root cause analysis to proactively resolve operational and product issues. Develop and document test cases, execute testing, and share UAT results as part of Agile sprints. Perform end-to-end data pipeline integration testing in compliance with change management processes and standards. Adhere to coding, design, and application standards, including Python and SQL best practices. Leverage Dev Ops practices for production deployments, including CI/CD, test-driven development, and version control using Jenkins, Jira, and Git. Collaborate with business stakeholders to define product objectives, business requirements, system designs, and key performance indicators (KPIs).