Data engineer

Roles and Responsibilities

Architect and maintain enterprise-grade ELT and ETL data pipelines using Python, Py Spark, Kafka, and Databricks to manage large-scale risk data. Build and deploy Gen AI agents utilizing Google ADK, Google Flash 2.5+ LLMs, and Model Context Protocol (MCP) integrated with Human-in-the-Loop workflows. Design, automate, and deploy microservice integrations for data-intensive applications on Open Shift and Kubernetes using robust CI/CD pipelines. Implement data federation layers supporting Lambda and Data Mesh architectures via Starburst to enable AI/ML and NLP use cases. Leverage agentic AI platforms and development assistants such as Devin. AI and Git Hub Copilot with prompt engineering to increase engineering velocity. Enforce data governance, risk management policies, and regulatory compliance standards across all data platforms.

Preferred Candidate Profile

Work Experience: 8+ years in large-scale application development with 5+ years in a Python and Py Spark Data Engineering lead role. Educational Background: Bachelor's degree in Computer Science, Engineering, or a related field (Master's degree preferred). Core Technical Skills: Python, Py Spark, Databricks, Google ADK, LLMs, Fast API, Spring Boot, Microservices, Kafka, SQL, Data Mesh, Starburst. Infrastructure and Cloud: Kubernetes, Open Shift, Docker, Cloud-Native Infrastructure, CI/CD pipelines. Industry Context: Data engineering experience in Banking Risk, Retail Products, Cards, Mortgage, Deposits, or Wealth Management. Assumed Requirements / Certifications: Databricks Certified Data Engineer, AWS Certified Data Analytics, or Azure Data Engineer Associate.