Data Engineer - Databricks
Tata Consultancy Services · Mumbai, India
FULL TIME
Job Description
Greetings from TCS!!!
TCS is hiring Data Engineer (Databricks with Azure) Location – Any Years of Experience – 4-15 years
Job Description
Role - Senior Data Engineer – Databricks
Required Technical Skill Set Databricks, PySpark, Delta Lake, MLflow, Azure, Data Lakehouse architecture, CI/CD, DataOps
Job description-
• Data Pipeline Development: Design, build, and optimize scalable data pipelines using Azure Data Factory, Azure Databricks, and other Azure services for data ingestion and processing. • Data Architecture: Develop and implement scalable data architectures, data warehouses, and data storage solutions (SQL/NoSQL) on Azure. • Data Quality & Security: Ensure data accuracy, consistency, integrity, and security through robust governance and best practice implementation. • Performance Optimization: Monitor, troubleshoot, and optimize data systems and pipelines for optimal performance, efficiency, and cost-effectiveness. • Collaboration: Work closely with data scientists, data analysts, business stakeholders, and solution architects to understand requirements and deliver effective data solutions. • Automation: Implement automated data pipelines and workflows, and contribute to CI/CD (Continuous Integration/Continuous Delivery) using tools like Azure DevOps.
TCS is hiring Data Engineer (Databricks with Azure) Location – Any Years of Experience – 4-15 years
Job Description
Role - Senior Data Engineer – Databricks
Required Technical Skill Set Databricks, PySpark, Delta Lake, MLflow, Azure, Data Lakehouse architecture, CI/CD, DataOps
Job description-
• Data Pipeline Development: Design, build, and optimize scalable data pipelines using Azure Data Factory, Azure Databricks, and other Azure services for data ingestion and processing. • Data Architecture: Develop and implement scalable data architectures, data warehouses, and data storage solutions (SQL/NoSQL) on Azure. • Data Quality & Security: Ensure data accuracy, consistency, integrity, and security through robust governance and best practice implementation. • Performance Optimization: Monitor, troubleshoot, and optimize data systems and pipelines for optimal performance, efficiency, and cost-effectiveness. • Collaboration: Work closely with data scientists, data analysts, business stakeholders, and solution architects to understand requirements and deliver effective data solutions. • Automation: Implement automated data pipelines and workflows, and contribute to CI/CD (Continuous Integration/Continuous Delivery) using tools like Azure DevOps.
Details
| Company | Tata Consultancy Services |
| Location | Mumbai, India |
| Type | FULL TIME |
| Niche | general |
Similar Jobs