Senior Data Engineer

Ztek Consulting · Bangalore, India

Mandatory skills: Snowflake, Databricks, Spark

- 7–10 years of Data Engineering experience, with expertise in Snowflake, advanced SQL, OLAP platforms, and scalable reporting/data-store design.

- Strong data-modelling skills, including dimensional models, star/snowflake schemas, facts/dimensions, SCD Types 1/2, and Data Vault fundamentals.

- Hands-on experience building ETL/ELT pipelines using Python, Apache Spark, Databricks, dbt, and orchestration tools such as Airflow, ADF, or Databricks Workflows.

- Experience designing data lake/Lakehouse solutions for structured, semi-structured, and unstructured data using Parquet, Avro, JSON, and CSV, with effective partitioning and optimization.

- Proficiency in Snowflake features such as Snow pipe, Streams and Tasks, Dynamic Tables, Time Travel, clustering, secure data sharing, incremental processing, and cost/performance tuning.

Additional Desired Skills:

- Cloud experience with AWS, Azure, or GCP, including object storage such as S3, ADLS, or GCS; exposure to Kafka, CDC, Spark Structured Streaming, APIs, and integration tools is preferred.

- Strong understanding of data quality, reconciliation, observability, lineage, cataloguing, governance, RBAC, masking, encryption, and PII controls using tools such as Unity Catalog, Collibra, Alation, Great Expectations, or Soda.

- Knowledge of Git, CI/CD, automated testing, Terraform/IaC, Docker, Agile delivery, production monitoring, troubleshooting, and SLA management.

- Familiarity with Power BI, Tableau, or Looker, including semantic layers, query optimization, and workload management.

- Proven ability to own architecture and design reviews, translate business and reporting requirements into secure data products, communicate with stakeholders, estimate delivery, and mentor engineers.