Senior/Lead Data Engineer

Data Engineers – Global Investment Firm | New York & London
Apply now, read the job details by scrolling down Double check you have the necessary skills before sending an application.
Build the data platform behind trading, research and risk at a top-tier investment firm.My direct client, a premier global investment firm is investing heavily in its data. The CTO has a big mandate to modernise the stack and build a more robust platform, and the team is hiring urgently. The team is small and growing, so your work will be visible and you'll have a direct say in the architecture. I placed a key person on this team earlier this year who is doing well and loves the environment and work. New York: four openings. One Lead Data Engineer and three mid/senior individual contributors. London: one mid/senior individual contributor, working closely with the New York team.What you'll work onData pipelines for trading, alpha generation, research, risk and accountingGolden-source datasets the whole firm relies on: security master, account master and price masterShared Python libraries for data APIs, logging and core functionsThe firm's AI tools, including LLM search and summarisation of financial documents and market commentaryData quality, observability and governance using modern toolsClose work with quants, traders, risk managers, the Chief Data Officer and the Head of DevelopmentMust-havesHands-on experience with both Snowflake and DatabricksExcellent Python (including PySpark) and SQL, plus Spark, dbt and AWS3+ years of development experience, with at least 2 years focused on data engineeringDeep ETL/ELT experience, strong data modelling, and SQL performance tuning (PostgreSQL or SQL Server)Docker and Kubernetes in cloud environmentsSome finance experience. A trading firm or hedge fund is ideal, or another fast-paced financial services firm.A degree in Computer Science or a related fieldNice to haveA software engineering background as well as data engineering. The hiring manager values varied technical experience.Data catalogue and data lineage experience (for example OpenMetadata or Great Expectations)LangChain/LangGraph, vector databases and RAGMarket data knowledge across equities, fixed income, futures and optionsFastAPI, Dask, or building BI dashboardsWhy this roleReal modernisation work with backing from a new CTOA small team where you'll have a lot of influenceBonus buy-outs possible for the right candidatesProcess: a Zoom with the hiring manager, a Python-focused take-home on CoderPad, then a half-day onsite that includes a live coding session. Onsite 4 days a week in both New York and London Please note:Unfortunately, our client can't offer visa sponsorship. Candidates must be authorised to work in the US or UK without sponsorship, now and in the future.No third-party recruiters or agencies, please. xzbcwsv We're only considering direct applicants.