Data Engineer - SC Cleared

MORELA

Increase your chances of reaching the interview stage by reading the complete job description and applying promptly.
AI & Engineering Recruitment

Data Engineer (RAG & Retrieval)
2-year contract | Central London | Active SC | Outside IR35
Morela is supporting a specialist technology consultancy on a 2-year Data Engineer contract in Central London, delivering on a UK government programme. Outside IR35, rate TBC. We need a genuinely strong all-round data engineer first and foremost, with recent hands-on RAG experience on top. Active SC is essential - eligibility will not be considered.
About the Client
A specialist technology consultancy embedding engineers into SC-cleared government delivery teams. They focus on long-term programme work and give their engineers real ownership of what they build.
The Role
You will own the data platform behind the programme, from ingestion and modelling through to the retrieval layer that powers its AI capability. The core of the job is solid data engineering - reliable pipelines, clean models and well-governed data. On top of that you will build the RAG pipelines that turn large volumes of unstructured material into accurate, access-controlled answers.
Responsibilities
Core data engineering
Design, build and maintain robust batch and streaming pipelines (ETL/ELT) across structured and unstructured sources
Model data for analytics and downstream use, with clear schemas, lineage and documentation
Own data quality, validation, monitoring and alerting so the platform can be trusted
Optimise pipelines and storage for performance, cost and scale
Apply security, governance and access controls appropriate to an SC environment
RAG and retrieval
Build ingestion for unstructured content, including document parsing, OCR and metadata enrichment
Build production RAG pipelines: chunking, embeddings, hybrid search (vector + BM25), reranking and context assembly
Enforce document-level access controls at query time
Evaluate retrieval and generation quality with measurable metrics, and iterate
Work with self-hosted and open-weight models in secure or air-gapped environments
SC clearance: Active SC held, not eligibility
Essential Requirements
Core data engineering
Strong commercial data engineering background, building and running production pipelines end to end
Excellent Python and SQL
Solid grasp of data modelling, warehousing and lakehouse concepts
Hands-on with orchestration (e.g. Airflow, Dagster) and distributed processing (e.g. Spark)
Experience with cloud data platforms (AWS, Azure or GCP) and containerised deployment (Docker, Kubernetes)
Good engineering practice: version control, CI/CD, testing and infrastructure as code
RAG
Recent RAG delivery: Hands-on RAG systems built and shipped recently - current work, not an old proof of concept
Vector databases (e.g. pgvector, OpenSearch, Qdrant, Weaviate), embeddings, hybrid search and reranking
Experience evaluating RAG systems properly (e.g. , faithfulness)
Streaming platforms such as Kafka
Agentic or multi-hop retrieval, GraphRAG, query rewriting
vLLM or similar for on-prem model serving
LangChain, LlamaIndex, RAGAS or Langfuse
Government, defence or national security programmes
Desirable Experience
Working Environment
Central London, SC-cleared environment, embedded in the programme team. xrexnqw Outside IR35, 2-year contract.
For immediate consideration email