Lead Data Engineer, Technology Consulting- London, Leeds, Manchester or Newcastle
The Lead Data Engineer will lead complex data engineering projects, building AI-enabled data platforms and pipelines on AWS for diverse clients. The role involves managing teams, mentoring junior staff, and implementing data foundations for machine learning and Generative AI solutions.
- Compensation
- £84K–£100K / yr
- Location
- London, London, United Kingdom
- Work setup
- Hybrid
- Seniority
- Lead
- Employment
- Full time
- Experience
- Not specified
- Category
- Data engineering
- Posted
Skills
Must have
- Leading multiple, complex areas of data engineering delivery
- Experience managing teams to deliver agreed scope
- Building data pipelines using Spark, Python or SQL
- Experience with AWS and data/AI/ML services such as SageMaker
- Experience with big data and lakehouse platforms on AWS (EMR, Databricks or Snowflake)
- Building data platforms such as Data Lakes, Lakehouses or Data Meshes
- Understanding of Data Security, Data Governance and responsible AI principles
- Experience productionising and operating data and AI/ML solutions
- Coaching and mentoring engineers
- Active SC clearance
Nice to have
- Building automated data quality checks and metrics
- CI/CD tools (GitHub Actions, Azure DevOps, Jenkins, CircleCI)
- Building LLM or foundation-model applications using LangChain, vector databases or agentic AI
- MLOps and Infrastructure as Code (Terraform, CloudFormation)
- Data reporting and visualisation tools (Power BI, Tableau, Qlik)
- AI or Agentic techniques using GitHub CoPilot, Claude Code, or equivalent
Full description
Agency:
Credera
Job Function:
Consulting
Job Subfunction:
Technology Consulting
Job Description:
Would you like to be part of a fast-growing technology and engineering organisation that nurtures an open, collaborative learning environment? Are you passionate about technology, people and developing your coding skills? If this describes you, we’re trying to create the best engineering consultancy in the UK and would love for you to be part of the journey.
We draw on our reputation for outstanding delivery to allow our engineers to do the right thing for our clients, and not necessarily the easy thing. Innovation is in our DNA, and we encourage our engineers and consultants to work together to rethink conventional wisdom on how problems should be solved.
Our engineers take a holistic approach that addresses technology, process, and structure to make a lasting impact.
At Credera, you will work on diverse client projects, such as building modern, AI-enabled data platforms on AWS using DevOps and MLOps practices, including large, distributed workloads, batch and streaming pipelines, and the data foundations that power machine learning, Generative AI and Large Language Model solutions. Working with architects, technology consultants and public sector client stakeholders, you will help customers make use of the data available to them while gaining exposure to regulated environments, networks and the latest AI technologies.
Based out of any of our UK offices, this is a permanent role within our Data Function. At Credera, we operate a flexible hybrid working model. This requires 3 days per week working onsite - either at client site (as required) or in your local Credera office (London, Manchester, Leeds or Newcastle). We value collaboration and client impact, so our hybrid model is designed to strike a balance between flexibility, in-person connection, and delivering exceptional outcomes for our clients.
Requirements
As a Lead Engineer at Credera, you will spend most of your time on client work, with the remainder of your time spent working on internal projects as well as training and development. Consultancy to us is about customer delivery first, carried out by technically excellent people. Our aim is not to pigeon-hole but rather upskill our engineers so they can learn and expand their expertise, as well as deliver on a variety of business-critical client projects.
You are perceptive, personable, culturally sensitive and demonstrate a high degree of emotional intelligence. Being able to work collaboratively in a matrix organisation is essential, as is the ability to build data platforms using either cloud native products or commercial data analytics / data warehouse software.