Lead Data Engineer
Remote lead data engineer on a government contract (inside IR35, umbrella, until 30/04) paying £700–£900/day. Leads design of secure, scalable ETL/ELT pipelines and permission-aware AI integrations using AWS, Python, SQL and Terraform; SC clearance required.
- Compensation
- £700–£900 / day
- Location
- London, United Kingdom
- Work setup
- Not specified
- Seniority
- Lead
- Employment
- Contract
- Experience
- Not specified
- Category
- Data engineering
- Posted
Skills
Must have
- SC security clearance
- Proven expertise in scalable ETL/ELT pipelines, API integrations, vector indexing, and enterprise AI retrieval architectures
- Strong knowledge of identity and access management, identity federation, data classification, UK GDPR and data protection
- Experience assessing data quality, validating AI readiness, and developing synthetic testing environments
- Advanced AWS, Python, and SQL skills, including query optimisation, Terraform, CI/CD, automated testing, and metadata management
- Demonstrated ability to lead multidisciplinary teams in agile environments and translate technical risks into strategic advice
Full description
Lead Data Engineer
Contract until April£Market rates/Day Inside IR35RemoteClearance: SC
We are seeking a Lead Data Engineer our Government client on a remote basis. The Lead Data Engineer role will be an initial contract until 30/04 paying between £700 - £900/Day Inside IR35. Ideally they want to be under the top end budget.
This role is inside IR35 - Due to the service of the role it will now be based on an Umbrella solution.
Lead the design and delivery of secure, scalable data pipelines and integrations supporting enterprise AI and workflow automation. Establish engineering standards, ensure data quality and compliance, and align solutions with wider enterprise data architectures and roadmaps.
Clearance: SC
Role Profile:
- Data Integration & Pipelines: Design and implement ETL/ELT workflows, APIs, and integrations across enterprise platforms and databases, aligning with existing standards.
- Security & Governance: Implement access controls, SSO, data classification, and permission-aware AI integrations in line with UK GDPR and data protection requirements.
- Data Readiness & Innovation: Assess data quality and suitability for AI, develop synthetic test datasets in secure sandboxes, and build domain-specific knowledge and retrieval systems.
- Engineering Standards & Observability: Develop reusable, secure, maintainable solutions, including metadata management, automated testing, CI/CD, and monitoring.
- Technical Leadership & Stakeholder Engagement: Lead cross-functional engineering teams, translate business requirements into technical solutions, and communicate data risks and recommendations to senior stakeholders.
- Data Engineering: Proven expertise in scalable ETL/ELT pipelines, API integrations, vector indexing, and enterprise AI retrieval architectures.
- Security & Compliance: Strong knowledge of identity and access management, identity federation, data classification, UK GDPR, and data protection requirements.
- Data Quality & Diagnostics: Experience assessing data quality, validating AI readiness, and developing synthetic testing environments for automation and machine learning.
- Cloud & Software Engineering: Advanced AWS, Python, and SQL skills, including query optimisation, Terraform, CI/CD, automated testing, and metadata management.
- Leadership & Communication: Demonstrated ability to lead multidisciplinary teams in agile environments and translate complex technical risks into clear strategic advice.