Senior Data Engineer: Data Platform DevOps & Architecture
- Compensation
- Not disclosed
- Location
- London, GB
- Work setup
- Hybrid
- Seniority
- Senior
- Employment
- Full time
- Experience
- 5+ years
- Category
- Data engineering
- Posted
Skills
Full description
About the Role
We’re seeking a Senior Data Engineer to transform our analytics infrastructure from monolithic SQL/PostgreSQL systems into a modern Azure-based data platform with DevOps at its core. This permanent position focuses on building an automated, self-service architecture designed for continuous evolution.
You’ll bring 5+ years of data engineering experience with proven expertise in implementing enterprise-scale data platforms and DevOps methodologies. You’ll architect solutions using modern patterns, establish automated data pipelines with CI/CD practices, implement infrastructure-as-code, and mentor our team while building systems that adapt to emerging technologies.
We need someone who prioritises maintainable, scalable, and adaptable systems. We’re committed to Azure but flexible on specific products – we value pragmatic solutions based on actual needs.
Structure & Expectations
- Reports to: Product organisation – you’ll work closely with product teams to align data capabilities with business objectives
- Location: London office 3 days per week, flexible remote work the rest of the time
- Team: You’ll be the senior technical lead and engineer for data platform initiatives, working with business analysts, engineers, and product stakeholders
- Timeline: Initial focus on migration (6 months), then platform evolution, DevOps maturity, and capability expansion
Core Technical RequirementsAzure Data Platform Architecture
- Microsoft Azure data stack expertise: ADF, Delta Lake, ADLS Gen2, Synapse, Databricks (with flexibility to integrate non-Microsoft solutions)
- Experience with PostgreSQL and SQL Server, including data replication/integration patterns for migration
- Dimensional and relational data modeling expertise
- Medallion architecture implementation (bronze→silver→gold)
Data Engineering & Pipelines
- Proficiency in Python and SQL for ELT/ETL pipelines
- Batch, streaming, and micro-batching architectures
- CDC patterns and incremental load strategies at scale
- Experience with Delta Change Data Feed or equivalent real-time sync mechanisms
- Experience building ingestion pipelines from diverse external APIs (CRM, analytics, finance, observability tools) into a unified lakehouse
DevOps & Infrastructure
- Infrastructure as Code (Terraform preferred)
- CI/CD with Azure DevOps or GitHub Actions