Senior Data Engineer
Senior Data Engineer on C&W Services' internal EMEA & APAC data transformation team in London, building agentic AI workflows, Databricks-native Lakehouse pipelines, semantic models and an AI governance gateway. Day-to-day: designing data models, automating ingestion, and troubleshooting production data issues.
- Compensation
- Not disclosed
- Location
- London, United Kingdom
- Work setup
- Not specified
- Seniority
- Senior
- Employment
- Not specified
- Experience
- Not specified
- Category
- Data engineering
- Posted
Skills
Must have
- Systems thinking — build for scale, not just the ticket at hand
- Production troubleshooting and root cause analysis of pipeline failures, performance and data quality issues
- Data modeling — cross-functional data products, data contracts, complex business rules
- Advanced SQL including window functions, query optimization, MERGE/UPSERT
- Python and PySpark — reusable, parameterized functions; JSON, CSV, Parquet
- Deep Databricks experience (Declarative Pipelines, DABS, Delta Lake, Spark optimization, job orchestration)
- Familiarity with the Azure ecosystem
- Working understanding of Unity Catalog
- Around 6+ years of relevant experience
- Degree in a quantitatively rigorous field (computer science, data science, econometrics, mathematics, physics)
- Strong written communication — async updates and clear documentation
Nice to have
- Agentic data engineering experience — coding agents in real delivery, harness around them (repo conventions, reusable context, tests, guardrails)
- Geospatial data processing (H3 indexes, spatial SQL, point-in-polygon at scale)
- Recursive CTEs and complex SQL patterns
- Infrastructure familiarity (Azure Portal, resource management, CLI)
- Strong Git workflows and code review habits
- Real estate domain experience
Full description
Job Title
Senior Data Engineer
Job Description Summary
Senior Data Engineer | Databricks | Agentic AI | EMEA & APAC
We are the team leading the data transformation of our EMEA and APAC business, and we are hiring a Senior Data Engineer to help build what comes next.
Our Databricks-native Lakehouse is established and mature and already supports the business end to end. The work ahead is more ambitious: applying agentic capabilities at genuine enterprise scale, across a breadth of domains, and shaping how we win in the age of AI.
Job Description
Role: Senior Data Engineer
Senior Data Engineer | Databricks | Agentic AI | EMEA & APAC
We are the team leading the data transformation of our EMEA and APAC business, and we are hiring a Senior Data Engineer to help build what comes next.
Our Databricks-native Lakehouse is established and mature and already supports the business end to end. The work ahead is more ambitious: applying agentic capabilities at genuine enterprise scale, across a breadth of domains, and shaping how we win in the age of AI.
This is a hands-on senior role on key transformation projects across EMEA and APAC. You will work directly with the team lead and a group of analytical engineers who push each other to stay at the frontier of what is possible.
You will take on problems without a documented precedent, make architectural decisions of real consequence, and build depth across multiple domains early enough to influence where the platform goes next. Very few organisations in our sector have started this work, so the skill set you build here is not one you could assemble elsewhere in this industry.
What you will work on
- Driving the shift from a code writing team to an AI orchestration team. This means building agentic workflows that handle data engineering tasks and CI/CD, designing semantic models for downstream consumption, and implementing an AI Gateway for governance.
- Stewarding our AI amplified Databricks based framework, maintaining it and improving it incrementally as we learn.
- Designing data models that bring sources from anywhere in the world into a single coherent shape.
- Building data solutions for domains like custom MDM, geocoding, client mastering, and profiling. Real estate context is a plus but not required .
- Designing and implementing stable, automated ingestion pipelines from diverse sources.
- Owning production troubleshooting and root cause analysis. When things go wrong, you diagnose and resolve pipeline failures, performance issues, and data quality problems by working through code, logs, and documentation systematically.