Data Engineer - Senior Consultant
Senior data engineering consultant at UK tech consultancy Opencast, designing and building lakehouse/warehouse data platforms for clients using Microsoft Fabric, Azure, PySpark and SQL, and contributing mentoring and technical leadership. Hybrid role with regular travel to client sites across the UK; UK SC security clearance (or eligibility) is required.
- Compensation
- Not disclosed
- Location
- United Kingdom
- Work setup
- Remote
- Seniority
- Senior
- Employment
- Full time
- Experience
- Not specified
- Category
- Data engineering
- Posted
Skills
Must have
- Microsoft Fabric and Azure data services, including lakehouse, warehouse, notebooks and pipeline capabilities
- Building enterprise-scale data platforms, lakehouses and data warehouses
- Data modelling and dimensional modelling techniques such as Kimball
- ETL/ELT development and large-scale data integration
- PySpark, Spark SQL and modern transformation techniques
- Advanced SQL and relational databases including SQL Server, PostgreSQL and Oracle
- Integrating structured, semi-structured and unstructured data sources
- Migration and modernisation of legacy platforms such as Oracle, SQL Server/SSIS or Synapse
- Power BI semantic models, curated datasets and supporting analytics platforms
- Data quality, governance, metadata management and lineage
- Data security, classification and access control approaches
- Automated testing, validation, reconciliation testing and performance optimisation
- DataOps, CI/CD and version control practices
- Active Security Check (SC) clearance or eligibility and willingness to undergo the clearance process
- UK citizenship and continuous UK residence for the past five years (per SC vetting requirements)
- Weekly on-site work and regular travel to client locations across the UK
- Consulting skills: navigating ambiguity, working with technical and non-technical stakeholders, acting as a technical advisor
Nice to have
- Sharing knowledge, mentoring others and helping raise engineering standards
- Keeping pace with emerging technologies and evaluating new approaches
Full description
Hybrid - Newcastle, Edinburgh, Glasgow, Leeds, Manchester, Birmingham, Bristol, London
Commitment:
Permanent - Employee
Ready to build data platforms, not just pipelines?
At Opencast, we believe great data engineering is about more than moving and transforming data. It's about creating reliable, scalable foundations that help organisations make better decisions and deliver better services.
You’ll tackle complex engineering challenges across integration, migration, modelling and platform design, working with Microsoft Fabric, Azure and modern data technologies. We value engineering craft, thoughtful problem-solving and building solutions that are simple, robust and built to last.
Our data engineering capability is growing, and you'll help guide its direction through mentoring, recruitment and technical leadership, as well as your client work. If you want to shape both the solutions and capability around them, this could be the role for you.
The role
Opencast data engineers help organisations transform how they manage,useand learn from data. The work ranges from building modernlakehouseand warehouse platforms in Microsoft Fabric to modernising legacy environments, migrating complex workloads and improving the quality,governanceand reliability of critical data services.
This role offers the opportunity to work across the full data lifecycle, combining platform design, engineering,integrationand optimisation. Expect to work with structured, semi-structured and unstructured data, building batch and near real-time data pipelines that underpin critical business and public services.
You'llcollaborate with architects, analysts, softwareengineersand client teams to solve complex challenges and influence technical direction. The role also offers opportunities to contribute beyond delivery through mentoring, recruitment, community building and helping to define engineering standards and best practice as our capability continues to grow.
Requirements
This role involves weekly on-site work, and you’ll be required to travel regularly to client locations across the UK. We’re looking for people who value in-person collaboration, enjoy building strong relationships, and are passionate about delivering solutions that make a real difference for our clients.
Due to the nature of this role, you must either hold active Security Check (SC) clearance or be eligible and willing to undergo the clearance process.
SC clearance is granted in line with UK Government security vetting requirements. Typically, applicants must have UK citizenship and have lived continuously in the UK for at least the past five years, with no single absence exceeding 28 consecutive days. Nationality and immigration status may also be considered as part of the vetting process. We are unable to progress applications from candidates who do not meet these requirements.