Senior Data Engineer
Senior Data Engineer at Novatus, a Series B RegTech SaaS and consulting firm, building a configuration-driven Databricks platform and Snowflake warehouse for regulatory transaction reporting. Day to day: designing Python/PySpark and Kafka pipelines, ensuring auditability and data quality, optimizing Spark/Snowflake workloads, and mentoring engineers during a migration from Python scripts and MySQL
- Compensation
- Not disclosed
- Location
- London, United Kingdom
- Work setup
- Not specified
- Seniority
- Senior
- Employment
- Not specified
- Experience
- 5+ years
- Category
- Data engineering
- Posted
Skills
Must have
- 8+ years as a software engineer, including 5+ years in data engineering, owning system design and delivery
- Experience designing auditable, reproducible data pipelines in regulated or high-integrity environments using Python and PySpark
- Able to write and optimize complex SQL queries on large data sets
- Strong data modeling and warehouse design fundamentals
- Strong software engineering fundamentals: clean code, DRY, SOLID, automated testing, CI/CD, and observability
- Deep experience with a cloud lakehouse / data warehouse (Databricks, Snowflake, BigQuery, Redshift, EMR, or similar)
- Experience with a workflow orchestrator (e.g. Airflow, Dagster, or Databricks Workflows)
- Proven system and platform design judgment; owns architectural decisions and can defend trade-offs
- Comfortable translating complex regulatory requirements into technical specifications
- Able to own large, ambiguous initiatives and deliver them in incremental, shippable slices
- Strong communication and collaboration skills, including translating between technical and non-technical stakeholders
Nice to have
- Experience with event-driven architecture patterns
- Hands-on experience with AWS cloud infrastructure
- Practical experience with infrastructure as code (e.g. Terraform)
- Experience modernizing legacy systems
- FinTech, RegTech, or financial services background
Full description
Novatus is a Series B scale-up RegTech SaaS provider and boutique consulting practice, enabling financial services firms to solve complex challenges and redefine what’s possible through expert-led technology and consulting. Across both our technology and consulting services, Novatus delivers practical solutions to some of the most complex regulatory, data and operational challenges in financial services.
Our SaaS platform, En>ACT (short for ensuring more accurate, complete and timely reporting), is a market-leading solution for regulatory transaction reporting and reconciliation, providing real-time, independent oversight of transaction reporting across all global regulatory regimes. En>ACT is the only real-time assurance solution that tests 100% of transactions and 100% of fields, replacing sample-based reviews and static PDF reports delivered months after the event with live, actionable insights. This helps firms remediate quickly, reduce risk, and meet regulatory obligations with confidence. Already trusted by more than 45 firms worldwide, the platform is scalable, proven, and designed for both business and technical users.
Alongside our SaaS offering, our unique model delivers consulting services to solve complex challenges, combining deep expertise, ownership and delivery to create lasting change. Our expertise spans Strategy & Operations, Data & AI, Risk & Resilience, Compliance, Transaction Reporting, and ESG. We embed ourselves within client teams to deliver both insight and execution, taking ownership of outcomes and driving measurable impact.
Since our launch in 2019, we have scaled rapidly, creating space for our people to grow the business. Backed by North American private equity investment, and partnerships with the London Stock Exchange Group and Snowflake’s global data platform, we are shaping the future of financial services through innovation in both consulting and technology. Now a team of 120 globally, our growth plans do not stop here: we are looking for top-tier talent to join us on our journey and enable our next phase of success.
Job Overview
As a Senior Data Engineer, you will be developing our configuration-driven data platform in Databricks, enabling non-engineers to define regulatory logic, and our Snowflake data warehouse, ensuring scalability, auditability, and fitness for client-facing regulatory use cases. You’ll be writing clean, maintainable, and well-tested code that follows best practices. As a senior member of the team, you will be providing technical leadership and mentorship to your colleagues.
You’ll join at a pivotal stage as we modernize our data infrastructure, migrating from Python scripts and MySQL to Databricks and Snowflake. Our systems power regulatory reporting for major financial institutions, requiring precision, traceability, and reliability.