Senior Data Engineer
Senior Data Engineer who leads the design and evolution of Hiscox's Europe data platform on Microsoft Azure and Databricks — building scalable pipelines, setting engineering standards, owning CI/CD and FinOps cost management, and leading a small engineering pod through a Databricks migration.
- Compensation
- Not disclosed
- Location
- Portugal
- Work setup
- Not specified
- Seniority
- Senior
- Employment
- Full time
- Experience
- 5+ years
- Category
- Data engineering
- Posted
Skills
Must have
- Hands-on experience with Databricks, including Spark/PySpark, Delta Lake, job orchestration, cluster configuration, and performance tuning
- Experience with Unity Catalog, data governance, and access management within Databricks
- Deep understanding of data modelling (star schema) and ETL processes
- Advanced experience with SQL and Python
- Strong knowledge of Azure Data Services, including Azure Data Factory
- Experience with CI/CD, version control, and deployment automation for data pipelines (e.g., Azure DevOps, Git)
- Experience with platform monitoring, observability, and cloud cost management (FinOps)
- Ability to translate business requirements into scalable technical solutions
- Strong communication skills, able to explain technical decisions to non-technical stakeholders
- Experience leading the day-to-day work of a small engineering team
- 5+ years of experience in Data Engineering or Data & Analytics roles
- Bachelor's or master's degree in Computer Science, Engineering, Mathematics, or a related field
- Fluent in English, both spoken and written
Nice to have
- Experience leading platform migrations, ideally within the insurance industry
Full description
Job Type:
Permanent
Build a brilliant future with Hiscox
About Hiscox
Hiscox is an international insurer specialising in luxury goods & property markets, as well as high-end professional insurance. As experts in risk, we empower individuals and businesses to achieve their ambitions. With offices in 9 European countries, we boast a highly international workforce, fostering a culture of inclusion, collaboration, and personal and professional growth.
About the Hiscox Data Team
The Europe Data Team is a cross-country, multidisciplinary group responsible for driving and enabling Hiscox Europe’s data capabilities. We combine expertise in Data Engineering, Data Analysis, and Data Management to deliver high-quality data pipelines, insightful reporting, and strong governance frameworks, working closely with business teams across Europe.
Our mission is to ensure trusted, consistent, and accessible data that empowers smarter decision-making and supports the evolution of our enterprise Hiscox Data Strategy.
The Role
The Senior Data Engineer for the Europe Data Team leads the design and evolution of our European data platform, which uses Microsoft Azure and Databricks. This role drives technical decisions aligned with the Hiscox Data Strategy, ensuring our data solutions are scalable, robust, efficient, and cost-effective.
Working with Data Analysts, Data Management, and business stakeholders across Europe, the Senior Data Engineer owns end-to-end engineering delivery for key platform components, sets the engineering standards the team works to, and leads the day-to-day work of a small engineering pod, while appraisal and progression remain with the Data Engineering Manager.
With ongoing data transformation in Europe, including the migration to Databricks and the decommissioning of legacy platforms, this position offers the chance to shape the technical direction of the platform while supporting daily operations and existing commitments.
Key Responsibilities
Platform Design & Engineering
- Lead the design and implementation of complex data pipelines and platform architecture on Databricks;
- Own the technical direction of the platform, including compute strategy, governance, and Delta Lake standards;
- Ensure the scalability, performance, and reliability of the Europe Data Platform;
- Lead data modelling and optimise data structures to ensure scalability, performance, and maintainability.
Platform Operations & Cost
- Own deployment execution, CI/CD, and observability, ensuring reliability and consistency of release processes;