Data Engineer
Data Engineer at an independent technology consultancy delivering a major data modernisation programme, likely for an insurance/financial services client. Day to day: designing, building and optimising ELT/ETL pipelines and data models in Snowflake, with Kafka (KSQL/KStreams) for real-time ingestion, plus Airflow orchestration and cloud (AWS/Azure/GCP) work.
- Compensation
- €60K / yr
- Location
- Portugal
- Work setup
- Hybrid
- Seniority
- Not specified
- Employment
- Not specified
- Experience
- Not specified
- Category
- Data engineering
- Posted
Skills
Must have
- Solid commercial experience as a Data Engineer delivering cloud-based data solutions
- Previous experience within insurance or financial services
- Strong knowledge of Snowflake, including data modelling and performance optimisation
- Hands-on experience with Kafka for data ingestion and stream processing
- Practical experience with KStreams and KSQL pipeline development and deployment
- Strong SQL skills and experience with Python or another scripting language
- Experience with data orchestration tools such as Airflow
- Experience with at least one major cloud platform (AWS, Azure or GCP)
- Strong understanding of data ingestion, transformation, modelling and integration
- Ability to work collaboratively with technical and non-technical stakeholders
- Experience with CI/CD practices and DevOps teams
Nice to have
- Experience building data pipelines for Policy Administration Systems, particularly Guidewire
- Understanding of data schemas and Confluent Schema Registry
- Broader experience across AWS, Azure or GCP
Full description
Location: Portugal or Spain (Hybrid/Remote)
Salary: Up to €60,000
About the Company
We are working with an independent technology consultancy that works across data, cloud and technology transformation, helping clients modernise their platforms and make better use of their data.
About the Role
As part of a major data modernisation programme, you will be responsible for designing, building and optimising the data flows that power a new cloud-based platform.
The role will focus heavily on data ingestion, transformation and modelling, with Snowflake as the core data warehouse and Kafka supporting real-time data capture and processing.
This is an opportunity for a Data Engineer who enjoys working with real-time data and solving technical challenges around streaming, integration and large-scale data transformation.
Responsibilities
- Design, build and maintain scalable ELT/ETL data pipelines within Snowflake.
- Implement data ingestion from a range of sources, including real-time streams using Kafka.
- Develop and deploy KSQL and KStreams pipelines for real-time data processing.
- Develop efficient and scalable data models within Snowflake to support analytics and business intelligence.
- Ensure data quality, integrity and reliability throughout the data lifecycle.
- Optimise data pipelines and Snowflake workloads for performance and scalability.
- Work closely with architects, analysts and data scientists to understand requirements and deliver effective data solutions.
- Contribute to data engineering standards, best practices and continuous improvement across the platform.
- Solid commercial experience as a Data Engineer, with a track record of delivering cloud-based data solutions.
- Previous experience within insurance or financial services.
- Strong knowledge of Snowflake, including data modelling and performance optimisation.
- Hands‑on experience with Kafka for data ingestion and stream processing.
- Practical experience with KStreams and KSQL pipeline development and deployment.
- Strong SQL skills and experience with Python or another scripting language.
- Experience with data orchestration tools such as Airflow.
- Experience working with at least one major cloud platform, such as AWS, Azure or GCP.
- Strong understanding of data ingestion, transformation, modelling and integration.