Data engineer in financial industry
Senior big data engineer building a modern data platform on Databricks and Microsoft Azure for a financial-industry client: designing scalable data solutions, building batch/streaming ETL/ELT pipelines, implementing medallion architecture with Delta Tables, and integrating sources like SFTP and Azure Event Hub, with use of AI-assisted development tools.
- Compensation
- Not disclosed
- Location
- Warszawa, Poland
- Work setup
- Not specified
- Seniority
- Senior
- Employment
- Not specified
- Experience
- 5+ years
- Category
- Data engineering
- Posted
Skills
Must have
- At least 5 years of experience as a Data Engineer, Big Data Engineer, or similar role
- Strong hands-on experience with Python and PySpark
- Proven experience with Databricks, Delta Tables, and modern data architectures
- Practical knowledge of the Microsoft Azure ecosystem, including Azure Data Factory and Azure Event Hub
- Experience building and maintaining scalable ETL/ELT pipelines
- Advanced level of English
- Interest in modern technologies and experience using AI-assisted development tools in everyday work
Nice to have
- Familiarity with Elasticsearch
- Strong analytical skills and a proactive approach to solving complex data-related challenges
Full description
Описание
We are looking for a Senior Big Data Developer to join our team in building a modern data platform using Databricks and the Microsoft Azure ecosystem. In this role, you will be responsible for designing and developing scalable data solutions, implementing and optimizing ETL/ELT pipelines, and integrating data from multiple sources. If you are passionate about modern data engineering, AI-assisted development, and creating impactful data solutions.
Задачи
- Design and develop data solutions using Databricks, Python, and PySpark
- Build and optimize ETL/ELT pipelines for batch and streaming data processing
- Implement and maintain medallion architecture using Delta Tables
- Integrate data from multiple sources, including SFTP files, Delta Tables, and Azure Event Hub
- Work with Azure services such as Azure Data Factory to support end-to-end data workflows
- Utilize AI-powered development tools to improve code quality, testing, documentation, and productivity
Требования
- At least 5 years of experience in a Data Engineer, Big Data Engineer, or similar role
- Strong hands‑on experience with Python and PySpark
- Proven experience with Databricks, Delta Tables, and modern data architectures
- Practical knowledge of the Microsoft Azure ecosystem, including Azure Data Factory and Azure Event Hub
- Experience building and maintaining scalable ETL/ELT pipelines
- Advanced level of English
- Interest in modern technologies and experience using AI-assisted development tools in everyday work
- Будет плюсом: Familiarity with Elasticsearch; Strong analytical skills and a proactive approach to solving complex data-related challenges
Условия
- Great Place to Work
- Solid financial situation
- Contracts with the biggest brands
- Centre of internal trainings
- Many experts you can learn from
- Open and accessible management team
- Profit sharing
- Passion
- Sponsorship program
- Regular integration events and trips
- Comfortable and well-equipped offices
- MySii app
- Medical care