Senior Data Engineer-Product Data Quality
Build and maintain cloud-based data pipelines and microservices that process product data at scale using Databricks, Spark, Airflow and AWS, then surface insights to customers.
- Compensation
- Not disclosed
- Location
- Dublin
- Work setup
- Not specified
- Seniority
- Senior
- Employment
- Not specified
- Experience
- Not specified
- Category
- Data engineering
- Posted
Skills
Must have
- deep experience designing and implementing data pipelines with Databricks, Spark and Airflow
- experience with Python, PostgreSQL, Elastic Search, Kubernetes and AWS services
- ability to collaborate with ML engineers, applied scientists and software engineering teams
Nice to have
- experience with Kotlin and cloud-based microservice architectures
- strong communication skills for documentation and stakeholder management
Full description
ABOUT THE ROLE & TEAM
The Product Data Quality team is looking for a talented data engineer who has a passion for building applications at the intersection of data engineering, software development and data science. Our team consists of a great mix of Data and backend Software Engineers, Product Managers and Applied Scientists that are tasked with gaining insights into our assortment of hundreds of thousands of live products using AI, and then bringing these insights to our customers at scale in new and engaging ways.
As a member of the Product Data Quality team, you will collaborate closely with your engineering colleagues, including those in product and applied science teams, to design, build, deploy, and monitor cloud-based data processing architectures. These systems intelligently gather machine learning insights from multiple independent data sources and combine them to enhance our product information in a scalable and maintainable manner. In addition to data engineering pipelines, the team also develops and maintains cloud-based microservice architectures, for example a flexible rules engine and an image customization service. We work with a leading edge technology stack composed of Databricks, Python, Airflow, Kotlin, Kubernetes, Elastic Search, PostgreSQL and other AWS services amongst others - check our Tech Blog for more information on the technologies we use.
WHAT WE’D LOVE YOU TO DO (AND LOVE DOING)
- Lead the design, development, and deployment of the team's data pipelines, dashboards and other systems, demonstrating seasoned expertise.
- Exhibit robust stakeholder management skills, fostering effective communication and collaboration with engineering peers, applied scientists, and product owners throughout the organisation.
- Design, construct, and maintain resilient architectures adaptable to evolving product requirements.
- Consistently deliver high-quality, testable, and dependable systems, fostering a culture of excellence through comprehensive reviews and mentorship.
- Embrace agility by iteratively testing features with stakeholders while maintaining a holistic perspective on project goals.
- Prioritise team processes, automation, communication, and collaboration to bolster the team's velocity and output, driving toward collective success.
WE’D LOVE TO MEET YOU IF
- You have deep experience and passion in designing and implementing data pipelines, utilising technologies such as Databricks, Spark and Airflow.
- You thrive in collaborative environments, seamlessly integrating with various stakeholders and navigating complex domains, such as ML Engineers, Applied Scientist and other Software Engineering teams.
- You’re excited about not just the “how” but also the “what”, putting yourself in a customer mindset to build compelling experiences.