Senior Data Engineer - Cloud Data Platform & ML Pipelines
Build and maintain cloud data pipelines and ML infrastructure on Databricks/AWS, collaborating with data scientists and BI teams to enable analytics and machine learning at scale.
- Compensation
- Not disclosed
- Location
- Central Plaza, Australia
- Work setup
- Not specified
- Seniority
- Senior
- Employment
- Not specified
- Experience
- 5+ years
- Category
- Data engineering
- Posted
Skills
Must have
- Minimum a bachelor’s degree qualified in Computer Science, Analytics, IT or a relevant discipline
Full description
Who We Are
iSelect and Compare the Market are trusted, well-known brands dedicated to helping millions of Australians compare and buy personal finance and household products such as insurance, energy, and loans. Together, we provide a seamless, customer-focused experience that makes it easier for Australians to find the right products to suit their needs.
What You’ll Be Doing
Reporting to the Data Engineering Manager, the Senior Data Engineer designs, builds, and evolves the organisation’s cloud data platform, delivering trusted, scalable, and secure data solutions that support business intelligence, advanced analytics, machine learning, and AI initiatives.
The role develops and optimises cloud-based data pipelines, products, and platform services, ensuring data is accessible, governed, and fit for reporting, analytics, predictive modelling, and Generative AI use cases.
Working closely with stakeholders across the business and technology teams, the Senior Data Engineer helps transform data into strategic assets that drive decision-making and innovation, while supporting AI-enabled solutions through strong data foundations and governance.
As a senior technical leader, the role provides expertise across data architecture, engineering standards, platform optimisation, automation, and DataOps/MLOps practices, contributing to the ongoing maturity of the organisation’s data and AI capabilities.
- Design, develop, and support scalable cloud-based data warehouse, lakehouse, and AI-enabled data platform solutions that support enterprise analytics, reporting and future ML/AI initiatives
- Build, optimise, and maintain batch, streaming, and near real-time data pipelines and data products to deliver trusted, high-quality, and governed data for business, analytics, and up-coming AI workloads
- Design, develop and deploy data workflows as an individual contributor with support of intelligent automation solutions which leverages AI agents, agentic workflows, LLMs, orchestration frameworks, and modern AI platform capabilities to improve operational efficiency and business outcomes
- Support advanced analytics and AI initiatives through the delivery of AI-ready datasets, vector search capabilities, RAG architectures, and secure, scalable data solutions that enable model development and operationalisation
- Implement and champion modern engineering practices including DataOps, SecOps, CI/CD, IaC automated testing, observability, platform automation, and responsible data access controls
- Ensure compliance with data governance, security, privacy, and quality standards, while collaborating with business and technology stakeholders to continuously improve platform capability, innovation, scalability, performance, and cost efficiency