Data Engineer - AWS, Pyspark, Databricks
A 12-month contract Data Engineer role in Melbourne, placed by Profusion Group at a major global enterprise. Day to day you build scalable AWS and PySpark data pipelines, support a cloud lakehouse/Databricks migration, and own data assets end-to-end with a focus on data quality and reconciliation.
- Compensation
- Not disclosed
- Location
- Melbourne, Australia
- Work setup
- Not specified
- Seniority
- Not specified
- Employment
- Contract
- Experience
- Not specified
- Category
- Data engineering
- Posted
Skills
Must have
- Strong commercial Data Engineering experience, incl. complex enterprise environments
- Advanced Python and SQL development skills
- Strong hands-on experience with Spark and PySpark
- Experience with AWS data services such as Glue, Athena, S3, Lambda and Step Functions
- Strong data modelling, data quality and reconciliation expertise
- Experience with CI/CD practices, Git-based development and Infrastructure as Code
- Ability to work autonomously, solve complex problems and own outcomes
Nice to have
- Databricks / Delta Lake experience
Full description
Job Description
- 12-month contract with a major enterprise organisation
- End-to-end ownership across cloud data engineering and data transformation initiatives
- Work with AWS, PySpark, Databricks and modern lakehouse technologiesCompany Overview
Our client is a highly respected global organisation undertaking a significant data transformation program.
Role Overview
As a Data Engineer, you'll play a key role in modernising enterprise data platforms and delivering high-quality data solutions that support critical business functions.
Working within a small, high-performing team, you'll own data assets from ingestion through to transformation, reconciliation, and production delivery.
This is an opportunity to work as part of a major cloud lakehouse migration, helping reshape how data is managed, consumed, and trusted across the organisation.
Key Responsibilities
- Design and build scalable AWS and PySpark data pipelines.
- Drive data quality, reconciliation, and platform reliability.
- Solve complex data challenges and optimise large-scale transformations.
- Support a major cloud lakehouse and Databricks migration initiative.
- Partner with business and technology stakeholders to deliver trusted data outcomes
Skills & Experience
We are seeking experienced Data Engineers who bring:
- Strong commercial experience in Data Engineering, including experience working within complex enterprise environments.
- Advanced Python and SQL development skills.
- Strong hands-on experience with Spark and PySpark.
- Experience working with AWS data services such as Glue, Athena, S3, Lambda and Step Functions.
- Databricks / Delta Lake experience highly regarded but not mandatory.
- Strong data modelling, data quality and reconciliation expertise.
- Experience with CI/CD practices, Git-based development and Infrastructure as Code principles.
- The ability to work autonomously, solve complex problems and take ownership of outcomes.
What's on Offer
People enjoy working in this team because they are trusted to solve challenging problems, own meaningful outcomes, and contribute to a large-scale transformation program that genuinely impacts the organisation's future direction.
Please click APPLY or reach out to for a confidential discussion.