Data Engineer - Databricks
12-month contract Data Engineer role at a major financial services/insurance organisation's data modernisation program. Day to day: building scalable pipelines and Bronze/Silver/Gold models in Databricks with Python/PySpark, SQL, dbt and Delta Lake, and migrating legacy (SAS) platforms to a cloud lakehouse. Hybrid in Sydney or Brisbane.
- Compensation
- Not disclosed
- Location
- Sydney, Australia
- Work setup
- Not specified
- Seniority
- Not specified
- Employment
- Contract
- Experience
- Not specified
- Category
- Data engineering
- Posted
Skills
Must have
- Strong hands-on experience with Databricks, PySpark and SQL
- Experience with dbt, Delta Lake and lakehouse architecture
- Proven experience developing and maintaining ETL/ELT pipelines
- Experience with cloud platforms, ideally Azure or AWS
- Understanding of CI/CD, data testing and production support
Nice to have
- Experience in insurance or financial services highly regarded
- Exposure to AI-assisted development tools such as GitHub Copilot or Databricks Genie
Full description
Location: Sydney / Brisbane (Hybrid)
Contract: 12 Months
Industry: Financial Services / Insurance
Data Engineer - Databricks
Location: Sydney / Brisbane (Hybrid)
Contract: 12 Months
Industry: Financial Services / Insurance
The Opportunity
We are seeking an experienced Data Engineer to join a major financial services organisation undergoing a significant data modernisation program.
You'll work within a modern Databricks environment, building scalable data pipelines, transforming complex datasets and supporting the migration of legacy data platforms to a cloud-based lakehouse architecture.
Key Responsibilities
- Design, develop and optimise data pipelines using Databricks, Python/PySpark and SQL.
- Build and maintain data transformations using dbt and Delta Lake.
- Develop scalable data models across Bronze, Silver and Gold layers.
- Support data migration from legacy platforms, including SAS.
- Implement data quality checks, testing and performance optimisation.
- Collaborate with data engineers, analysts and business stakeholders.
Skills & Experience
- Strong hands-on experience with Databricks, PySpark and SQL.
- Experience with dbt, Delta Lake and lakehouse architecture.
- Proven experience developing and maintaining ETL/ELT pipelines.
- Experience with cloud platforms, ideally Azure or AWS.
- Understanding of CI/CD, data testing and production support.
- Experience in insurance or financial services highly regarded.
- Exposure to AI-assisted development tools such as GitHub Copilot or Databricks Genie advantageous.
Why Join?
- Work on a large-scale data transformation program.
- Modern Databricks and cloud technology environment.
- Collaborative engineering team with hybrid flexibility.
- Opportunity to contribute to AI-enabled data engineering practices.
Additional information
- Long Term Engagement
- Sydney or Brisbane - Hybrid