CareCone Group
Data Engineer - Azure Databricks
A hands-on data engineer who designs, builds, and optimizes high-throughput ETL/ELT pipelines on Azure Databricks using Spark (Scala), SQL, and Python, following medallion architecture and Delta Lake best practices, with real-time integrations and CI/CD operationalization via Git/Azure DevOps. Permanent role in Sydney.
- Compensation
- Not disclosed
- Location
- Sydney, Australia
- Work setup
- Not specified
- Seniority
- Not specified
- Employment
- Full time
- Experience
- Not specified
- Category
- Data engineering
- Posted
Skills
Must have
- Azure Databricks
- Scala Spark
- SQL
- Python
- Design & develop scalable ETL/ELT pipelines in Azure Databricks using Spark (Scala) and SQL, following medallion architecture (Bronze/Silver/Gold) and Delta Lake best practices
- Proven expertise in designing real time and near real time integrations
- Operationalize workloads: jobs & workflows scheduling, CI/CD with Git (e.g., Azure DevOps), secrets management, and observability
Nice to have
- Data warehousing concepts
Full description
Role : Data Engineer - Azure Databricks
Type of role - Permanent Position
Location : Sydney, Australia
Primary or Mandatory skills
- Azure Databricks
- Scala Spark
- SQL
- Python
Good to have skills
Databricks, SQL, Python, Data warehousing concepts
Detailed Job description (Role & Responsibility, Skill and Experience range)
- We are looking for a hands on Developer Programmer to design, build, and optimize high throughput data pipelines on Azure Databricks using Spark (Scala) and SQL
- Design & develop scalable ETL/ELT pipelines in Azure Databricks using Spark (Scala) and SQL, following medallion architecture (Bronze/Silver/Gold) and Delta Lake best practices.
- Proven expertise in designing real time and near real time integrations
- Operationalize workloads: jobs & workflows scheduling, CI/CD with Git (e.g., Azure DevOps), secrets management, and observability.