Senior Data Engineer (Microsoft Fabric / Azure) – Hybrid, Sydney Preferred
Senior Data Engineer at an Australian Microsoft Data & AI consultancy, designing and building modern data platforms on Microsoft Fabric (Lakehouse, OneLake, Data Factory, Notebooks) for multiple clients. Core stack: Python, SQL/Spark, Azure DevOps/GitHub CI/CD, Power BI, with Databricks and Purview as pluses.
- Compensation
- Not disclosed
- Location
- Sydney, Australia
- Work setup
- Hybrid
- Seniority
- Senior
- Employment
- Not specified
- Experience
- Not specified
- Category
- Data engineering
- Posted
Skills
Must have
- Hands-on Microsoft Fabric delivery experience (Lakehouse, OneLake, Data Factory/pipelines, Notebooks, Warehouse, semantic models)
- Strong Python and SQL, including Spark-based transformation and performance tuning
- Modern data architecture: lakehouse and medallion patterns, dimensional modelling, batch and incremental ingestion, data quality
- CI/CD and DevOps practices using Azure DevOps and/or GitHub (Git branching, automated deployment of data assets, environment promotion)
- Power BI: building and optimising datasets and reports
- Consulting delivery experience: client-facing work, workshops, managing multiple concurrent projects
- Strong teamwork and communication with technical and business stakeholders, including mentoring
- Experience supporting enterprise data migration and transformation programs
- Full, unrestricted Australian work rights and based in Australia
Nice to have
- Databricks: Lakehouse architecture, Delta Lake, Spark pipeline design/optimisation, Unity Catalog
- Microsoft Purview: data governance, Data Hub/Unified Catalog, data quality, lineage, classification, sensitivity labelling, compliance
- Experience delivering into Australian government or other regulated environments
- Microsoft and/or Databricks certifications (e.g. DP-600, DP-700)
Full description
Job Description
About Us
At Agile Insights, we are passionate about empowering organisations to transform through the power of Data & AI. We are an independent, Australian-owned consulting firm and one of Australia’s leading Microsoft Data & AI partners — placed in the top 1% globally for Microsoft Data and AI expertise. We are a Microsoft Solutions Partner and Trusted Advanced Specialisation Partner, as well as a Databricks and ESRI partner.
We work with government agencies and mid-to-large enterprises across energy and utilities, financial services, healthcare, education, local government, retail, and transport — delivering modern data platforms, analytics, governance, and AI solutions end to end.
The Role
We’re seeking an experienced Senior Data Engineer to design and build modern data platforms on Microsoft Fabric , working across multiple clients and projects as part of our Data & Analytics Practice.
Sydney is our preferred location , but we will consider strong candidates based in other Australian states.
What you’ll bring (essential)
- Microsoft Fabric — hands-on delivery experience across Lakehouse, OneLake, Data Factory / pipelines, Notebooks, Warehouse, and semantic models. Fabric is the primary technology for this role.
- Strong Python and SQL , including Spark-based transformation and performance tuning.
- Modern data architecture — lakehouse and medallion patterns, dimensional modelling, batch and incremental ingestion, data quality.
- CI/CD and DevOps practices — source control and deployment pipelines using Azure DevOps and/or GitHub (Git branching strategies, automated deployment of data assets, environment promotion).
- Power BI — building and optimising datasets and reports on top of the platforms you deliver.
- Consulting delivery experience — comfortable working directly with clients, running workshops, and balancing multiple concurrent projects and priorities.
- Strong teamwork and communication — you work collaboratively in project teams, share knowledge, mentor others, and communicate confidently with both technical and business stakeholders.
- Experience supporting enterprise data migration and transformation programs.
- Full, unrestricted Australian work rights and based in Australia.
Nice to have
- Databricks — Lakehouse architecture, Delta Lake, Spark pipeline design and optimisation, Unity Catalog.
- Microsoft Purview — data governance, Data Hub / Unified Catalog, data quality, lineage, classification, sensitivity labelling, and compliance.