Azure Data Engineer - Fabric

About Data Elephant

Data Elephant is a Canadian Data, Analytics & AI consulting firm helping organizations solve complex data, analytics, AI, and machine learning challenges. We combine business understanding with hands‑on technical delivery to build modern data platforms and turn data into useful outcomes.

The role

We are looking for a hands‑on Senior Data Engineer to build reliable data pipelines and transformation solutions across Microsoft Azure and Fabric. You will turn source data into trusted, usable datasets for reporting, analytics, AI, and machine learning. Working closely with architects, analysts, and clients, you will own engineering work from source analysis and development through testing, deployment, and production support.

What you will do

• Build and maintain ingestion and orchestration pipelines using Fabric Data Factory and Azure Data Factory, integrating databases, APIs, files, and enterprise applications.

• Develop ETL/ELT transformations using SQL, Python, PySpark, Fabric notebooks, and Dataflow Gen2, selecting the right approach for the workload.

• Implement OneLake, Lakehouse, Warehouse, and Delta Lake solutions, organizing raw, refined, and curated data using medallion architecture where appropriate.

• Develop full and incremental load patterns, change data capture where supported, parameterized pipelines, and reusable ingestion frameworks.

• Build dimensional models and curated datasets with analysts and business stakeholders; implement business rules, historical tracking, and source-to-target mappings.

• Implement data quality checks, automated tests, reconciliation, and validation. Handle schema changes, duplicate records, missing values, and late‑arriving data.

• Make pipelines reliable through logging, monitoring, alerts, retries, dependency management, and safe reruns; troubleshoot failures and document operational runbooks.

• Use Git and CI/CD to promote changes across environments; optimize SQL, Spark, and pipeline performance, implement agreed access controls, and participate in code reviews and mentoring.

What you bring

• Strong experience building and operating production data pipelines, with ownership of complex engineering work through deployment and support.

• Hands‑on delivery experience with Microsoft Fabric, including OneLake, Lakehouse or Warehouse, Fabric Data Factory pipelines, and notebooks.

• Strong SQL, Python, and PySpark skills, with practical experience processing data in Apache Spark and Delta Lake.

• Practical Azure experience with Azure Data Factory, ADLS Gen2, and Azure SQL, including integrating Azure services with Fabric.

• Solid understanding of ETL/ELT, orchestration, incremental processing, data modeling, and structured and semi‑structured data.