Azure Data Engineer

We are looking for an experienced Azure Data Engineer to join our growing data and analytics team. The ideal candidate will have strong expertise in designing, developing, and maintaining scalable cloud-based data solutions using Microsoft Azure technologies. You will work closely with business stakeholders, data analysts, and engineering teams to build reliable and high-performing data pipelines and platforms.

Responsibilities:

• Design, develop, and maintain scalable ETL/ELT data pipelines using Azure Data Factory (ADF).
• Build and optimise big data processing solutions using Azure Databricks and PySpark.
• Develop and manage cloud-based data architectures on Microsoft Azure.
• Integrate data from multiple sources, including APIs, databases, flat files, and cloud platforms.
• Create and optimise SQL queries, stored procedures, and database objects.
• Implement data warehousing solutions and support reporting/analytics requirements.
• Ensure data quality, security, governance, and performance optimisation.
• Collaborate with cross-functional teams, including data analysts, BI developers, and solution architects.
• Monitor, troubleshoot, and improve existing data workflows and pipelines.
• Participate in CI/CD implementation and deployment automation for data engineering projects.
• Work with Agile/Scrum teams and contribute to sprint planning and technical discussions.

Requirements:

• Strong experience in Azure data engineering technologies.
• Hands-on expertise in Azure Data Factory (ADF).
• Experience with Azure Databricks and PySpark.
• Strong proficiency in SQL and relational databases.
• Experience building ETL/ELT pipelines and data integration solutions.
• Good understanding of Data Warehousing concepts and architectures.
• Experience with Azure Synapse Analytics.
• Strong knowledge of Python for data processing and automation.
• Familiarity with CI/CD pipelines and DevOps practices.
• Experience working with cloud-based data platforms and distributed systems.

Preferred Skills:

• Experience with Power BI or other visualisation tools.
• Knowledge of Delta Lake, Spark optimisation, and big data technologies.
• Exposure to Azure DevOps and Git version control.
• Understanding of data governance and security best practices.
• Certifications in Microsoft Azure Data Engineering are a plus.

Educational Qualification:

• Bachelor's degree in computer science, information technology, or a related field.