Data Engineer
Designs and builds scalable data pipelines and warehouses in Azure, using T-SQL, KQL, Python, and Power BI to support analytics and AI integrations for a Microsoft cloud and AI-focused company.
- Compensation
- Not disclosed
- Location
- South Africa
- Work setup
- Not specified
- Seniority
- Not specified
- Employment
- Full time
- Experience
- 3+ years
- Category
- Data engineering
- Posted
Skills
Must have
- 3+ years of experience as a Data Engineer or similar role.
- Strong experience with Azure-based data services, particularly Microsoft Fabric, Data Factory, Power BI, and Data Lakes.
- Proficient in T‑SQL, KQL, and Python for data processing and scripting.
- Experience with Databricks, Delta Lake, or Apache Spark technologies.
- Solid understanding of data warehousing principles, dimensional modelling, and ETL/ELT processes.
- Familiarity with DevOps practices, including CI/CD pipelines in Azure DevOps.
- Experience working in Agile teams and collaborating across disciplines.
- AGILE Development principles and practices.
Full description
Get AI-powered advice on this job and more exclusive features.
As a frontier partner, we grow through great people, smart tech, and teamwork between humans and AI.
Cloud Direct is looking for a full-time Data Engineer to focus on designing and building consistent, scalable data solutions across the business. The role involves developing and maintaining data pipelines, supporting data warehousing, and ensuring high-quality, cleansed, and normalised datasets for reporting, analytics, and integration into AI-driven tooling.
Working as part of a close-knit development team within a leading company in the realm of Microsoft cloud and AI, this role at Cloud Direct offers the perfect opportunity for an ambitious individual to develop their career!
What you'll do:
- Act as the team’s go-to expert for data engineering and analytics solutions.
- Work with product and business stakeholders to translate reporting and AI integration requirements into technical solutions.
- Develop and manage data warehouse solutions, ensuring data is cleansed, normalised, and structured for analytics and AI.
- Write high‑quality, well‑documented T‑SQL, KQL, and Python code for transformations, validations, and orchestration.
- Collaborate with developers to enable data access for machine learning, LLMs, and other intelligent systems.
- Support reporting and dashboarding needs via Power BI, including data modelling and DAX.
- Work closely with developers and analysts to review and implement designs.
- Complete full functional testing of new data solution releases.
- Ensure data systems are reliable, secure, and meet compliance requirements.
- Manage development backlog & assigned tasks within Azure DevOps.
- Weekly progress reporting against targets, projects, and development backlog.
- Work directly on customer projects if required.
What you'll need:
- 3+ years of experience as a Data Engineer or similar role.
- Strong experience with Azure-based data services, particularly Microsoft Fabric, Data Factory, Power BI, and Data Lakes.
- Proficient in T‑SQL, KQL, and Python for data processing and scripting.
- Experience with Databricks, Delta Lake, or Apache Spark technologies.
- Solid understanding of data warehousing principles, dimensional modelling, and ETL/ELT processes.
- Familiarity with DevOps practices, including CI/CD pipelines in Azure DevOps.