Senior Data Engineer (CH1223)
Senior Data Engineer for a South African payment aggregator, designing and maintaining scalable ETL/ELT pipelines and warehouse solutions that integrate transactional, accounting, and CRM data. Core stack is Python, SQL, SQL Server, and PostgreSQL, with orchestration tools like Airflow.
- Compensation
- Not disclosed
- Location
- Stellenbosch, South Africa
- Work setup
- Not specified
- Seniority
- Senior
- Employment
- Not specified
- Experience
- 8+ years
- Category
- Data engineering
- Posted
Skills
Must have
- Degree in a quantitative field (Information Systems, Computer Science, Mathematics, Statistics, or Finance)
- 8+ years relevant experience in data engineering or related roles
- Proven experience in Python-based data engineering projects
- Strong proficiency in SQL and Python for data transformation and automation
- Experience with data orchestration tools (e.g., Airflow)
- Excellent problem-solving and communication skills
- Legal right to work in South Africa at time of application
Nice to have
- Understanding of dimensional modelling and data warehouse design
- Experience with systems like SQL Server, PostgreSQL, Sage One, Xero, Business Central, HubSpot and Microsoft Dataverse
- Agile (Scrum) development experience
- Experience in the payments or financial services industry
Full description
Our client, is a leading South African payment aggregator, enabling secure and efficient bill payments across a wide network of payment channels. They process millions of transactions monthly and are building a modern data infrastructure to support financial reporting, fraud detection, AI, and operational analytics.
They are seeking a skilled Senior Data Engineer to join our growing team. You will be instrumental in designing, building, and maintaining scalable, robust data pipelines and warehouse solutions that integrate transactional, accounting, and CRM data across multiple systems including SQL Server, PostgreSQL, Sage One, Xero, and Dataverse.
- Responsible for the preparation of data products and advanced technical thought leadership for the organisation.
- Deliver fit-for-purpose data products for consumption by advanced analytics.
- Provide clean, usable data to whomever may require it.
- Design, construct, install, test and maintain highly scalable data management systems.
- Build high-performance prototypes and proof of concepts. Research opportunities for data acquisition and new uses for existing data.
- Develop data set processes for data modelling, mining and production.
- Integrate new data management technologies and software engineering tools into existing structures and create custom software components (e.g. specialised UDFs) and analytics applications.
- Employ a variety of languages and tools (e.g. scripting languages) to marry systems together.
- Install and update disaster recovery procedures.
- Collaborate with data architects, modelers and IT team members on project goals and recommend ways to improve data reliability, efficiency and quality.
- In conjunction with Data Architect, determine which data management systems are appropriate.
Key responsibilities:
Data Pipeline Development:
- Build and maintain robust ETL/ELT pipelines using Python, SQL, and cloud native tools to ingest and transform data from 40+ payment networks and internal systems.
Data Integration:
- Join and harmonize data across diverse sources including transactional databases, accounting systems, and CRM platforms.
Collaboration & Governance:
- Work closely with developers, analysts, and business stakeholders to ensure data accuracy, security, and accessibility.
Monitoring & Optimization:
- Implement data quality checks, performance tuning, and monitoring for pipelines handling tens of millions of rows monthly.