Principal Data Engineer
Principal Data Engineer leading data platform initiatives for a maritime logistics organization in Mossel Bay, South Africa (hybrid). Day to day: architecting scalable data pipelines, warehouses, and lakes on cloud platforms using Spark, Airflow, SQL, and Python/Scala, while mentoring junior engineers.
- Compensation
- Not disclosed
- Location
- Mossel Bay, South Africa
- Work setup
- Hybrid
- Seniority
- Principal
- Employment
- Not specified
- Experience
- 8+ years
- Category
- Data engineering
- Posted
Skills
Must have
- Master's degree in Computer Science, Engineering, or a related quantitative field
- 8+ years of experience in data engineering or a related role
- Extensive experience with big data technologies (e.g., Spark, Hadoop ecosystem)
- Proficiency in SQL and experience with various database systems (relational and NoSQL)
- Strong knowledge of data warehousing concepts and ETL tools
- Experience with cloud platforms (AWS, Azure, GCP) and their data services
- Proficiency in programming languages such as Python or Scala
- Experience with data pipeline orchestration tools (e.g., Airflow)
- Excellent leadership, communication, and problem-solving skills
- Ability to thrive in a hybrid work environment
Full description
About the Role Our client, a prominent organization in the maritime logistics sector, is seeking a highly experienced Principal Data Engineer to lead their data platform initiatives. This role, based in the scenic coastal city of Mossel Bay, Western Cape , offers a hybrid work arrangement, blending office collaboration with remote flexibility. You will be instrumental in designing, building, and maintaining scalable and robust data pipelines and infrastructure. The ideal candidate possesses a deep understanding of big data technologies, data warehousing, and ETL processes, with a proven ability to architect complex data solutions. You will guide technical direction, mentor junior engineers, and collaborate with data scientists and analysts to unlock the full potential of the organization's data assets. This is a senior leadership role with significant impact.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines and ETL processes.
- Architect and build data warehouses and data lakes to support analytics and reporting needs.
- Optimize data infrastructure for performance, reliability, and cost-efficiency.
- Implement data governance policies and ensure data quality and security.
- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements.
- Select and implement appropriate big data technologies and tools.
- Mentor and guide junior data engineers.
- Troubleshoot and resolve complex data-related issues.
- Stay abreast of emerging trends and best practices in data engineering.
- Drive innovation in data platform capabilities.
Requirements
- Master's degree in Computer Science, Engineering, or a related quantitative field.
- 8+ years of experience in data engineering or a related role.
- Extensive experience with big data technologies (e.g., Spark, Hadoop ecosystem).
- Proficiency in SQL and experience with various database systems (relational and NoSQL).
- Strong knowledge of data warehousing concepts and ETL tools.
- Experience with cloud platforms (AWS, Azure, GCP) and their data services.
- Proficiency in programming languages such as Python or Scala.
- Experience with data pipeline orchestration tools (e.g., Airflow).
- Excellent leadership, communication, and problem-solving skills.
- Ability to thrive in a hybrid work environment.
Benefits
- Highly competitive salary of R per annum.
- Hybrid work model for a balanced lifestyle.
- Comprehensive medical aid and retirement fund.
- Generous annual leave allowance.
- Opportunities for professional development and attending industry conferences.
- Relocation assistance may be considered for exceptional candidates.