Senior Data Engineer - Cloud Platforms
A senior data engineer designs, builds, and maintains scalable cloud data pipelines and ETL/ELT processes (AWS, Azure, or GCP) for large oil-and-gas datasets, supporting analytics and machine learning. Fully remote role focused on data warehousing, modeling, performance optimization, and governance.
- Compensation
- Not disclosed
- Location
- Upington, South Africa
- Work setup
- Remote
- Seniority
- Senior
- Employment
- Not specified
- Experience
- 7+ years
- Category
- Data engineering
- Posted
Skills
Must have
- Bachelor's degree in Computer Science, Engineering, or a related quantitative field
- Over 7 years of experience in data engineering, with strong emphasis on cloud-based data solutions
- Extensive experience with cloud platforms such as AWS (S3, Redshift, Glue), Azure (Data Lake, Synapse), or GCP (BigQuery, Dataflow)
- Proficiency in SQL and at least one programming language like Python or Scala
- Experience with big data technologies (e.g., Spark, Hadoop) and data warehousing concepts
- Strong understanding of data modeling, database design, and data governance principles
- Excellent problem-solving skills and ability to work effectively in a remote team
Full description
About the Role
Our client is seeking a highly skilled and experienced Senior Data Engineer to join their fully remote, cloud-native team, focusing on building and optimizing robust data pipelines for the energy sector. This role is pivotal in architecting, developing, and maintaining scalable data infrastructure on major cloud platforms, enabling advanced analytics and machine learning initiatives. You will work with diverse datasets from exploration, production, and operational systems to ensure data quality, accessibility, and performance. This is a fantastic remote opportunity for a data engineering expert passionate about leveraging cloud technologies to drive data-driven decision-making in the complex and dynamic oil and gas industry.
Key Responsibilities
- Design, build, and maintain scalable and reliable data pipelines on cloud platforms (AWS, Azure, GCP).
- Develop ETL/ELT processes for ingesting, transforming, and loading data from various sources into data warehouses and data lakes.
- Optimize data storage and processing performance for large-scale datasets.
- Implement data governance and security best practices in cloud environments.
- Collaborate with data scientists and analysts to ensure data is readily available and meets their needs.
- Evaluate and recommend new technologies and tools to enhance the data infrastructure.
Requirements
- Bachelor's degree in Computer Science, Engineering, or a related quantitative field.
- Over 7 years of experience in data engineering, with a strong emphasis on cloud-based data solutions.
- Extensive experience with cloud platforms such as AWS (S3, Redshift, Glue), Azure (Data Lake, Synapse), or GCP (BigQuery, Dataflow).
- Proficiency in SQL and at least one programming language like Python or Scala.
- Experience with big data technologies (e.g., Spark, Hadoop) and data warehousing concepts.
- Strong understanding of data modeling, database design, and data governance principles.
- Excellent problem-solving skills and ability to work effectively in a remote team.
Benefits
- Highly competitive salary and comprehensive remote benefits package.
- Generous home office setup budget and ongoing technology support.
- Flexible working hours and the ability to work from anywhere.
- Opportunities to work on transformative data projects in the energy sector.
- A collaborative and forward-thinking remote work culture.