Data Engineer
Contract, onsite Data Engineer in Malaysia building Enterprise Data Lake ETL/ELT pipelines across Bronze/Silver/Gold layers for a finance-sector client: schema design, data cleansing/transformation, API and Kafka streaming integration, plus validation, monitoring and documentation. Core stack: Python, NiFi, Airflow, Apache Iceberg, Kafka.
- Compensation
- Not disclosed
- Location
- Malaysia
- Work setup
- Not specified
- Seniority
- Not specified
- Employment
- Contract
- Experience
- 3+ years
- Category
- Data engineering
- Posted
Skills
Must have
- 3-5 years of experience in data engineering, ETL and data warehousing
- Bachelor's or Master's degree in Engineering, Computer Science, IT or related technical field
- Hands-on experience with NiFi, Airflow, Apache Iceberg and Kafka
- Proficiency in Python, PowerShell, Bash and YAML
- Solid Linux and networking foundation
- Strong understanding of Data Warehousing, Data Lake modeling, SCD data models and the ELK stack
Nice to have
- Exposure to AWS, Azure or GCP
- Exposure to Docker, Kubernetes
- Exposure to MongoDB, HBase or PostgreSQL
- Data Engineering certification
Full description
Tentacle Tech is an IT and software development consulting company serving some of Malaysia's largest enterprises. We deliver technology solutions, software development services, and IT consulting across multiple industries. We are looking for a talented professional to join our growing team.
The Role
We are seeking a Data Engineer to join us on a contract basis, working onsite on a data-driven project in the finance sector. You will take ownership of data mapping, design, development, testing and implementation for Enterprise Data Lake pipelines across the Bronze, Silver and Gold layers, delivering high-quality data products on time. If you enjoy building robust, automated data workflows and working with modern streaming and orchestration tools, this is a great opportunity to make a real impact.
What You'll Do
- Design, build and maintain end-to-end ETL/ELT pipelines, from sourcing to final data product delivery, for structured and unstructured data
- Design warehouse schemas and analytical models for implementation
- Clean, preprocess and transform data to create and load data products, while providing recommendations that improve productivity and business performance
- Integrate data from multiple sources into centralised platforms for analytics and reporting, including API integration with internal and external applications and Kafka for real-time streaming
- Build frameworks and programs for data validation, monitoring and error handling
- Prepare documentation covering data mapping, technical specifications, SIT plans and test cases
What We're Looking For
- 3 to 5 years of experience in data engineering, ETL and data warehousing
- Bachelor's or Master's degree in Engineering, Computer Science, IT or a related technical field
- Hands-on experience with NiFi, Airflow, Apache Iceberg and Kafka
- Proficiency in Python, PowerShell, Bash and YAML, with a solid Linux and networking foundation
- Strong understanding of Data Warehousing, Data Lake modeling, SCD data models and the ELK stack
- Advantageous: exposure to AWS, Azure or GCP, Docker, Kubernetes, MongoDB, HBase or PostgreSQL, and a Data Engineering certification
What We Offer
- Opportunity to work on a large-scale Enterprise Data Lake project in the finance sector
- Training and upskilling opportunities to keep your skills current
- Mentorship and career progression within a growing consulting company
- A supportive, collaborative team environment
Tentacle Technologies was founded in November 2001 with humble beginnings. Headquartered in Malaysia, today we operate in countries like Singapore, India and the Philippines. Our core services are Business Consulting, Software Development, Quality Assurance & IT solutions in Digital Transformation. We are techies with the right techniques to revamp businesses!