Senior Data Engineer (Apache NiFi, Airflow & Kafka) | Tentacle Tech | Selangor, Malaysia
Senior Data Engineer on a renewable contract in Selangor, Malaysia, owning enterprise data lake architecture, ETL/ELT pipeline optimization, and real-time streaming analytics for a banking client. Core stack: Apache NiFi, Airflow, Iceberg, and Kafka, with Python/PowerShell/Bash scripting and ELK-based monitoring.
- Compensation
- Not disclosed
- Location
- Selangor, Malaysia
- Work setup
- Not specified
- Seniority
- Senior
- Employment
- Contract
- Experience
- 5+ years
- Category
- Data engineering
- Posted
Skills
Must have
- Bachelor's or Master's degree in Computer Science, IT, Software Engineering, Data Analytics, or related discipline
- 5-7 years of hands-on experience in data engineering, ETL architecture, and big data systems
- Operational expertise in Apache NiFi, Apache Airflow, Apache Iceberg, and Apache Kafka
- Proficiency in Python, PowerShell, Bash, and YAML scripting
- Linux systems administration, enterprise networking, and distributed data store optimization
- Data Lake modeling, dimensional warehousing, and complex SCD data models
- Analytical problem-solving, architectural vision, and stakeholder communication skills
- Malaysian citizenship or current legal residency in Malaysia
Nice to have
- Certifications such as Certified Data Professional, AWS Certified Data Analytics, or Google Cloud Professional Data Engineer
- Familiarity with cloud-native data platforms, Docker, Kubernetes, and CI/CD pipelines
- Prior experience in multinational banking institutions, central banks, or major fintech enterprises
- Experience with agile tracking platforms (Jira, Confluence)
- Participation in open-source data engineering communities, conference speaking, or advisory boards
Full description
Position Summary
Driving executive technical leadership, enterprise data platform architecture, and real-time streaming analytics for a premier banking project in Selangor, Malaysia, Tentacle Tech is actively seeking an accomplished Senior Data Engineer for a renewable contract professional engagement. In this vital senior technical role, you will take full operational, architectural, and strategic ownership of enterprise data lake modeling, complex ETL/ELT pipeline optimization, and scalable data warehousing platforms utilizing Apache NiFi, Airflow, Iceberg, and Kafka. Bringing 5 to 7 years of progressive professional experience in advanced data engineering, you will empower top-tier banking institutions to achieve uncompromised data reliability, sub-second transaction streaming, and resilient enterprise analytics. This prestigious career opportunity offers top-tier remuneration, high technical visibility, and professional growth within a premier global IT staffing and digital consulting enterprise.
Detailed Job Description
As a Senior Data Engineer supporting Tentacle Tech on a renewable contract basis in Selangor, Malaysia, you will take full operational, tactical, and technical ownership of enterprise data architecture design, advanced ETL pipeline engineering, ELK stack log monitoring, and team technical mentoring. Your day-to-day responsibilities include architecting distributed data pipelines with Apache NiFi, Airflow, and Kafka, optimizing large-scale datasets on Apache Iceberg, scripting automation workflows in Python, PowerShell, Bash, and YAML, guiding junior engineers, collaborating with banking stakeholders, and interacting directly with technical recruiters like Saf Suhaimi.
Key Responsibilities
- Architect, design, develop, and govern enterprise-scale ETL/ELT data pipelines, streaming architectures, and banking data platforms.
- Lead the implementation of distributed data orchestration and messaging frameworks utilizing Apache NiFi, Apache Airflow, Apache Iceberg, and Kafka.
- Author advanced automation scripts, data ingestion frameworks, and transformation modules using Python, PowerShell, Bash, and YAML.
- Oversee Linux system performance tuning, network optimization, and high-availability configuration for mission-critical data platforms.
- Design and maintain sophisticated Data Lake architectures, enterprise data warehouses, and Slowing Changing Dimension (SCD) models.
- Implement and manage centralized log analysis and real-time monitoring solutions utilizing the ELK Stack (Elasticsearch, Logstash, Kibana).
- Provide expert technical escalation support, conduct rigorous code reviews, and mentor junior data engineering team members.
- Maintain meticulous architectural documentation, data governance policies, disaster recovery blueprints, and deployment runbooks.