Senior Infrastructure Engineer, Data Compute Platform
Senior engineer on Grab's Data Compute Platform infrastructure: designing and operating the multi-tenant Kubernetes (EKS) and AWS foundations - Terraform IaC, GitLab CI/CD, Go-based Kubernetes operators - that run Spark, Ray, Trino, Starrocks and Airflow, with ownership of reliability, security and cost efficiency. Onsite in Petaling Jaya.
- Compensation
- Not disclosed
- Location
- Selangor, Malaysia
- Work setup
- Not specified
- Seniority
- Senior
- Employment
- Not specified
- Experience
- Not specified
- Category
- Devops
- Posted
Skills
Must have
- Design, build, and operate a multi-tenant Kubernetes (EKS) platform including cluster lifecycle, node provisioning, autoscaling, scheduling and resource isolation
- Build Kubernetes operators and custom resources with kubebuilder / controller-runtime in Go
- Lead infrastructure-as-code with Terraform and CI/CD with GitLab for the AWS estate (EKS, S3, IAM, RDS, VPC, networking)
- Design and implement identity, access and security model across AWS IAM, Kubernetes RBAC and service identities
- Build observability, alerting, capacity planning and incident tooling; contribute to SRE practice (SLOs, runbooks, on-call, post-incident reviews)
- Own compute cost efficiency: spot and right-sizing, bin-packing, idle reclamation, cost attribution
- Drive architectural improvements and migrations with design, phased rollout and rollback plans
- Take ownership of well-scoped infrastructure projects from design through rollout and operation as a senior engineer
- Support other engineers through code review and knowledge sharing
- Work onsite at the Petaling Jaya office
Nice to have
- Familiarity with distributed data compute engines: Apache Spark, Ray, Trino, Starrocks, Airflow, Michelangelo
- Knowledge of modern open-source table formats such as Apache Iceberg and Delta
Full description
The Data Compute Platform team is an important contributor to Grab's data ecosystem, allowing growth through democratization of data at scale. We build and operate Grab's data infrastructure and efficient platform that supports internal data processes and company-wide data lake access. Our tech stack uses industry-leading distributed compute engines like Apache Spark, Ray, Trino, and Starrocks, orchestrated by Airflow and Michelangelo and backed by AWS S3. Our evolving Data Lake storage architecture uses modern open-source formats like Apache Iceberg and Delta in addition to traditional Apache Hive Parquet tables.
Underneath these engines sits a large, multi-tenant Kubernetes and AWS Infrastructure that we own end to end. This infrastructure includes the clusters, the operators, the autoscaling, the networking, the identity and access model, the observability, and the cost controls. These components work together to keep the platform fast, reliable, and affordable for thousands of pipelines and queries every day.
Get to Know the Role
You will be an important contributor to the infrastructure layer of the Data Compute Platform. This layer consists of the Kubernetes, AWS, and infrastructure-as-code foundations. Spark, Ray, Trino, Starrocks, Airflow, and Michelangelo run on these foundations. You will design how compute is provisioned, scaled, secured, observed and paid for, and you will drive the reliability and cost-efficiency of the platform as it grows. As a senior engineer, you will take ownership of well-scoped infrastructure projects from design through rollout and operation. You will also contribute to the team's engineering and SRE standards. Additionally, you will support other engineers through code review and knowledge sharing. You will also explore new developments in the cloud-native and data infrastructure space and integrate them into our ecosystem to the benefit of the data community at Grab.
You will report to our Data Engineering Manager II, and you will based onsite in our Petaling Jaya office.
The Critical Tasks You Will Perform
- You will design, build, and operate the multi-tenant Kubernetes (EKS) platform. This platform runs Grab's workloads, including Spark, Ray, Trino, Starrocks, Airflow, and Michelangelo. Additionally, your responsibilities will include cluster lifecycle, node provisioning, and autoscaling, and scheduling and resource isolation.
- You will build Kubernetes operators and custom resources (kubebuilder / controller-runtime, Go) that automate the provisioning and lifecycle of compute engines and their tenants.
- You will lead the infrastructure-as-code (Terraform) and CI/CD (GitLab) that provision and change our AWS estate. This estate includes EKS, S3, IAM, RDS, VPC, and networking. You will drive it towards safe, reviewable, automated change.