Infosys Consulting
Senior Data Engineer- Malaysia
A senior data engineer designs, builds, and maintains AWS-based data pipelines and ETL workflows, deploying changes directly to production and supporting a migration to Databricks. Core tech includes AWS (Glue, S3, Lambda, Step Functions), PySpark, and advanced SQL.
- Compensation
- Not disclosed
- Location
- Kuala Lumpur, Malaysia
- Work setup
- Not specified
- Seniority
- Senior
- Employment
- Not specified
- Experience
- 7+ years
- Category
- Data engineering
- Posted
Skills
Must have
- 7 to 10 years of experience and above
- Expert-level fluency in AWS data engineering services (Glue, S3, Lambda, Step Functions, etc.)
- Strong proficiency in PySpark for distributed data processing
- Advanced SQL skills for querying and optimizing data operations
- Experience with data lake architectures, big data technologies, and data pipeline orchestration
- Familiarity with CI/CD practices for data engineering
- Comfortable pushing changes live without formal review, ensuring quality through self-validation
- Strong problem-solving skills and attention to detail
- Ability to work independently and manage tasks in a siloed environment
- Currently based in Kuala Lumpur, Malaysia
Nice to have
- Databricks experience
- AWS Certification (e.g., AWS Certified Data Analytics – Specialty or Solutions Architect)
Full description
- Design, develop, and maintain data pipelines and ETL workflows using AWS services.
- Implement orchestration and automation for data workflows.
- Work with large datasets to ensure data integrity, scalability, and performance.
- Collaborate with stakeholders to understand data requirements and deliver solutions.
- Deploy changes directly to production environments with confidence and accountability.
- Support migration efforts to Databricks and optimize workflows for performance
Qualifications
- 7 to 10 years of experience and above
- Experience with data lake architectures, big data technologies, and data pipeline orchestration.
- Familiarity with CI/CD practices for data engineering.
- AWS Certification (e.g., AWS Certified Data Analytics – Specialty or Solutions Architect) is a plus.
- Strong problem-solving skills and attention to detail.
Key Skills and Requirements
- Expert-level fluency in AWS services relevant to data engineering, including Glue, S3, Lambda, Step Functions, and other related services.
- Strong proficiency in PySpark for distributed data processing.
- Advanced SQL skills for querying and optimizing data operations.
- Comfortable with pushing changes live without formal review, ensuring quality through self-validation.
- Databricks experience is a plus.
- Ability to work independently and manage tasks in a siloed environment.
- Currently based in Kuala Lumpur, Malaysia