Senior Data Engineer (Remote)
Senior Data Engineer who designs, builds, and runs scalable ETL/ELT pipelines and AWS data workflows (S3, Athena, EC2, EMR) using SQL, Hive, Python, and Spark, orchestrated with Airflow. Also leads and mentors a small team of 2-3 engineers, owning solutions from analysis through deployment and production support.
- Compensation
- Not disclosed
- Location
- Makati City, Metro Manila
- Work setup
- Remote
- Seniority
- Senior
- Employment
- Not specified
- Experience
- 5+ years
- Category
- Data engineering
- Posted
Skills
Must have
- 5+ years of professional experience in data engineering with ownership of production data pipelines and large-scale data processing
- Strong SQL expertise: complex joins, window functions, transformations, query tuning, troubleshooting
- Strong working knowledge of Hive and the Hadoop ecosystem, including partitioning, storage formats, and distributed data processing
- Proficiency in Python for data processing, automation, integration, and maintainable production code
- Hands-on experience with Amazon S3, Athena, EC2, and EMR
- Strong experience with Apache Airflow or another ETL/workflow orchestration tool (Airflow preferred)
- Experience implementing data quality checks, monitoring, error handling, and operational support processes
- Ability to translate business requirements into technical designs and deliver with minimal supervision
- Strong documentation and communication skills for business and technical stakeholders
- Working experience with Jira and Confluence
Full description
About the role
You will lead and manage the Software quality assurance function and help maintain and improve the quality of software products, development practices and client implementations. You will be responsible for ensuring products are performing and scalable. This means you will create well-structured test plans and test cases and lead the testing activities for your team.
We are looking for an experienced Senior Data Engineer to design, build, and manage scalable data solutions while providing technical guidance to a small engineering team. This role combines hands-on development with responsibility for solution design, production reliability, delivery planning, and team mentoring. The candidate should work independently, translate business requirements into technical solutions, and take ownership of projects from initial analysis through deployment and ongoing support.
Key responsibilities
- Own data pipeline development: Design, develop, and maintain reliable ETL/ELT pipelines for ingesting, transforming, and delivering large-scale datasets across multiple platforms and sources.
- Lead technical design: Define processing logic, data models, integration approaches, and workflow dependencies, considering scalability, maintainability, security, and cost.
- Develop and optimize data processing: Write efficient SQL, Hive queries, and Python code. Apply distributed processing frameworks such as Spark where appropriate.
- Build and manage AWS workflows: Develop and support data solutions using Amazon S3, Athena, EC2, and EMR, including monitoring and resource optimization.
- Manage workflow orchestration: Build and maintain automated workflows using Apache Airflow or comparable orchestration tools, including scheduling, dependencies, retries, alerting, and recovery.
- Ensure production reliability: Monitor pipeline performance and data freshness, investigate failures, perform root cause analysis, and implement improvements that prevent recurring issues.
- Maintain data quality and governance: Establish validation checks, reconciliation processes, lineage documentation, and appropriate access controls throughout the data lifecycle.
- Lead and mentor a small team: Guide 2-3 engineers, break requirements into actionable tasks, review code and designs, support troubleshooting, and track delivery commitments.
- Collaborate with stakeholders: Work closely with analytics, product, engineering, and data science teams to clarify requirements, agree on priorities, communicate progress, and manage dependencies.
- Maintain technical documentation: Create and update solution designs, data flow diagrams, data dictionaries, operating procedures, and support runbooks using Confluence or similar platforms.