aryan solutions pte. ltd.
Data Lead Data Engineering
A lead data engineering role in Singapore (7-12 years experience, permanent) focused on diagnosing, optimizing, and stabilizing enterprise ETL and batch-processing platforms. Day-to-day involves tuning Azure Data Factory, Databricks (PySpark/Spark SQL), SSIS, and SQL Server pipelines, managing Control-M/Autosys batch scheduling, resolving failures, and improving monitoring.
- Compensation
- Not disclosed
- Location
- Singapore, Singapore
- Work setup
- Not specified
- Seniority
- Lead
- Employment
- Full time
- Experience
- 7+ years
- Category
- Data engineering
- Posted
Skills
Must have
- 7-12 years of experience
- Strong expertise in Azure Data Factory (ADF)
- Strong expertise in Databricks (PySpark / Spark SQL)
- Strong expertise in SSIS and SQL Server
- Hands-on experience with batch orchestration tools (Control-M / Autosys)
- Hands-on experience with ETL/ELT pipeline design and optimization
- Strong experience in SQL development and performance tuning
- Troubleshooting and debugging complex data pipelines
- Exposure to large-scale data processing and batch systems
Nice to have
- Familiarity with monitoring and alerting tools (Azure Monitor, Log Analytics)
Full description
Experience:7-12 YearsLocation:SingaporeFulltime Permanent Role
We are looking for a Data Lead - Data Engineeringto drive technical diagnostics, optimisation, and stabilisation of enterprise data platforms supporting large-scale ETL and batch processing systems.
Key Responsibilities
- Strong expertise in: Azure Data Factory (ADF),Databricks (PySpark / Spark SQL preferred),SSIS and SQL Server
- Hands-on experience with: Batch orchestration tools (Control-M / Autosys),ETL/ELT pipeline design and optimization
- Strong experience in: SQL development and performance tuning, Troubleshooting and debugging complex data pipelines
- Familiarity with: Monitoring and alerting tools (Azure Monitor, Log Analytics)
- Exposure to large-scale data processing and batch system
- Review and optimize ADF pipelines, SSIS packages, and Databricks workflowsfor performance, scalability, and
- Analyze batch scheduling and orchestration frameworks(Control-M / Autosys), including dependencies, triggers, and SLA adherence
- Investigate and resolve pipeline failures, job delays, and runtime issues, ensuring faster recovery and minimal business
- Evaluate source-to-target integration flows, SQL logic, and transformation layersfor efficiency and correctness
- Define and improve error handling, retry mechanisms, restart ability, and reprocessing strategiesacross ETL and batch
- Optimize runtime performance of data pipelines, including query tuning and batch execution improvements
- Establish and enhance monitoring and alerting frameworksusing tools like Azure Monitor, Control-M dashboards, and custom logging solutions
- Identify automation opportunitiesto reduce manual intervention and improve operational efficiency
- Ensure data quality, consistency, and reliabilityacross ETL processes and reporting outputs
- Support modernization initiatives, including migration from legacy ETL (SSIS) to cloud-based platforms like Databricks
- Collaborate with data engineers, architects, business, and support teamsto drive improvements and ensure stable production systems