Lead Data Engineer - AWS/Azure Databricks

Lead Data EngineerExperience : 5+ Years

Duration : 12 Months

Work Mode : Hybrid

Role Overview :

We are looking for a hands-on Lead Data Engineer with strong technical leadership and expertise in designing and building high-volume, production-grade data platforms. The ideal candidate will have extensive experience with modern cloud data technologies, data lakes, data warehouses, ETL/ELT pipelines, real-time data processing, and data architecture. The role requires someone who can combine strong engineering expertise with technical leadership to architect scalable data solutions and guide a technical team.

Key Responsibilities :

• Lead the design, development, and implementation of scalable data engineering solutions.

• Architect and build high-volume data lakes and data warehouses using modern cloud technologies.

• Design and optimize ETL/ELT pipelines supporting batch and real-time/streaming workloads.

• Implement Change Data Capture (CDC) pipelines and data integration solutions.

• Work with Databricks, Snowflake, AWS, and Azure to develop enterprise-grade data platforms.

• Develop robust data pipelines and integrations using Python and advanced SQL.

• Build and manage workflows using Apache Airflow and Airbyte.

• Design and implement integrations with REST APIs and third-party APIs.

• Establish and maintain Git-based CI/CD pipelines, preferably using GitLab.

• Containerize applications and data workloads using Docker and Kubernetes.

• Define and implement effective data architecture and data modeling strategies.

• Collaborate with engineering, analytics, data science, and business teams to translate requirements into scalable data solutions.

• Provide technical leadership, conduct code reviews, and mentor data engineers.

• Apply data engineering best practices for performance, scalability, security, reliability, and maintainability.

• Contribute to MLOps initiatives and support data infrastructure for machine learning workloads.

• Leverage AI coding assistants to improve engineering productivity, code quality, and development efficiency.

Must-Have Skills & Experience :

• 5+ years of experience in Data Engineering with strong technical leadership capabilities.

• Strong expertise in Databricks, Snowflake, AWS, and/or Azure.

• Hands-on experience building large-scale Data Lakes and Data Warehouses.