Technical Lead Data Engineer - Databricks

Designation : Technical Lead Data Engineer (Databricks)

Location : Hyderabad (Hybrid)

Experience : 12+ years

Role Overview :

We are seeking a Senior Technical Leader to provide hands-on technical leadership for clinical data engineering, data platform development, and analytics delivery. This role will lead the design and execution of scalable data pipelines, lakehouse architecture, cloud services integration, and delivery governance across active clinical data initiatives.

The ideal candidate will have strong experience with Databricks Lakehouse, Delta Lake, Unity Catalog, Medallion architecture, SQL Warehouses, AWS services, data engineering delivery, agile execution, and stakeholder management in a life sciences or regulated data environment.

Key Responsibilities :

• Serve as the technical lead for clinical data engineering and platform delivery.

• Own architecture and implementation across Databricks Lakehouse, Delta Lake, Unity Catalog, Medallion architecture, and SQL Warehouses.

• Lead end-to-end delivery of data pipelines, curated datasets, data products, and analytics-ready data assets.

• Manage and guide a team of 5 to 10+ data engineers across onsite and offshore delivery.

• Provide technical direction on pipeline design, data modeling, performance optimization, security, scalability, and production readiness.

• Work with AWS services such as S3, EMR/Athena, Secrets Manager, IAM, and related cloud platform components.

• Drive sprint planning, backlog management, technical design reviews, code reviews, release planning, and delivery governance.

• Partner with Clinical Operations, Biostatistics, Data Management, IT, data managers, and business stakeholders to translate requirements into technical solutions.

• Coordinate cross-functional delivery across data engineering, analytics, AI/ML, platform, QA, and business teams.

• Ensure engineering outputs meet quality, security, compliance, performance, and maintainability expectations.

• Support issue resolution across data ingestion, transformation, pipeline failures, data quality defects, and platform dependencies.

• Provide technical oversight for automation, AI/ML-enabled workflows, and human-in-the-loop data operations where applicable.

Required Qualifications :

• 12+ years of experience in data engineering, data architecture, analytics engineering, or cloud data platform delivery.

• Strong hands-on experience with Databricks Lakehouse, including :