Novartis
Assoc Director- Platform & Data Engineer
A hands-on leadership role at Novartis in Hyderabad designing and building enterprise data platforms and AI-ready foundations — scalable batch/streaming/CDC pipelines, integrations, and RAG/agentic AI capabilities using AWS, Databricks, Snowflake, Spark, and Python — with strong governance, security, and automation.
- Compensation
- Not disclosed
- Location
- Hyderabad (Office)
- Work setup
- Not specified
- Seniority
- Not specified
- Employment
- Not specified
- Experience
- Not specified
- Category
- Data engineering
- Posted
Skills
Must have
- Bachelor's or Master's degree in Computer Science, Software Engineering, Data Engineering, IT, or related discipline (or equivalent practical experience)
- 12+ years of experience in software and/or data engineering with delivery of large-scale production data platforms
- Advanced hands-on expertise in Python, SQL, data modeling, ETL/ELT, CDC, batch, streaming, and event-driven architectures
- Strong experience with Apache Spark (PySpark), distributed data processing, performance tuning, and troubleshooting
- Proven hands-on experience with Databricks, Delta Lake, and Snowflake in enterprise-scale environments
- Deep knowledge of AWS cloud services, cloud-native data architectures, security, observability, Infrastructure as Code, and automation
- Experience implementing CI/CD pipelines, automated testing, monitoring, governance, data quality, security controls, and operational support
- Practical exposure to Generative AI technologies including RAG architectures, embeddings, vector search, LLM integration, context engineering, and agentic workflows
- Strong understanding of enterprise integration patterns, APIs, SaaS integrations, metadata management, lineage, data governance, and access control frameworks
- Excellent stakeholder management, communication, mentoring, problem-solving, and delivery leadership skills with a highly hands-on engineering mindset
Full description
Job Description Summary
This role leads the design and delivery of enterprise data platforms and AI-ready solutions using AWS, Databricks, Snowflake, Spark, and Python. It is a highly hands-on technical position focused on building scalable data pipelines, integrations, data products, RAG and agentic AI capabilities, while ensuring strong governance, security, reliability, automation, and performance
Job DescriptionMajor Accountabilities
- Design, build, and optimize scalable batch, streaming, CDC, and event-driven data pipelines using modern data engineering best practices and production-grade Python and SQL.
- Architect secure, resilient, high-performance data platforms and translate business and technical requirements into scalable cloud-based solutions.
- Lead hands-on development and optimization of Databricks, Apache Spark, Delta Lake, and Lakehouse solutions, ensuring reliability, performance, and cost efficiency.
- Design and implement Snowflake data engineering solutions, including ingestion, transformation, governance, orchestration, and secure data consumption patterns.
- Engineer enterprise data platforms on AWS, leveraging cloud-native services to deliver secure, automated, observable, and highly available data solutions.
- Develop reusable integration frameworks, APIs, CDC processes, and connectivity patterns across enterprise applications, SaaS platforms, governance tools, ERP, and CRM systems.
- Build AI-ready data foundations, including RAG pipelines, vector retrieval, context engineering, and agentic workflows, enabling enterprise AI and GenAI use cases.
- Drive engineering excellence through CI/CD, Infrastructure as Code, security-by-design, observability, governance, mentoring, and end-to-end ownership of data products and platform capabilities.
Minimum Requirements
- Bachelor's or Master's degree in Computer Science, Software Engineering, Data Engineering, Information Technology, or a related discipline, or equivalent practical experience.
- 12+ years of experience in software engineering and/or data engineering with proven delivery of large-scale production data platforms.
- Advanced hands-on expertise in Python, SQL, data modeling, ETL/ELT, CDC, batch, streaming, and event-driven architectures.
- Strong experience with Apache Spark (PySpark), distributed data processing, performance tuning, and troubleshooting.
- Proven hands-on experience with Databricks, Delta Lake, and Snowflake in enterprise-scale environments.
- Deep knowledge of AWS cloud services, cloud-native data architectures, security, observability, Infrastructure as Code, and automation.