EY - GDS Consulting - AI and DATA - Databricks Kafka Data Engineer - Senior
- Compensation
- Not disclosed
- Location
- Not listed
- Work setup
- Not specified
- Seniority
- Senior
- Employment
- Not specified
- Experience
- 5+ years
- Category
- Data engineering
- Posted
Skills
Must have
- Databricks (Expert Level or Databricks Champion)
- Apache Spark (PySpark / Spark SQL)
- Delta Lake
- Kafka Streaming
- Semi-structured data processing like json, xmls
- AWS Cloud Services
- ETL/ELT Design and Development
- Data Lakehouse Architecture
- Data Modeling and Dimensional Modeling
- Data Warehousing Concepts
- SQL and Performance Optimization
- Real-Time Data Processing
- Data Pipeline Orchestration
Nice to have
- Experience designing enterprise-scale Data Lakehouse solutions.
- Hands-on implementation of Raw, Silver, and Gold architectural patterns.
- Experience building metadata-driven ingestion and transformation frameworks.
- Exposure to Insurance, Banking, or Financial Services domain data platforms.
- Experience supporting analytics, reporting, and AI/ML workloads on Databricks.
- Experience leading distributed teams and managing technical delivery.
Full description
At EY, you’ll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And we’re counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all.
EY Hiring RequisitionSenior Consultant - Databricks Kafka Data EngineerExperience: 5-7 Years
Job Summary
EY is seeking a highly skilled Data Engineering and Data Architecture professional with strong expertise in Databricks, Kafka, AWS, and modern Data Lakehouse architecture.
The ideal candidate will have hands-on experience designing and implementing scalable data platforms, building enterprise-grade data pipelines, and delivering analytics-ready data products.
The candidate should possess deep technical knowledge across the data engineering lifecycle, from ingestion and transformation to modelling and consumption, while driving delivery through engineering best practices, automation, and stakeholder collaboration.
Key Responsibilities
- Design and implement modern Data Lakehouse platforms using Databricks on AWS
- Build ETL/ELT pipelines
- Implement Kafka streaming solutions
- Design Fact and Dimension models
- Implement CI/CD using Azure DevOps and GitHub
- Provide technical leadership
Data Architecture & Platform Engineering
- Design and implement modern cloud-based Data Lakehouse platforms using Databricks on AWS.
- Define and implement scalable Raw, Silver, and Gold layer architectures.
- Architect end-to-end data solutions supporting reporting, BI, advanced analytics, and AI use cases.
- Establish data platform standards, governance controls, and engineering best practices.
Data Engineering & ETL
- Design, develop, and optimize large-scale ETL/ELT pipelines.
- Build high-performance Spark-based data processing solutions.
- Develop reusable and metadata-driven ingestion and transformation frameworks.
- Ensure data quality, reliability, scalability, and operational efficiency across data pipelines.
- Implement data validation, reconciliation, and monitoring mechanisms.
- Handling semi-structured data formats- json, xml
Streaming & Real-Time Data Processing
- Design and implement real-time data ingestion frameworks using Kafka.