svan global consultancy
Data Engineer
- Compensation
- ₹2.7M–₹4.8M / yr
- Location
- Pune
- Work setup
- Not specified
- Seniority
- Not specified
- Employment
- Not specified
- Experience
- 9+ years
- Category
- Data engineering
- Posted
Skills
Must have
- - 9 - 16 years of proven experience in software engineering, with a focus on data infrastructure and engineering.
- - Expertise in object-oriented programming, design patterns, algorithm optimization, and problem-solving from first principles.
- - Strong experience working with unstructured log data, real-time data processing, distributed computing frameworks, and streaming data frameworks.
- - Proficient in Python; experience with Java or Scala is a plus.
- - Deep expertise in data engineering technologies, including ETL/ELT pipelines, data integration, and operational monitoring.
- - Experience drafting proofs of concept (POCs) and collaborating cross-functionally to develop prototypes and production-ready solutions.
- - Proficiency with AWS services (Lake, Glue, EMR) and Airflow Key Responsibilities :
- - Design, build, and maintain event-driven distributed data infrastructure and data applications.
- - Develop and optimize robust ETL/ELT pipelines to support batch and real-time data workflows.
- - Integrate and normalize diverse data sources, ensuring high standards of data quality, accuracy, and consistency.
- - Lead the design and implementation of real-time data processing systems for analytics, operational intelligence, and reporting use cases.
- - Collaborate cross-functionally with data scientists, ML engineers, and product teams to design end-to-end data solutions.
- - Establish best practices for data engineering, including testing, monitoring, deployment automation, and observability.
- - Mentor engineers and contribute to architectural decisions, technical strategy, and roadmap planning for the data platform.
Full description
**Description : Job Summary :
As a Senior Staff Software/Data Engineer, you will collaborate closely with the data engineering team and work on developing and maintaining distributed data infrastructure,ETL/ELT pipelines, dataapplications, and integration solutions. You will get a chance to work with brilliant minds in the industry, work on complex use cases enabling steep learning curve, and build products from scratch while modernizing various applications.
Qualifications :
- 9 - 16 years of proven experience in software engineering, with a focus on data infrastructure and engineering.
- Expertise in object-oriented programming, design patterns, algorithm optimization, and problem-solving from first principles.
- Strong experience working with unstructured log data, real-time data processing, distributed computing frameworks, and streaming data frameworks.
- Proficient in Python; experience with Java or Scala is a plus.
- Deep expertise in data engineering technologies, including ETL/ELT pipelines, data integration, and operational monitoring.
- Experience drafting proofs of concept (POCs) and collaborating cross-functionally to develop prototypes and production-ready solutions.
- Proficiency with AWS services (Lake, Glue, EMR) and AirflowKey Responsibilities :
- Design, build, and maintain event-driven distributed data infrastructure and data applications.
- Develop and optimize robust ETL/ELT pipelines to support batch and real-time data workflows.
- Integrate and normalize diverse data sources, ensuring high standards of data quality, accuracy, and consistency.
- Lead the design and implementation of real-time data processing systems for analytics, operational intelligence, and reporting use cases.
- Collaborate cross-functionally with data scientists, ML engineers, and product teams to design end-to-end data solutions.
- Establish best practices for data engineering, including testing, monitoring, deployment automation, and observability.
- Mentor engineers and contribute to architectural decisions, technical strategy, and roadmap planning for the data platform.