Mytech
Azure Data Engineer
Senior (6-11 years) Azure Data Engineer role in Pune building big-data pipelines: Spark and Azure Databricks, the Hadoop ecosystem, stream processing with Storm/Spark Streaming, Kafka messaging, plus SQL and NoSQL data stores. Also expects ETL design, Spark performance tuning, and team leadership on Agile projects.
- Compensation
- Not disclosed
- Location
- Pune
- Work setup
- Hybrid
- Seniority
- Not specified
- Employment
- Full time
- Experience
- 6+ years
- Category
- Data engineering
- Posted
Skills
Must have
- 6+ years of experience in Big Data & data-related technologies
- Expert-level understanding of distributed computing principles
- Expert-level knowledge and experience with Apache Spark
- Hands-on programming with Python
- Proficiency with Hadoop v2, MapReduce, HDFS, Sqoop
- Experience building stream-processing systems (Apache Storm or Spark Streaming)
- Experience with messaging systems such as Kafka or RabbitMQ
- Understanding of Big Data querying tools such as Hive and Impala
- Experience integrating data from multiple sources (RDBMS like SQL Server, Oracle, ERP, files)
- Good understanding of SQL queries, joins, stored procedures, relational schemas
- Experience with NoSQL databases such as HBase, Cassandra, MongoDB
- Knowledge of ETL techniques and frameworks
- Performance tuning of Spark jobs
- Experience with native cloud data services, Azure Databricks
- Ability to lead a team efficiently
- Experience designing and implementing Big Data solutions
- Practitioner of Agile methodology
Full description
- 6+ Years of in Big Data & Data related technology experience
- Expert level understanding of distributed computing principles
- Expert level knowledge and experience in Apache Spark
- Hands on programming with Python
- Proficiency with Hadoop v2, Map Reduce, HDFS, Sqoop
- Experience with building stream-processing systems, using technologies such as Apache Storm or Spark-Streaming
- Experience with messaging systems, such as Kafka or RabbitMQ
- Good understanding of Big Data querying tools, such as Hive, and Impala
- Experience with integration of data from multiple data sources such as RDBMS (SQL Server, Oracle), ERP, Files
- Good understanding of SQL queries, joins, stored procedures, relational schemas
- Experience with NoSQL databases, such as HBase, Cassandra, MongoDB
- Knowledge of ETL techniques and frameworks
- Performance tuning of Spark Jobs
- Experience with native Cloud data services Azure databricks
- Ability to lead a team efficiently
- Experience with designing and implementing Big data solutions
- Practitioner of AGILE methodology
WE OFFER
- Opportunity to work on technical challenges that may impact across geographies
- Vast opportunities for self-development: online university, knowledge sharing opportunities globally, learning opportunities through external certifications
- Opportunity to share your ideas on international platforms
- Sponsored Tech Talks & Hackathons
- Possibility to relocate to any EPAM office for short and long-term projects
- Focused individual development
- Benefit package:
- Health benefits, Medical Benefits
- Retirement benefits
- Paid time off
- Flexible benefits
- Forums to explore beyond work passion (CSR, photography, painting, sports, etc.
Thanks!