Data Engineering Specialist - ETL Pipelines

Job Title : Data Engineering Specialist

Experience : 8+ Years

Location : Pune

Role Summary :

We are looking for an experienced Data Engineering Specialist to design, develop, and optimize scalable data ingestion and ETL pipelines across the Hadoop ecosystem. The ideal candidate should have strong hands-on experience with Apache NiFi, Spark/PySpark, Hadoop/HDFS, Apache Airflow, Cloudera Data Engineering (CDE), Hive, SQL, and Python.

Mandatory Skills :

• Apache NiFi

• Spark / PySpark

• Hadoop / HDFS

• Apache Airflow

• Cloudera Data Engineering (CDE)

• Hive & SQL

• Python

Key Responsibilities :

• Design, develop, and maintain scalable ETL and data ingestion pipelines.

• Build and optimize data processing solutions using Spark, PySpark, and NiFi.

• Develop and manage Airflow workflows for scheduling and orchestration.

• Work with Hadoop, HDFS, Hive, and CDE for enterprise data processing.

• Write efficient Hive SQL and Python code for data transformation and automation.

• Monitor, troubleshoot, and optimize production data pipelines.

• Collaborate with cross-functional teams to deliver reliable data solutions.

Required Qualifications :

• 8+ years of experience in Data Engineering and Big Data technologies.

• Strong expertise in NiFi, Spark/PySpark, Hive SQL, Python, and Airflow.

• Good understanding of Hadoop, HDFS, distributed data processing, and ETL best practices.

• Hands-on experience with Cloudera Data Engineering (CDE) or similar enterprise data platforms.

• Experience supporting and optimizing production data pipelines.

• Strong analytical, problem-solving, and communication skills.