Senior Data Engineer - Python/PySpark

Role Overview :

As a Senior Data Engineer, you will serve as a technical cornerstone in building and scaling robust data pipelines that power our core analytical platforms.

You will work closely with cross-functional teams, including Data Scientists, Product Managers, and Business Analysts, to transform complex raw data into actionable business intelligence.

By architecting high-performance processing frameworks, you will directly influence the speed and accuracy of data-driven decision-making, ensuring that our stakeholders have reliable, real-time insights to drive strategic business outcomes across global markets.

Key Responsibilities :

• Design and implement scalable data processing pipelines using PySpark to handle large-scale datasets, ensuring optimal performance and reliability for downstream analytical applications.

• Develop complex SQL queries and stored procedures to facilitate efficient data extraction, transformation, and loading (ETL) processes that support critical business reporting needs.

• Collaborate with engineering teams to optimize existing data architectures, reducing latency and improving data quality to enhance the overall user experience of our internal data products.

• Mentor junior developers by conducting code reviews and promoting best practices in Python development to maintain high engineering standards across the department.

• Partner with stakeholders to translate ambiguous business requirements into technical specifications, ensuring that data solutions align perfectly with long-term organizational goals.

Required Skillset :

• Demonstrated expertise in building distributed data systems using PySpark and Python, with a deep understanding of performance tuning and memory management in big data environments.

• Advanced proficiency in SQL for complex data manipulation and database schema design, coupled with the ability to troubleshoot performance bottlenecks in high-volume production databases.

• Proven ability to communicate complex technical concepts to non-technical stakeholders, fostering a collaborative environment that bridges the gap between engineering and business strategy.

• Strong analytical mindset with a track record of solving intricate data problems independently while maintaining a high level of code quality and documentation.

• Educational background in Computer Science, Engineering, or a related quantitative field, providing a solid foundation for technical problem-solving.

• Adaptability to thrive in a hybrid work environment across our Bangalore, Chennai, Pune, or Hyderabad offices, demonstrating the ability to manage projects effectively in a distributed team setting.

• Minimum of 6 to 8 years of hands-on experience in data engineering roles, showcasing a consistent trajectory of technical growth and project ownership.