Data Engineer - Python

About Avanse :

Avanse Financial Services Limited is a new age, education focused NBFC on a mission to provide seamless and affordable education financing for every deserving Indian Student.

Avanse is on a journey to strengthen its leadership position in Education Lending by creating solutions for Individuals (Education Loans) and Indian Institutions (Education Infrastructure Financing) by providing solutions to every segment of the Indian education sector covering school education to post graduation while also contributing to build the education infrastructure in the country.

Warburg Pincus, a leading global private equity firm holds 59.32 percent of Equity. Kedaara an Indian equity firm holds 17.71 percent stake in Avanse. Mubadala, a fund based at UAE & Avendus hold 10.28 & 1.14 percent stake respectively in Avanse. The International Finance Corporation, an arm of the World Bank holds 11.55 percent equity stake.

Avanse as an equal opportunity employer : We give utmost importance to our cultural pillars which have embedded values aligned to Governance, Transparency, Meritocracy, Inclusivity and Happiness Quotient.

Avanse is committed to workplace diversity and to cultivating, fostering and preserving a culture of inclusion. We believe that workplace Diversity and Inclusion refers to the recognition of employee differences, which if properly managed will enable work to be done more efficiently and effectively. Diversity and Inclusion does not focus exclusively on issues of discrimination but instead concentrates on recognizing and respecting the differences between people.

Dimensions of Diversity include, but are not limited to, gender, race, ethnicity, language, age, religion, socio-economic status, physical ability, thinking style, experience and education. Aligned to its business objective of enabling education through financing for a diverse segment of customers and to its cultural pillar of Inclusivity, Avanse believes that the wide array of perspectives that result from such diversity promotes innovation and business success. Managing diversity makes us more creative, flexible and competitive. It also creates a productive working environment in which everyone feels valued and where their talent is fully utilized thereby making it inclusive.

Passionate individuals who love to accept challenges with a problem-solving attitude will enjoy the work culture at Avanse. Our teams thrive to make the work environment exciting and vibrant and the organization promotes the culture of entrepreneurship and innovative mindset for a holistic growth.

Job Purpose :

This is an intermediate-level position responsible for establishment and implementation of new developments, along with potential enhancements within the Technology & Transformation vertical. The primary objective is to act as a catalyst in Avanse's growth journey, through impactful contribution to Application Development & programming role.

Key Responsibilities :

• Design and develop scalable, batch and real time data pipelines, addressing business requirements.

• Identify the key problems, analyze information and recommend precise solutions to various technical & domain-specific issues.

• Act as an SME to various technical and functional stakeholders with minimum direct supervision.

Key Accountabilities :

• Drive best practice in Data platform management and software development, ensuring high quality in terms of data, architecture, testing & deployment.

• Manage team responsibilities of addressing production issues promptly, if any.

• Work closely with product managers, Business Stakeholders to develop technical roadmaps.

• Build effective relationships with cross-functional teams to swiftly address issues.

Required Experience :

• 3+ years of robust experience in Python programming, demonstrating a strong command of Object-Oriented Programming (OOP) principles and their application in scalable solutions.

• Proven expertise in designing, building, and maintaining comprehensive data platforms, encompassing Data Governance, ETL processes, Data Warehouses, and Data Lakes.

• Deep proficiency in complex SQL queries, with a demonstrated ability to optimize query performance for large datasets.

• Hands-on experience with Big Data technologies, specifically Apache Spark and the Hadoop ecosystem.

• Expertise in Spark performance tuning and optimization techniques to ensure efficient data processing.

• 4-6 years of practical experience with Amazon Web Services (AWS) data engineering services, including Glue, EMR, Lambda, Redshift, Athena, Glue Catalog, and Lake Formation.

• Familiarity with UI and Business Dashboard development using tools such as Tableau, Qlik, QuickSight, or other similar frontend visualization platforms (Good to have).

• Experience working within an Agile methodology, utilizing tools like JIRA, Git, Confluence, and Jenkins for project management and CI/CD pipelines.

• Good to have domain expertise on BFSI/NBFC/FinTech.

Required Qualification :

Bachelors degree/University degree or equivalent of B.E/B.Tech.

Technical Competencies :

• Design and develop scalable, batch and real time data pipelines, addressing business requirements.

• Continuously enhance team practices and technical processes aligning with broader organizational goals.

Behavioral Competencies :

• Collaboration & Teamwork : Working effectively with cross-functional teams (data scientists, analysts, software engineers, business stakeholders) to achieve common goals.

• Problem-Solving & Analytical Thinking : The ability to dissect complex data challenges, identify root causes, and scalable solutions.

• Adaptability & Flexibility : The willingness to learn new technologies, adapt to changing project requirements, and navigate evolving data landscapes.

• Communication Skills : Clearly articulating technical concepts to non-technical audiences, documenting processes, and effectively discussing solutions with peers.

Personal Attributes :

• Integrity : Handling sensitive data responsibly and maintaining ethical standards.

• Patient & Persistent : Tackling complex, often time-consuming data integration and debugging tasks with resilience.

• Curious & Inquisitive : A natural inclination to explore data and understand underlying systems.

• Self-Motivated : Driven to achieve results and improve processes independently.