Data Engineer - Azure Databricks

Role Overview :

We are seeking a seasoned Data Engineer to join our high-performing analytics team based in Chennai, Kolkata, Hyderabad, or Pune.

In this role, you will architect and maintain scalable data pipelines, transforming complex raw data into actionable business intelligence.

You will collaborate closely with cross-functional data science teams, product managers, and business stakeholders to translate technical requirements into robust data solutions.

By optimizing our cloud infrastructure and ensuring seamless data flow, you will directly influence the speed and accuracy of our decision-making processes, ultimately driving significant value for our global customer base.

Key Responsibilities :

• Design and implement end-to-end ETL pipelines using Azure Data Factory to ensure high-quality data ingestion from diverse source systems.

• Develop complex data transformation logic and processing workflows using PySpark within the Azure Databricks environment to support large-scale analytics.

• Optimize SQL queries and database performance to ensure efficient data retrieval and reporting for internal stakeholders.

• Collaborate with engineering teams to troubleshoot data quality issues and implement automated monitoring solutions that maintain system reliability.

• Translate business requirements into scalable technical architectures, ensuring that data models align with long-term organizational goals.

Required Skillset :

• Demonstrated expertise in building and managing cloud-native data platforms using Azure, with a deep understanding of distributed computing principles.

• Proficiency in writing production-grade PySpark code and complex SQL queries to handle large datasets effectively.

• Proven ability to design and orchestrate sophisticated ETL/ELT workflows using Azure Data Factory to meet stringent business SLAs.

• Strong interpersonal skills with the ability to communicate complex technical concepts to non-technical stakeholders and work effectively in a collaborative, team-oriented environment.

• A Bachelors degree in Computer Science, Engineering, or a related quantitative field, paired with a minimum of 4 - 8 years of relevant professional experience.

• High adaptability to hybrid or remote work environments, maintaining productivity and clear communication across distributed teams.