Design, develop, and maintain scalable data pipelines for processing large volumes of structured and unstructured data.•Implement and optimize ETL workflows to support analytics and reporting requirements.•Collaborate with data analysts, data scientists, and business stakeholders to understand data needs and deliver relevant solutions.•Manage and monitor data infrastructure, ensuring high availability, security, and performance.•Integrate data from multiple sources, including databases, APIs, and external platforms.•Utilize Big Data tools such as Hadoop and Spark for distributed data processing.•Develop and maintain SQL queries, stored procedures, and data models.•Work with cloud platforms (AWS, Azure, GCP) to deploy and scale data solutions.•Ensure data quality and consistency through validation and cleansing techniques.•Document data flows, architecture, and operational procedures.
Analytics - Team Member - Data Engineer - west at Kotak Neo (Mumbai, Maharashtra, India) — aijobsearch