Data Engineer - Python/Scala

Role Overview :

As a Data Engineer at Rayosys, you will be responsible for designing, building, and maintaining the data infrastructure that powers our clients' analytical capabilities. You will work closely with cross-functional teams, including data scientists, software developers, and business stakeholders, to transform raw data into reliable, high-quality assets. This role is pivotal in ensuring that our data pipelines are efficient, scalable, and secure, directly influencing the speed and accuracy of business intelligence and decision-making processes across the organization.

Key Responsibilities :

• Architect and maintain robust data pipelines to ingest, process, and store large volumes of structured and unstructured data from diverse sources.

• Develop and optimize complex ETL/ELT workflows to ensure high data availability and integrity for downstream analytical applications.

• Collaborate with stakeholders to translate business requirements into efficient data models that support long-term scalability.

• Implement real-time data streaming solutions using Kafka to provide low-latency insights for critical business operations.

• Manage and tune big data environments, including Hadoop and Spark clusters, to ensure optimal performance and cost-efficiency.

• Design secure and resilient data warehousing solutions on AWS, ensuring compliance with data governance and security standards.

Required Skillset :

• Demonstrated expertise in designing complex data architectures using Python, Scala, and advanced SQL for high-performance data processing.

• Proven ability to manage large-scale distributed systems, including deep hands-on experience with Hadoop, Hive, and Spark.

• Strong proficiency in data modeling techniques and the implementation of data warehousing strategies to support enterprise-level reporting.

• Experience in building and monitoring real-time data pipelines using Kafka and integrating them within AWS cloud environments.

• Ability to work effectively with NoSQL databases and manage large datasets with a focus on performance tuning and optimization.

• Excellent communication skills, with the ability to articulate technical concepts to non-technical stakeholders and collaborate effectively within a distributed team environment.

• Candidates should possess 6 - 12 years of relevant experience in data engineering roles, with a strong track record of delivering end-to-end data solutions.

• Flexibility to work across our Bangalore, Hyderabad, Delhi NCR, or Pune locations as per organizational requirements.