Senior Data Engineer - AWS Technologies

Job Description :

Eucloid is looking for a Data Engineer to join our Data Platform team supporting various business applications.

The ideal candidate will support development of data infrastructure for our clients by participating in activities which may include starting from technology selection to designing and building of different components.

Candidate will also involve in projects like integrating data from various resources, managing big data pipelines that are easily accessible with optimized performance of overall ecosystem.

The ideal candidate is an experienced data wrangler who will support our software developers, database architects and data analysts on business initiatives.

You must be self-directed and comfortable supporting the data needs of cross-functional teams, systems, and technical solutions.

Qualifications :

• Hands-On experience in AWS technologies (AWS S3, AWS EMR, Athena etc.), Redshift, Python.

• Must have around 4+ years of experience in delivering data engineering projects.

• Have strong knowledge in ETL, Data warehousing, Business intelligence.

• Should be able to configure pipelines to ingest data from landing zone to the data platform.

• Should be able to configure pipelines to ingest and process data from supplier landing zone to the data platform.

• Monitor and respond to scheduled workloads that feed data from and to the data platform.

• Communicate and work on schema drift and/or other Supplier issues.

• Create and execute quality scripts to monitor and maintain the accuracy of our data.

• Hands-On experience with Databricks or snowflake is a huge plus.

Preferred Skills :

• Hands-on expertise in one or more Amazon Web Services (AWS) technologies, such as Kinesis, S3, Redshift, Athena, Lambda.

• Experience with following aspect of data engineering :

• Building and optimizing big data data pipelines, architectures and data sets.

• Manipulating, processing and extracting value from large, disconnected datasets.

• Development of data pipeline and workflow management tools : Azkaban, Luigi, Airflow, etc.

• Good experience with at least one ETL tool like Talend, Informatica etc.

• Build processes supporting data transformation, data structures, metadata, dependency and workload management.

• Expertise in at least one of the NoSQL databases.