Astra North Infoteck Inc.
Senior Data Engineer (AWS, Big Data, ETL, Netezza, Spark, Kafka, Data Warehousing)
- Compensation
- Not disclosed
- Location
- Toronto, Canada
- Work setup
- Not specified
- Seniority
- Senior
- Employment
- Not specified
- Experience
- Not specified
- Category
- Data engineering
- Posted
Skills
Nice to have
- • Python
- Operating Systems
- • Strong experience in Unix/Linux environments.
- Key Qualifications
- • 10+ years of experience in Data Engineering / Big Data / Platform Engineering.
- Key Responsibilities
- • Design, develop, maintain, and support scalable data pipelines using Big Data and AWS technologies.
- • Lead and support data platform migration initiatives (On-Prem to AWS Cloud), ideally with Netezza background.
- • Develop and manage ETL/ELT processes using tools like SSIS, Pentaho, or similar.
- • Implement and manage AWS services such as:
- • EMR
- • S3
- • EC2
- • Redshift
- • Glue
- • Airflow
- • Build and optimize data workflows and orchestration pipelines using Airflow.
- • Work with real-time streaming technologies such as:
- • Kafka
- • Spark Streaming
- • Perform data ingestion, transformation, and validation from multiple data sources.
- • Optimize SQL queries for performance and scalability.
- • Monitor system performance, troubleshoot issues, and ensure system reliability.
- • Collaborate with cross-functional teams including:
- • Developers
- • Architects
- • Infrastructure teams.
Full description
Senior Data Engineer | AWS Cloud | Big Data | ETL | Netezza | Spark | Kafka | Data Platform Modernization
Job Summary
The ideal candidate will have strong expertise in designing, developing, and supporting scalable data pipelines and distributed systems, along with hands-on experience in Big Data ecosystem tools, AWS services, and real-time data processing.
This role involves working on data platform modernization, cloud migrations, Data Warehousing, and ETL in a fast-paced enterprise environment.
Required Technical Skills
Cloud Technologies
- Strong experience in AWS Cloud services:
- EMR
- EC2
- S3
- VPC
- RDS
- Redshift
- AWS Glue
- IAM
- CloudWatch
- CloudFormation
- Experience with:
- Airflow (or AWS Managed Workflows)
Databases
- Experience working with:
- Netezza (Mandatory)
- SQL-based systems:
- SQL Server
- PostgreSQL
- Data warehouses:
- Teradata
- Redshift
- Netezza
ETL Tools
- Hands-on experience with:
- SSIS
- Pentaho or similar ETL tools
Programming & Scripting
- Strong proficiency in:
- SQL
- Shell scripting
- Good to have:
- Python
Operating Systems
- Strong experience in Unix/Linux environments.
Key Qualifications
- 10+ years of experience in Data Engineering / Big Data / Platform Engineering.
Key Responsibilities
- Design, develop, maintain, and support scalable data pipelines using Big Data and AWS technologies.
- Lead and support data platform migration initiatives (On-Prem to AWS Cloud), ideally with Netezza background.
- Develop and manage ETL/ELT processes using tools like SSIS, Pentaho, or similar.
- Implement and manage AWS services such as: