Apptad
Job Title: AWS Data Engineer
- Compensation
- Not disclosed
- Location
- Hyderabad
- Work setup
- Hybrid
- Seniority
- Not specified
- Employment
- Not specified
- Experience
- 3+ years
- Category
- Data engineering
- Posted
Skills
Must have
- 3+ years of experience in data engineering, application development, or related roles.
- Strong hands-on experience with AWS cloud services, PySpark, Python, and AWS Glue.
- Experience designing and developing ETL/ELT pipelines and reusable data processing frameworks.
- Working experience with Snowflake and cloud-based data warehouse solutions.
- Strong understanding of AWS S3, Lambda, SNS, SQS, Step Functions, and AWS Batch.
- Experience in data extraction, ingestion, transformation, enrichment, and integration.
- Knowledge of data lakes, data warehouses, data structures, and cloud-based data storage solutions.
- Experience in performance tuning, troubleshooting, and optimization of ETL frameworks.
- Exposure to GCP and cross-platform data processing solutions.
- Strong analytical, problem-solving, communication, and stakeholder management skills.
- Ability to work independently, make technical decisions, and manage changing priorities.
- Bachelor's or Master's degree in Computer Science, Information Technology, or a related field, or equivalent practical experience.
Nice to have
- Experience in the Retail domain.
- Knowledge of Talend Cloud ETL.
- Exposure to Apache Kafka.
- Experience with Power BI and business intelligence solutions.
- Understanding of data governance and enterprise data management practices.
- Experience working in Agile/Scrum environments.
Full description
Job Title: AWS Data Engineer PySpark, GCP & SnowflakeJob Summary
We are looking for an experienced AWS Data Engineer with strong hands-on expertise in AWS cloud services, PySpark, AWS Glue, and Snowflake. The ideal candidate will be responsible for designing, developing, maintaining, and optimizing ETL frameworks, data pipelines, data warehouses, and data lakes to support business intelligence and analytics requirements.
The candidate should have experience working with cloud-based data integration solutions, developing scalable data processing frameworks, and collaborating with business and technical stakeholders. Experience with GCP and the Retail domain will be an added advantage.
Key Responsibilities
- Design, develop, maintain, and enhance data engineering solutions, ETL frameworks, data warehouses, and data lakes.
- Build and optimize data ingestion, extraction, transformation, and enrichment pipelines using AWS Glue, PySpark, and Python.
- Develop data processing logic by reading data from AWS S3 and applying transformations using Python, PySpark, and cloud services.
- Work with Snowflake and AWS cloud technologies to support data integration and analytics requirements.
- Understand, maintain, and enhance existing ETL frameworks built using AWS Glue and PySpark.
- Implement and manage event-driven workflows using AWS Lambda, SNS, SQS, Step Functions, and AWS Batch.
- Analyze existing data pipelines to identify performance improvements, framework optimizations, and cloud cost-saving opportunities.
- Collaborate with business teams, data analysts, application teams, and platform teams to understand requirements and resolve technical issues.
- Support the onboarding of new data feeds into existing data integration frameworks.
- Develop prototypes, proof of concepts, and production-ready solutions aligned with enterprise architecture standards.
- Follow data engineering best practices for data quality, data governance, scalability, security, and maintainability.
- Troubleshoot data pipeline failures and coordinate with platform teams to resolve operational issues.
- Evaluate existing environments, recommend suitable data engineering solutions, and adapt to evolving business requirements.
- Work effectively in an Agile environment while managing multiple priorities and delivery timelines.
Required Skills & Qualifications
- 3+ years of experience in data engineering, application development, or related roles.
- Strong hands-on experience with AWS cloud services, PySpark, Python, and AWS Glue.
- Experience designing and developing ETL/ELT pipelines and reusable data processing frameworks.
- Working experience with and cloud-based data warehouse solutions.