AWS DATA ENGINEER & ETL EXPERT ( FREELANCE REMOTE )

We are looking for part time Freelancer for AWS Data Engineer Position.

Releavant Exp : 9+

Mandatory Tech Stack required: **

Source: Salesforce (ECRM & OSC)
Backup: Grax
Cloud: AWS (EC2, S3, CloudWatch, DynamoDB, RDS, Secrets Manager, ALB, ASG)
Infrastructure as Code: Terraform
File Format: Parquet
Target: Enterprise Data Lake (EDL)
Schema: Blue Schema
Database: Blue Database
Query Language: SQL

JD :

• Build and maintain AWS data pipelines

• Develop ETL/ELT pipelines using AWS Glue, PySpark, Python, and SQL.
• Ingest data from sources like Salesforce, databases, APIs, and S3.
• Load curated data into Amazon Redshift and data lake storage.

• Optimize Athena and data lake performance

• Convert JSON/CSV data into Parquet.
• Use Snappy compression.
• Design proper partitioning strategies.
• Resolve split limit and performance issues.
• Optimize Athena query costs.

• Manage modern data lake architecture

• Work with Apache Iceberg tables.
• Perform migrations from traditional Parquet tables.
• Support schema evolution, time travel, and ACID transactions.

• Production support and troubleshooting

• Investigate Glue jobs that suddenly become slow.
• Debug Lambda timeouts.
• Fix missing records and data quality issues.
• Resolve Redshift performance problems.
• Perform root cause analysis (RCA).

• Infrastructure as Code

• Build AWS infrastructure using Terraform.
• Create reusable modules.
• Manage Auto Scaling Groups, ALBs, IAM, Lambda, Secrets Manager, S3, Redshift, and DynamoDB.
• Troubleshoot Terraform state and production deployment issues.

• Security

• Manage AWS Secrets Manager.
• Configure Lambda-based secret rotation.
• Ensure Terraform does not overwrite rotated passwords.
• Implement IAM least-privilege access.

• Backup and Disaster Recovery

• Work with enterprise backup tools like Rubrik (their environment may use Grax for Salesforce).
• Validate backup jobs.
• Perform restores.
• Support disaster recovery testing.
• Verify restored data.

• Data Quality

• Validate source and target record counts.
• Maintain audit/control tables (for example in DynamoDB).
• Check checksums and duplicate records.
• Troubleshoot data discrepancies reported by business users.

• AWS Services

• S3
• Glue
• Athena
• Lambda
• Redshift
• DynamoDB
• EventBridge
• Step Functions
• CloudWatch
• Secrets Manager
• IAM
• Auto Scaling Groups
• Application Load Balancer