iShare Inc
Data Engineer GCP AWS & Databricks
- Compensation
- Not disclosed
- Location
- Not listed
- Work setup
- Remote
- Seniority
- Not specified
- Employment
- Contract
- Experience
- 5+ years
- Category
- Data engineering
- Posted
Skills
Must have
- 5+ years of professional data engineering experience.
- Strong hands-on experience with both GCP and AWS.
- Expertise in Databricks, Apache Spark, and PySpark.
- Strong programming skills in Python and SQL.
- Proven experience developing ETL/ELT pipelines and cloud data platforms.
- Experience with data warehouses, data lakes, and dimensional data modeling.
- Experience with orchestration tools such as Apache Airflow or Google Cloud Composer.
- Understanding of data security, governance, monitoring, and quality frameworks.
- Strong analytical, troubleshooting, and problem-solving skills.
- Excellent communication and cross-functional collaboration skills.
- GCP | AWS | Databricks | Apache Spark | PySpark | Python | SQL | ETL/ELT | Airflow/Cloud Composer | Data Warehousing | Data Lakes
- This is a remote position.
Nice to have
- Experience with GCP services such as BigQuery, Cloud Storage, Dataflow, Dataproc, Pub/Sub, and Cloud Composer.
- Experience with AWS services such as S3, Glue, EMR, Redshift, Lambda, and Kinesis.
- Familiarity with Delta Lake and Databricks Lakehouse architecture.
- Experience with streaming technologies such as Apache Kafka.
- Familiarity with CI/CD, Git, Terraform, and cloud infrastructure automation.
- Experience working in Agile development environments.
Full description
Data Engineer – GCP, AWS & Databricks
Location: Remote – United States
Employment Type: Contract
Experience Required: 6+ Years
Position Overview
We are seeking an experienced Data Engineer to design, develop, and maintain scalable cloud-based data pipelines and data platforms. The ideal candidate will have strong hands-on experience with GCP, AWS, Databricks, Apache Spark, PySpark, Python, and SQL.
This role requires expertise in building reliable ETL/ELT pipelines, integrating data from multiple sources, and optimizing cloud data solutions for performance, security, scalability, and cost.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines.
- Build cloud-based data solutions using GCP and AWS services.
- Use Databricks, Apache Spark, and PySpark for large-scale data processing.
- Integrate structured, semi-structured, and unstructured data from multiple sources.
- Develop and optimize batch and real-time data-processing workflows.
- Improve pipeline performance, reliability, scalability, and cost efficiency.
- Implement data-quality checks, monitoring, security, and governance standards.
- Design and support cloud data warehouses and data lakes.
- Troubleshoot production issues and perform root-cause analysis.
- Collaborate with data architects, analysts, application teams, and business stakeholders.
- Create and maintain technical documentation for pipelines, data models, and workflows.
Required Qualifications
- 5+ years of professional data engineering experience.
- Strong hands-on experience with both GCP and AWS.
- Expertise in Databricks, Apache Spark, and PySpark.
- Strong programming skills in Python and SQL.
- Proven experience developing ETL/ELT pipelines and cloud data platforms.
- Experience with data warehouses, data lakes, and dimensional data modeling.
- Experience with orchestration tools such as Apache Airflow or Google Cloud Composer.
- Understanding of data security, governance, monitoring, and quality frameworks.
- Strong analytical, troubleshooting, and problem-solving skills.
- Excellent communication and cross-functional collaboration skills.