xTag Services
GCP Data Engineer - Python
- Compensation
- Not disclosed
- Location
- Chennai
- Work setup
- Not specified
- Seniority
- Not specified
- Employment
- Not specified
- Experience
- 2+ years
- Category
- Data engineering
- Posted
Skills
Full description
GCP Data Engineer - Python / PySparkExperience : 5.5 - 9 Years
Openings : 8
Role Overview :
We are looking for experienced Data Engineers with strong expertise in Python or PySpark, SQL, GCP cloud services, data processing, data warehousing, and data pipelines. The ideal candidate should have hands-on experience with BigQuery, Dataflow or Dataproc, Pub/Sub, Apache Kafka, Airflow/Cloud Composer, and Terraform.
Key Responsibilities :
- Design, develop, and maintain scalable data pipelines.
- Develop data processing solutions using Python and/or PySpark.
- Develop complex SQL queries and optimize data processing workloads.
- Work with GCP data services including BigQuery, Dataflow, Dataproc, Pub/Sub, and Cloud Functions.
- Build and maintain data ingestion and transformation pipelines.
- Develop and manage workflows using Airflow, Astronomer, or Cloud Composer.
- Work with Apache Kafka for real-time data processing and streaming.
- Implement infrastructure automation using Terraform.
- Monitor data pipelines and troubleshoot production issues.
- Develop monitoring dashboards and ensure pipeline reliability.
- Collaborate with Data Scientists, Software Engineers, business stakeholders, and other technical teams.
Mandatory Technical Skills :
- SQL - minimum 4 years
- Python or PySpark - minimum 4 years
- GCP - minimum 3 years
- BigQuery
- Dataflow or Dataproc
- Pub/Sub
- Scheduled Queries
- Cloud Functions
- Monitoring Tools / Dashboards
- Apache Kafka
- Terraform scripting - minimum 2 years
- Airflow / Astronomer / Cloud Composer - minimum 2 years
Good to Have :
- Data Fusion, Tekton Pipeline, CI/CD Pipeline, GCP Secret Manager
- Data/ML solutions and advanced analytics exposure
- Prompt Engineering, Generative AI, LLM, Agentic AI, LangGraph, CrewAI, OpenAI SDK, Google ADK
Soft Skills :
- Strong written and verbal communication skills
- Strong presentation and stakeholder management skills
- Excellent analytical and problem-solving abilities
- Strong collaboration and interpersonal skills