D2N Solutions
Cloud Data Engineer
- Compensation
- ₹3M–₹4M / yr
- Location
- Hyderabad; Pune
- Work setup
- Not specified
- Seniority
- Not specified
- Employment
- Not specified
- Experience
- Not specified
- Category
- Data engineering
- Posted
Skills
Must have
- - Strong proficiency in Python programming.
- - Handson experience with Google Cloud services: Dataflow, Pub/Sub, BigQuery, Cloud Storage, Cloud SQL, and Google Cloud SDK.
- - Familiarity with messaging systems such as Pub/Sub and Kafka.
- - Experience with CI/CD tools and practices.
- - Proficiency in Linux/Unix environments.
- - Experience with workflow/scheduling tools: UC4 Atomic, Airflow Composer, ControlM.
Nice to have
- - Knowledge of additional GCP services (e.g., Data Catalog, Looker).
- - Understanding of data modeling and schema design for analytical workloads.
- - Experience with containerization (Docker, Kubernetes) for data workloads.
- - Exposure to version control systems such as Git.
Full description
Job Title : Cloud Data Engineer
About the Role :
We are seeking a skilled Cloud Data Engineer to design, build, and maintain scalable data pipelines and analytics solutions on Google Cloud Platform. The role involves working with large datasets, implementing data ingestion, transformation, and orchestration processes, and ensuring reliable CI/CD deployment.
Key Responsibilities :
- Develop and maintain data pipelines using Python, Dataflow, and Pub/Sub/Kafka.
- Design, implement, and optimize data storage solutions in BigQuery, Cloud SQL, and Google Cloud Storage.
- Automate workflow scheduling with UC4 Atomic, Airflow Composer, and Control-M.
- Build and manage CI/CD pipelines for data engineering code and infrastructure.
- Monitor, troubleshoot, and improve performance of data processing jobs.
- Collaborate with crossfunctional teams to define data requirements and deliver actionable insights.
Required Skills :
- Strong proficiency in Python programming.
- Handson experience with Google Cloud services: Dataflow, Pub/Sub, BigQuery, Cloud Storage, Cloud SQL, and Google Cloud SDK.
- Familiarity with messaging systems such as Pub/Sub and Kafka.
- Experience with CI/CD tools and practices.
- Proficiency in Linux/Unix environments.
- Experience with workflow/scheduling tools: UC4 Atomic, Airflow Composer, ControlM.
Preferred Skills :
- Knowledge of additional GCP services (e.g., Data Catalog, Looker).
- Understanding of data modeling and schema design for analytical workloads.
- Experience with containerization (Docker, Kubernetes) for data workloads.
- Exposure to version control systems such as Git.
Qualifications :
- Bachelors degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience.