Deutsche Telekom Digital Labs
Senior Data Engineer II
- Compensation
- Not disclosed
- Location
- Gurgaon
- Work setup
- Not specified
- Seniority
- Senior
- Employment
- Not specified
- Experience
- 5+ years
- Category
- Data engineering
- Posted
Skills
Must have
- Strong hands-on experience with GCP and BigQuery.
- Advanced SQL and Python, Airflow / Cloud Composer, Dataform / dbt, Dataflow / Apache Beam, Dataproc / Spark / PySpark.
- Strong understanding of ETL/ELT and data pipeline development.
- Experience with batch, incremental, and idempotent processing.
- Good understanding of data modelling and data quality.
- Experience in pipeline optimization and troubleshooting.
- Hands-on experience with Git, CI/CD, and testing.
- Strong production support and problem-solving skills.
- 4-7.5 years of relevant data engineering experience.
- Strong experience building production-grade data pipelines.
- Excellent SQL and Python skills.
- Strong hands-on experience with the GCP data ecosystem, particularly BigQuery.
- Ability to work with large datasets and optimize data processing workflows.
- Strong analytical and troubleshooting skills.
- Good understanding of software engineering best practices, including version control, CI/CD, and testing.
- Strong communication and collaboration skills.
Full description
We are looking for a Data Engineer II with strong hands-on experience in GCP, BigQuery, Python, SQL, and modern data engineering technologies to build and maintain scalable, reliable, and production-grade data platforms. The ideal candidate should have strong expertise in data pipeline development, ETL/ELT, data modelling, pipeline optimization, and production support, with the ability to work effectively with large-scale datasets on GCP.
Responsibilities:
- Design, develop, and maintain scalable data pipelines on GCP.
- Work with large-scale datasets using BigQuery and other GCP data services.
- Develop robust ETL/ELT pipelines for batch and incremental data processing.
- Implement idempotent and reliable data processing workflows.
- Build and maintain workflows using Airflow / Cloud Composer.
- Develop data transformation pipelines using Dataform/dbt.
- Work with Dataflow / Apache Beam for distributed data processing.
- Work with Dataproc, Spark, and PySpark for large-scale data processing.
- Optimize data pipelines for performance, scalability, and cost efficiency.
- Implement data quality checks, testing, and monitoring to ensure reliable data products.
- Troubleshoot production issues and provide effective production support.
- Collaborate with engineering and business teams to deliver high-quality data solutions.
Requirements:
- Strong hands-on experience with GCP and BigQuery.
- Advanced SQL and Python, Airflow / Cloud Composer, Dataform / dbt, Dataflow / Apache Beam, Dataproc / Spark / PySpark.
- Strong understanding of ETL/ELT and data pipeline development.
- Experience with batch, incremental, and idempotent processing.
- Good understanding of data modelling and data quality.
- Experience in pipeline optimization and troubleshooting.
- Hands-on experience with Git, CI/CD, and testing.
- Strong production support and problem-solving skills.
- 4-7.5 years of relevant data engineering experience.
- Strong experience building production-grade data pipelines.
- Excellent SQL and Python skills.
- Strong hands-on experience with the GCP data ecosystem, particularly BigQuery.
- Ability to work with large datasets and optimize data processing workflows.
- Strong analytical and troubleshooting skills.
- Good understanding of software engineering best practices, including version control, CI/CD, and testing.
- Strong communication and collaboration skills.