Data Engineer - Google Cloud Platform

Job Description :

We are looking for an experienced GCP Data Engineer with strong expertise in Google Cloud Platform (GCP), Dataproc, PySpark, BigQuery, and Java. The ideal candidate will be responsible for designing, developing, and maintaining scalable data processing solutions and cloud-based data platforms that support analytics, reporting, and business intelligence initiatives.

The candidate should have hands-on experience in building large-scale data pipelines, processing structured and unstructured data, optimizing data workflows, and working with distributed computing frameworks in a cloud environment.

Key Responsibilities :

• Design, develop, and maintain scalable data engineering solutions on Google Cloud Platform (GCP).

• Build and optimize batch and real-time data processing pipelines using PySpark and Dataproc.

• Develop efficient ETL/ELT processes to ingest, transform, validate, and load large volumes of data from multiple sources.

• Create and manage data lakes and data warehouse solutions using BigQuery.

• Write high-performance PySpark jobs for distributed data processing and analytics.

• Develop and maintain Java-based applications and services that integrate with data platforms and cloud services.

• Optimize BigQuery datasets, tables, partitions, clustering, and SQL queries for performance and cost efficiency.

• Implement data quality checks, monitoring, validation, and error-handling mechanisms.

• Collaborate with business analysts, data scientists, architects, and stakeholders to understand data requirements and deliver scalable solutions.

• Ensure data security, governance, and compliance standards are followed across cloud environments.

• Troubleshoot performance bottlenecks and production issues in data pipelines and cloud infrastructure.

• Participate in code reviews, architecture discussions, and technical design sessions.

• Support CI/CD implementation and deployment automation for data engineering workloads.

• Create technical documentation, operational runbooks, and best practices for data platforms.

Required Technical Skills :Google Cloud Platform (GCP) :

• Strong hands-on experience with Google Cloud Platform services.

• Experience working with :

i. Dataproc

ii. BigQuery

iii. Cloud Storage (GCS)

iv. Pub/Sub

v. Cloud Composer