Full Stack Python Engineer – GCP & Data Engineering
- Compensation
- Not disclosed
- Location
- Bangalore Rural; Bengaluru
- Work setup
- Onsite
- Seniority
- Not specified
- Employment
- Full time
- Experience
- 6+ years
- Category
- Fullstack
- Posted
Skills
Must have
- Strong hands-on programming experience with Python.
- Strong experience building RESTful APIs and backend services.
- Strong knowledge of SQL and data-processing concepts.
- Hands-on experience with Google Cloud Platform (GCP).
- Strong experience working with BigQuery.
- Experience building data-processing solutions with Apache Beam.
- Experience with Dataproc / Spark / PySpark for distributed data processing.
- Understanding of both batch and streaming data-processing patterns.
- Experience integrating applications with databases, files, APIs and cloud services.
- Understanding of software engineering practices including modular design, testing, version control and CI/CD.
Full description
Key Responsibilities
Design and develop enterprise applications and backend services using Python.
Design and implement REST APIs and reusable service components for application and data integrations.
Build data ingestion, transformation and processing solutions on GCP.
Develop and optimize solutions using BigQuery for large-scale data processing and analytical workloads.
Develop Apache Beam pipelines for batch and streaming data-processing use cases. Beam provides native BigQuery connectors for reading from and writing to BigQuery.
Develop and support distributed data-processing workloads using Dataproc / Spark where applicable.
Integrate Python services with databases, cloud services, APIs and enterprise applications.
Design reusable frameworks for data ingestion, transformation, validation and publishing.
Implement application and pipeline-level logging, monitoring, exception handling and data-quality controls.
Develop unit and integration tests and participate in peer code reviews.
Troubleshoot application, API, data-pipeline and production issues.
Work with architecture, data engineering, DevOps, security and application teams to deliver production-ready solutions.
Follow enterprise engineering standards around security, scalability, maintainability and deployment.
Required Technical SkillsMust Have
Strong hands-on programming experience with Python.
Strong experience building RESTful APIs and backend services.
Strong knowledge of SQL and data-processing concepts.
Hands-on experience with Google Cloud Platform (GCP).
Strong experience working with BigQuery.
Experience building data-processing solutions with Apache Beam.
Experience with Dataproc / Spark / PySpark for distributed data processing.
Understanding of both batch and streaming data-processing patterns.
Experience integrating applications with databases, files, APIs and cloud services.
Understanding of software engineering practices including modular design, testing, version control and CI/CD.
Good to Have
Experience with Google Cloud Dataflow, particularly for running Apache Beam pipelines. Google Cloud documents Dataflow as supporting pipelines developed with the Apache Beam Python SDK.
Experience with Cloud Storage and Pub/Sub.