Software Engineer – Data Platform
Software engineering role on Continental ContiTech's global data platform team: design and operate production-grade data ingestion pipelines and APIs (from SAP, Salesforce, SharePoint, manufacturing systems), build Python/Scala extraction services, and develop data catalog and multi-layered data quality frameworks on Azure/Databricks using CI/CD practices.
- Compensation
- Not disclosed
- Location
- Bangalore, Karnataka, in
- Work setup
- Hybrid
- Seniority
- Not specified
- Employment
- Not specified
- Experience
- 5+ years
- Category
- Data engineering
- Posted
Skills
Must have
- Degree in Computer Science, Data Engineering, or a related field
- 5-8 years in software development with hands-on experience with inbound/outbound data exchange interfaces (APIs, CDC, endpoint management)
- Hands-on experience with Scala/Java, Python, and advanced SQL, using Git and CI/CD methodologies
- Experience building robust ingestion pipelines and ETL jobs in a software engineering context
- Track record with Azure or other hyperscaler resource management
- Comfortable working across geographies and time zones with global teams and enterprise stakeholders
Nice to have
- Familiarity with data lakehouse and data warehousing; Databricks and Azure knowledge is a plus
- Strong backend/software engineering background (Java, Scala, distributed systems) with interest in moving into data platform engineering
Full description
Job Description
Job Description
This is a software engineering role focused on building production-grade data platform components.
You will design and operate services, APIs, and data pipelines in a CI/CD-driven environment.
This is NOT a notebook-driven or analytics-focused role.
Job Description:
- Design, implement, and maintain secure, scalable and fault-tolerant data ingestion pipelines from a wide variety of source systems, including SAP, Salesforce, SharePoint, APIs, and (legacy) manufacturing platforms.
- Build and enhance production-grade metadata-driven data handling components and inbound-/outbound-interfaces.
- Own production data interfaces end-to-end.
- Develop reusable, version-controlled platform components, APIs, or data interfaces, following software engineering best practices (CI/CD, automated testing, version control, monitoring).
- Contributes to a global data engineering team delivering to all major business domains.
- Establish and maintain holistic data quality management, monitoring and reporting.
- Collaborates across business IT, cybersecurity, infrastructure, and architecture teams to ensure secure and sustainable delivery.
Main Tasks:
- Design and implement fault-tolerant, observable data pipelines, including logging, alerting, and failure handling strategies.
- Build and maintain Python- or Scala-based extraction services (e.g., Debezium Server, custom APIs, rclone)
- Implement CDC, delta, and event-based patterns.
- Build and enhance push-based HTTP, Kerberos-authenticated custom data lake transfer delivery component.
- Integrate with systems such as Salesforce, SharePoint, and other API- or file-based endpoints.
- Establish, operate and troubleshoot extraction from SAP using tools like Theobald Extract Universal.
- Establish and maintain a web-accessible data catalog application as part of the data platform, focusing on backend services, metadata management, and integration with platform components.
- Integrate dataset discoverability, preview/exploration options, and lineage information using Unity Catalog as a backend metadata system.
- Design and implement structured access request workflows including request submission, approval chains, audit trail, and enablement triggers.
- Perform design reviews with Cybersecurity.
- Ensure documentation and compliance for all interfaces and data ingress points.
- Manage audit and traceability requirements.
- Collaborate closely with IT and business users to translate requirements into scalable technical patterns.
- Serve as technical escalation point for complex source integration.