Montash Limited
Apache Data Engineer (SC Cleared)
Build and maintain ETL data pipelines into a Lakehouse architecture across AWS and on-premises, using change data capture and event-driven ingestion, data modeling, and Git-based CI/CD within a sprint team. Core stack is Apache tooling plus Java/Groovy; UK SC security clearance is required.
- Compensation
- Not disclosed
- Location
- London, United Kingdom
- Work setup
- Not specified
- Seniority
- Not specified
- Employment
- Not specified
- Experience
- Not specified
- Category
- Data engineering
- Posted
Skills
Must have
- SC (Security Check) clearance
- Experience building data pipelines within Lakehouse architectures
- Comfortable with Apache tooling for data engineering work
- Experience with event sourcing, change data capture and event-driven architectures
- Java and/or Groovy with a solid grasp of object-oriented design principles
- Experience working in an SDLC with an integrated team on CI/CD tooling (Git)
Nice to have
- Apache NiFi
- AWS, particularly EMR, S3 and SQS
- Awareness of domain-driven design
- Linux and scripting
- Postgres or PL/SQL
Full description
- Design and build data pipelines and ETL processes into a Lakehouse architecture across cloud (AWS) and on premises platforms
- Implement change data capture and event-driven patterns to ingest data from source systems
- Develop data models for the pipeline layer
- Handle non-functional requirements including security, performance and metadata labelling
- Write unit and integration tests for pipeline components
- Work within a defined SDLC using Git-based CI/CD tooling
- Contribute to sprint delivery as part of an integrated development team
Experience building data pipelines within Lakehouse architectures
- Comfortable with Apache tooling, for data engineering work
- Experience with event sourcing, change data capture and event-driven architectures
- Java and/or Groovy, with a solid grasp of object-oriented design principles
- Experience working in an SDLC with an integrated team on CI/CD tooling (Git)
Nice-to-have
- Apache NiFi
- AWS, particularly EMR, S3 and SQS
- Awareness of domain-driven design
- Linux and Scripting
- Postgres or PL/SQL