Flexible Real-Time Data Engineer | Kinesis & Databricks
Data Engineer at Accenture Australia in Canberra building real-time streaming pipelines on AWS: moving data from Amazon Kinesis into the Databricks Lakehouse using Spark Structured Streaming and Delta tables, optimising pipeline performance, and writing clean Python and SQL alongside batch data engineers.
- Compensation
- Not disclosed
- Location
- Canberra, Australia
- Work setup
- Not specified
- Seniority
- Not specified
- Employment
- Not specified
- Experience
- Not specified
- Category
- Data engineering
- Posted
Skills
Must have
- Hands-on experience building real-time data pipelines with Amazon Kinesis
- Experience with Databricks Lakehouse
- AWS data technologies
- Spark Structured Streaming
- Ingesting data into Delta tables
- Pipeline performance optimisation
- Python
- SQL
Full description
Accenture Australia in Canberra seeks a hands-on Data Engineer to build reliable real-time data pipelines using Amazon Kinesis and Databricks. You will work with AWS data technologies to move data from Kinesis into the Databricks Lakehouse with Spark Structured Streaming and Delta tables.
You will develop streaming solutions, ingest data into Delta tables, optimise pipelines for performance, and write clean Python and SQL while collaborating with batch data engineers.