Data Engineer - (AWS Data Services)
Data Engineer at the Australian Securities Exchange, embedded in a domain-aligned data product team supporting derivatives clearing. Day to day: ingesting and curating ASX's core data assets and building pipelines, files, reports, dashboards and APIs on an AWS-based central data platform using Python, Spark, Kafka, Airflow and Terraform.
- Compensation
- Not disclosed
- Location
- Sydney, Australia
- Work setup
- Hybrid
- Seniority
- Not specified
- Employment
- Part time
- Experience
- Not specified
- Category
- Data engineering
- Posted
Skills
Must have
- Hands-on engineer with deep experience across AWS services for ingestion, curation, storage, orchestration, data management, analytics and distribution
- Good knowledge of data modelling and structuring data in a data warehousing environment following best-practice data management standards
- Data product architecture and solution design experience with working knowledge of DevSecOps practices for automating complex data pipelines
- Experience in real-time streaming using Confluent Kafka and AWS services
- Experience with Airflow, AWS Glue, Apache Iceberg, Athena, Redshift and DataZone
- Experience with data visualization tools such as Tableau/Power BI/QuickSight
- Solid experience in Python, Spark and Terraform/Terragrunt
- Legally authorised to work in Australia on a permanent basis without any restrictions
Nice to have
- Experience working in a platform/product operating model leveraging frameworks and development kits for standardisation, re-use and scale
- Working knowledge of data mesh/data product concepts with federated domain-level data ownership on centralised platforms
- Experience in the Capital Markets domain and understanding of derivative products
Full description
Australian Securities Exchange – Sydney NSW
ASX: Powering Australia's financial markets
Why join the ASX?
When you join ASX, you're joining a company with a strong purpose - to power a stronger economic future by enabling a fair and dynamic marketplace for all.
In your new role, you'll be part of a leading global securities exchange with a strong brand. We are known for being a trusted market operator and an exciting data hub.
Want to know why we are a great place to work, click on the link to learn more.
www.asx.com.au/about/careers/a-great-place-to-work
We are more than a securities exchange!
The ASX team brings together talented people from a diverse range of disciplines.
We run critical market infrastructure, with 1 in 3 people employed within technology. Yet we have a unique complexity of roles across a range of disciplines such as operations, program delivery, financial products, investor engagement, risk and compliance.
We're proud to foster a workplace where diversity is celebrated and inclusion is part of our everyday culture. Our employee-led networks champion LGBTIQ+ inclusion, promote gender equality, accessibility and wellbeing, inspire giving and volunteering, and celebrate cultural and religious events, creating a sense of belonging for all. As an AWEI Bronze employer and member of the Champions of Change Coalition for gender equality, we're committed to a fair and inclusive workplace where everyone can thrive.
Your Team
The Derivatives Clearing Technology team is part of the Markets Technology division within ASX Technology. We are a trusted partner to the Clearing business, providing solutions that ensure a stable, efficient clearing and settlement facility for ASX's derivatives trading business.
Your responsibilities
- Embedded within one of our line-of-business data and reporting teams, you will collaborate with stakeholders to understand their data objectives and champion the use and evolution of the central data platform.
- You will work within a domain-aligned data product team consisting of line-of-business data owners and SMEs working alongside engineering resources provided by Technology.
- You will be hands-on in the ingestion of ASX's core data assets by the product teams, their curation into source-aligned data products and the creation of consumable products in the form of files, reports, dashboards and APIs.
- You will form part of a data engineering practice and community that will design and drive the adoption of standards, frameworks and best practices.
- You will ensure data pipelines and products are designed and built to be run within a critical market infrastructure environment by the persistent product teams who build them.