Data Engineer - Real-Time
Builds and operates Betsson's real-time data platform, designing frameworks and services for streaming data ingestion, transformation, and event-driven pipelines that other teams use to ship data products. Core stack: Kafka, Apache Spark, AWS event-driven services (MSK, Kinesis, EKS, EMR, Lambda), plus SQL/NoSQL data stores.
- Compensation
- Not disclosed
- Location
- Malta
- Work setup
- Onsite
- Seniority
- Not specified
- Employment
- Not specified
- Experience
- 3+ years
- Category
- Data engineering
- Posted
Skills
Must have
- University Degree in Computer Science, Engineering or equivalent
- At least 3 years hands-on experience building and operating real-time or event-driven data platforms and streaming applications
- Strong software engineering fundamentals with experience in Python, Scala or Java
- Experience building reusable frameworks, libraries, or internal tools
- Deep knowledge of SQL and data integration patterns
- Experience with data streaming technologies such as Kafka
- Strong experience building distributed data processing workloads using Apache Spark (PySpark and/or Scala)
- Hands-on experience designing and operating event-driven architectures on AWS (MSK, Kinesis Data Streams, EKS, EMR, Lambda, EventBridge)
- Experience with NoSQL databases such as HBase, Cassandra, and DynamoDB
- Excellent verbal and written English communication skills
Nice to have
- Experience with modern cloud database technologies such as Snowflake or Redshift
- Familiarity with real-time reporting tools like Grafana or Datadog
- Familiarity with data modelling concepts like Inmon, Kimball and Data Vault
- Knowledge of AWS services like Glue, Lambda, Step Functions and S3
- Experience orchestrating data pipelines using tools like Airflow
- Experience building or managing APIs and microservices
- Familiarity with infrastructure scripting like Terraform and CI/CD practices
Full description
The Wow
This role sits at the heart of our real-time data platform, empowering teams to build, deploy, and scale data products that drive critical business decisions and customer experiences.
You will design, develop, and operate the core platform capabilities that power streaming data pipelines, event-driven applications, and self-service data products. Working at the intersection of software engineering and data engineering, you will create scalable frameworks, tooling, and developer experiences that enable our teams to deliver reliable solutions with speed and confidence.
A taster of what you will be involved with:
- Design and build scalable, reusable frameworks and servicesfor data ingestion, transformation, and processing.
- Develop and maintain the platform's core capabilities, such as orchestration, data quality validation, and schema management tools.
- Evolve the platform's architecture to incorporate new technologies that improve scalability, reliability, and cost-effectiveness.
- Collaborate with Data DevOps Engineers to ensure platform features are automated, observable, and deployed via CI/CD.
- Act as a key contributor to the platform's roadmap, identifying and prioritizing features that unlock the most value for its users.
What we are looking for:
Must haves:
- University Degree in Computer Science, Engineering or equivalent.
- At least 3 years of hands-on experience building and operating real-time or event-driven data platforms and streaming applications.
- Strong software engineering fundamentals, with experience in a language like Python, Scala or Java.
- Experience building reusable frameworks, libraries, or internal tools.
- Deep knowledge of SQLand data integration patterns. Experience in modern cloud database technologies such as Snowflake or Redshift are considered an asset.
- Experience with data streaming technologies such as Kafka is a must.
- Strong experience building distributed data processing workloads using Apache Spark (PySpark and/or Scala).
- Hands-on experience designing and operating event-driven architectures on AWS, including services such as MSK, Kinesis Data Streams, EKS, EMR, Lambda, and EventBridge.
- Experience in NoSQL databases such as HBase, Cassandra, and DynamoDB.
- Excellent verbal and writing English communication skills.
Nice to have:
- Familiarity with real time reporting tools like Grafana or Datadog