M-League
Sr. Data Engineer
- Compensation
- Not disclosed
- Location
- Bangalore
- Work setup
- Not specified
- Seniority
- Senior
- Employment
- Not specified
- Experience
- 2+ years
- Category
- Data engineering
- Posted
Skills
Must have
- Experience: 2+ years in data engineering, with deep data warehousing exposure at scale.
- SQL and BigQuery: Expert-level SQL; hands-on BigQuery optimization (partitioning, clustering, slots, cost).
- Data Modeling: Strong command of medallion architecture, dimensional/fact modeling, grain definition, and schema design.
- Programming: Production-grade Python; PySpark on Dataproc (or Databricks/EMR equivalents).
- Streaming: Working experience with Kafka-based ingestion (Confluent preferred).
- Data Quality: Experience building validation, observability, and reconciliation into pipelines.
- Stakeholder Skills: Proven ability to gather requirements, challenge assumptions, and communicate trade-offs clearly.
- Education: Bachelor's/Master's in CS, engineering, or a related field (or equivalent experience).
Nice to have
- LLM-in-pipeline experience: AI-driven data quality checks, anomaly triage, or text-to-SQL for self-service analytics.
- Broader GCP: Pub/Sub, Cloud Run, Cloud Functions, Cloud SQL.
- Gaming, fintech, or other high-volume consumer event domains.
Full description
We are looking for a Senior Data Engineer (SDE 2) to own the design and evolution of our data warehouse. You will architect and build the medallion layers (bronze → silver → gold) on BigQuery, drive data modeling and design decisions, and partner directly with analytics, product, and game teams to turn ambiguous requirements into reliable, well-modeled datasets. You'll work as an AI-augmented engineer—modern AI tooling is part of how this team ships.
The core responsibilities for the job include the following:
Data Warehousing and Modeling:
- Own the design of silver/gold layer tables: grain, join keys, schema, partitioning, and clustering decisions for large-scale datasets.
- Build and maintain batch and streaming ETL pipelines (using big data technologies) feeding the medallion architecture.
- Design idempotent, observable pipelines with QC gates, backfill strategies, and clear failure semantics.
- Standardize data models and conventions across multiple game products/projects.
- Deliver high-quality data pipelines with utmost importance to the correctness and availability of the data to different target audiences/systems.
Design Decisions and Technical Ownership:
- Lead source-system profiling and drive grain, deduplication, and enrichment decisions backed by data, not assumptions.
- Own trade-off calls: cost vs. freshness, load strategy, schema evolution, and query/slot optimization.
- Document designs and decisions so they survive beyond the author.
Stakeholder Management:
- Partner with analysts, PMs, and game teams to translate business questions into warehouse requirements and reconcile instrumentation gaps.
- Communicate design proposals and data profiling findings to both technical and leadership audiences.
Requirements:
- Experience: 2+ years in data engineering, with deep data warehousing exposure at scale.
- SQL and BigQuery: Expert-level SQL; hands-on BigQuery optimization (partitioning, clustering, slots, cost).
- Data Modeling: Strong command of medallion architecture, dimensional/fact modeling, grain definition, and schema design.
- Programming: Production-grade Python; PySpark on Dataproc (or Databricks/EMR equivalents).
- Streaming: Working experience with Kafka-based ingestion (Confluent preferred).
- Data Quality: Experience building validation, observability, and reconciliation into pipelines.
- Stakeholder Skills: Proven ability to gather requirements, challenge assumptions, and communicate trade-offs clearly.
- Education: Bachelor's/Master's in CS, engineering, or a related field (or equivalent experience).