Principal Data Lead (Data Engineering with Full-Stack Dev) - Investment Systems
Hands-on technical leadership role at KKR (Gurugram, 4 days/week in office) owning end-to-end architecture of the firm's Reference Data Platform — an event-driven master-data system of record for parties, instruments, portfolios, and accounts. Core stack: Python, Spark, Kafka/CDC, Iceberg/Snowflake lakehouse, Terraform/Kubernetes, plus GenAI applied to data.
- Compensation
- Not disclosed
- Location
- Gurugram
- Work setup
- Hybrid
- Seniority
- Principal
- Employment
- Not specified
- Experience
- 12+ years
- Category
- Fullstack
- Posted
Skills
Must have
- Bachelor's or Master's degree in Computer Science, Engineering, or related field, or equivalent practical experience
- 12+ years professional software engineering experience, including significant time as technical lead or architect of data-intensive platforms
- Expert-level Python for production services, data pipelines, and platform frameworks
- Experience architecting MDM or reference/entity-data systems: entity resolution, survivorship, golden records, crosswalk, lineage
- Deep data engineering: data modeling, ETL/ELT design, distributed processing (Spark), data distribution patterns at scale
- Experience designing API-first services and event-driven/streaming architectures (Kafka, CDC)
- Strong command of modern data storage — relational, non-relational, lakehouse formats (Apache Iceberg) — and SQL
- Experience with infrastructure-as-code (Terraform) and cloud-native deployment (containers, Kubernetes/EKS, CI/CD)
- Deep domain experience with reference and master data in the financial industry — parties/legal entities, instruments, portfolios, accounts
- Hands-on experience integrating and mastering market data and vendor/administrator feeds (Bloomberg, exchange, fund-servicing sources)
- Practical experience applying Generative AI/LLMs to data platforms (grounded retrieval, entity-resolution/data-quality assistance, AI-ready data access with provenance)
- Track record of setting technical direction, driving architecture across teams, and mentoring senior engineers
Nice to have
- Familiarity with bitemporal/point-in-time data modeling and governed, auditable data systems
- Experience with workflow orchestration (Dagster, Airflow) and lakehouse catalogs (Polaris, Nessie, Glue)
- Exposure to commercial or vendor MDM/ontology platforms (e.g., Palantir Foundry) integrated as pluggable engines
- Experience building agentic workflows or RAG systems on enterprise data; familiarity with LLM evaluation, guardrails, and grounding
- Experience with legal-entity identifiers and standards (LEI, GLEIF) and regulatory reference-data requirements
Full description
COMPANY OVERVIEW
KKR is a leading global investment firm that offers alternative asset management as well as capital markets and insurance solutions. KKR aims to generate attractive investment returns by following a patient and disciplined investment approach, employing world-class people, and supporting growth in its portfolio companies and communities. KKR sponsors investment funds that invest in private equity, credit and real assets and has strategic partners that manage hedge funds. KKR’s insurance subsidiaries offer retirement, life and reinsurance products under the management of Global Atlantic Financial Group. References to KKR’s investments may include the activities of its sponsored funds and insurance subsidiaries.
POSITION SUMMARY
We are seeking a Principal Data Lead to set the technical direction of our Reference Data Platform — the system of record that masters, governs, and distributes the firm’s core reference data (parties, instruments, portfolios, and accounts. This is a hands-on technical leadership role: you will own the end-to-end architecture of a layered, event-driven data platform — spanning sourcing, ingestion, entity mastering, a bitemporal golden copy, and an API-first distribution layer — and drive it from vision to production. You will be the senior-most engineering voice on reference data, shaping platform strategy, establishing engineering standards across teams, and partnering with product, data, and infrastructure leaders to deliver a scalable, vendor-agnostic master data capability. The ideal candidate is an expert Python engineer and data architect with deep experience in master data management (MDM) and reference data within the financial industry — including market data and vendor feeds — combined with hands-on experience applying Generative AI to data platforms, distributed data processing, and building reusable, independently scalable platform components.
We are operating in a 4-day in office, 1-day flexible work arrangement.
ROLES & RESPONSIBILITIES
- Own the architecture of a layered reference-data platform, defining the contracts between sourcing, ingestion, mastering, golden-copy, and distribution layers so each is independently buildable, deployable, and scalable.
- Set technical direction for entity mastering — standardization, blocking, matching, survivorship, and crosswalk/lineage — designing rules as configuration (data, not code) that scale from exact-match fast paths to full-population fuzzy resolution.
- Architect the durable golden copy: a bitemporal, point-in-time source of truth with match evidence, per-attribute provenance, and a schema-last serving model that adapts to new sources and domains without migration.
- Define the event-driven backbone and cadence model (start-of-day, intraday, end-of-day, and on-demand) enabling batch today and real-time streaming (CDC / Kafka) without re-architecture.