Tech Lead Manager, Inference

The Role

##

Inference is where all of Luma’s compute meets all of Luma’s users. The inference platform team owns the entire serving stack — from request routing, scheduling, and queueing to fleet-wide orchestration across thousands of GPUs spanning multiple clusters, clouds, and hardware vendors. The team has a dual mandate: maximize the efficiency, reliability, and unit economics of production inference for millions of users, and enable research to move fast — new model architectures should go from research checkpoint to production in days, and our serving stack increasingly powers training itself through online reinforcement learning.

We are hiring a Tech Lead Manager to lead this team through its next phase of growth. This is a hands-on leadership role, not a pure management position: we expect you to spend at least 50% of your time as an individual contributor — designing, building, and debugging in the serving stack — alongside hiring and growing the team, setting technical direction, and partnering across research, product, and infrastructure. You lead by shipping, and you set the technical bar for the team through your own work.

What You’ll Do

• Spend at least half your time hands-on in the serving stack: architect and build core platform components, own the hardest design decisions, and debug the toughest production incidents yourself
• Lead, grow, and develop the inference engineering team: own hiring, coaching, and career growth, and build the team’s operational culture — on-call, incident response, capacity planning, and postmortems
• Set the technical roadmap for the serving platform: model serving engines, request routing and scheduling, autoscaling, caching, observability, and deployment pipelines
• Own the platform’s SLOs and economics: latency and availability targets, GPU utilization, and cost per generation across every model we serve
• Partner closely with research to ship new model architectures into production on day zero, and to integrate serving into online RL and evaluation loops
• Manage and optimize inference workloads across heterogeneous fleets — multiple clusters, clouds, and GPU vendors — including capacity planning and hardware bring-up
• Build sophisticated scheduling and queueing systems that optimally leverage expensive GPU resources against live traffic patterns, cluster availability, and user priority

Representative Projects

• Design intelligent routing and scheduling that optimizes request distribution across thousands of GPUs in multiple regions and clouds
• Stand up disaggregated prefill/decode serving with tiered KV-cache reuse across GPU memory, DRAM, NVMe, and network storage
• Autoscale and hot-swap models across the fleet to dynamically match GPU supply with live demand across production, research, and experimental workloads