Staff / Principal Software Engineer, Inference Platform - USA
- Compensation
- $280K–$350K / yr
- Location
- Mountain View, California, USA
- Work setup
- Hybrid
- Seniority
- Principal
- Employment
- Full time
- Experience
- 8+ years
- Category
- Software engineering
- Posted
Skills
Must have
- 8+ years building backend or distributed systems in Go, Rust, Java or C++.
- You've designed and shipped a multi-service system end to end, and can explain its trade-offs and what broke in production.
- Deep understanding of distributed systems: consistency, failure modes, queuing, load balancing and capacity modelling.
- Experience building a platform or developer product with external customers: cloud compute, PaaS, serverless, or an API platform.
- Clear technical writing. You can turn an ambiguous problem into a design doc that others build from.
Nice to have
- Schedulers or resource managers for scarce hardware (GPU clusters, batch systems, cloud compute).
- LLM inference serving (vLLM, SGLang, TensorRT-LLM) or ML training infrastructure.
- Multi-tenant systems with usage-based billing or contractual SLAs.
- Comfortable with Kubernetes, Terraform and GitOps (ArgoCD).
Full description
About Inworld
Inworld is a research lab and inference provider focused on realtime AI for consumer-facing applications. We build first-party speech models, serve LLMs, and run the inference behind modular APIs designed for high-volume, realtime workloads.
Hundreds of millions of users interact with Inworld powered apps every day and we serve over 10 trillion LLM tokens per month. Our models and infrastructure support consumer applications across companions, healthcare, fitness, education, media, and more. Our work spans model research, realtime inference, large-scale serving infrastructure, and the APIs developers use to bring these capabilities into production.
We’ve raised more than $125M from Lightspeed Venture Partners, Section 32, Kleiner Perkins, Microsoft’s M12 venture fund, Founders Fund, Meta, Stanford, and others. Our technology has powered experiences from companies including NVIDIA, Microsoft Xbox, Niantic, Logitech Streamlabs, Wishroll, Little Umbrella, and Bible Chat. Inworld has also been recognized by CB Insights as one of the 100 most promising AI companies globally and named one of LinkedIn’s Top 10 Startups in the USA.
About the role
Inworld Inference serves the models consumer apps run on. We're growing it from an API into a full platform: one place where a team can go from its first API call to its own model, tuned on its own traffic and running on capacity reserved for it.
As a Staff or Principal Software Engineer, you'll design that platform's core architecture and write much of its code. You'll define the abstractions every customer workload runs on, and own the system in production at tens of trillions of tokens a month.
Why this role
- A platform, not a feature. You'll shape the foundation that every future Inference product builds on. The architecture decisions are yours to make.
- Real scale from day one. Realtime inference for hundreds of millions of end users, with latency budgets measured in milliseconds.
- Hard problems. Scheduling scarce GPUs across many tenants and workloads, model lifecycles from weights to serving, and latency and uptime commitments customers pay for.
- Principal-level scope. You'll partner directly with research, product and leadership on where the platform goes next.
What you'll do
- Design and build the platform's core services: control plane, model and deployment lifecycle APIs, tenant isolation, metering and quotas.
- Own how GPU capacity is allocated and scheduled across shared and reserved workloads, with clear latency, cost and reliability trade-offs.
- Build the path that takes a customer's model from weights to production serving, and keeps it healthy as it changes.