Senior Software Engineer, Private Cloud Compute Inference Routing

Apple is looking for a senior software engineer to build the next generation of inference routing for Private Cloud Compute, the backbone of Siri AI and Apple Intelligence. You'll design distributed systems that direct inference requests across a global fleet of inference servers, balancing latency, availability, and compute efficiency while preserving PCC's privacy guarantees. Your work will directly shape how reliably and efficiently Apple delivers cloud inference at scale.

You'll join iCloud Platform Engineering team. We build the server stack behind Push Notifications, iMessage, FaceTime, iCloud, iCloud Private Relay and Private Cloud Compute. Our internet scale distributed systems deliver real-time experiences with very low latency and very high availability. They reach hundreds of millions of users across a billion devices worldwide and help them connect and share their lives securely and privately. Our work spans identity, messaging, queueing, presence, routing, relays, developer APIs and much more. We're a small, highly collaborative team with a large impact.

Routing LLM inference at Apple's scale requires systems that adapt to changing demand, available capacity, and failures while maintaining low latency and high utilization. PCC adds a fundamental architectural requirement: request content remains encrypted outside trusted compute nodes. You'll develop routing and scheduling strategies within this privacy model, working with inference and security teams to evaluate techniques such as cache aware request placement and prefill/decode disaggregation.

You'll design system architectures, specify custom network protocols and write high performance code. You'll also build scaling strategies for services used by hundreds of millions of people. Our team is small and highly visible, so your chances to make an impact are wide open. If you take opportunities like that and run with them, you'll enjoy working here.

You'll own technical decisions from early design through production, using benchmarks and operational evidence to evaluate tradeoffs. You'll work closely with model serving, infrastructure, security, and SRE engineers to turn new inference requirements into reliable routing capabilities.

Minimum Qualifications

• 3+ years of professional experience designing, building and operating large distributed systems in production

• Server development in Java, C/C++, C#, Go or Swift

• Knowledge of LLM inference serving, including inference request routing, KV-cache placement and reuse (e.g., prefix-aware routing), prefill/decode disaggregation, and the tradeoffs between latency, throughput and accelerator use

• Experience with network protocols, application level networking and client-server architectures, including request routing, load balancing or scheduling at scale