Principal Software Engineer, ML & Distributed Systems

Overview

We are looking for a Principal Software Engineer to design, build, and operationalize machine learning systems and hyperscale distributed services that power Microsoft's AI experiences. This is a role for an engineer who is equally at home training and shipping ML models and building the large-scale distributed infrastructure that serves them — and who has the appetite to do both. You will own systems end to end: from model development and evaluation, through the serving and orchestration layers, to reliable operation at hyperscale.

MSN is a personalized content feed powering user experiences across Microsoft. Our mission is to empower every person on the planet to be informed, entertained, and inspired. With nearly 30 years of history, MSN has evolved into a premier content destination with high-quality content, AI-powered user-controlled personalization, and massive global reach. Over the past 4 years, AI and Machine Learning technologies have fueled massive growth, transforming MSN’s content moderation, personalization, and content entry points.Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.

Starting January 26, 2026, Microsoft AI (MAI) employees who live within a 50- mile commute of a designated Microsoft office in the U.S. or 25-mile commute of a non-U.S., country-specific location are expected to work from the office at least four days per week. This expectation is subject to local law and may vary by jurisdiction.

Responsibilities

• Design, build, and operationalize scalable ML and deep learning models using containers and orchestration platforms (e.g., Kubernetes).
• Develop and refine LLM prompt and fine-tuning strategies, build evaluation pipelines, and continuously optimize model quality, latency, and cost.
• Architect, implement, and operate multi-tiered distributed services at hyperscale, with high availability, fault tolerance, and low latency.
• Build model serving and inference infrastructure, including caching, batching, GPU capacity management, and A/B experimentation at scale.
• Design scalable APIs, data pipelines, and feature/signal stores that ensure efficient, secure, and reliable data flow between ML systems and product surfaces.
• Drive live-site excellence: instrumentation, monitoring, capacity planning, and incident response for ML-backed services.
• Collaborate with applied scientists, data scientists, backend engineers, and product teams to translate requirements into production ML systems.
• Participate in code reviews and architectural discussions, and mentor engineers across both ML and systems disciplines.