Inference Systems Engineer: Linux Kernel Test and Validation
- Compensation
- $140K–$180K / yr
- Location
- Not listed
- Work setup
- Remote
- Seniority
- Not specified
- Employment
- Not specified
- Experience
- 3+ years
- Category
- Software engineering
- Posted
Skills
Must have
- 5-10 years of software engineering experience with Linux Kernel. Experience building internal tooling at a large company with recent experience.
- Understanding of NVIDIA NCCL and distributed systems in the context of AI for training and inference. Experience developing for multi-node and multi-GPU systems.
- Experience with inferencing stack including NVIDIA Dynamo, llm-d, and engines (sglang, vllm, etc).
- Kernel-level depth: reads kernel and driver source, and root-causes defects at the kernel/driver/syscall boundary rather than only reproducing them at the surface. Comfortable operating below the abst
- Proven expert-level mastery of at least one performance- or correctness-critical Linux systems domain (networking data plane, virtualization/hypervisor, storage and block I/O, scheduling, memory manag
- Performance and latency engineering: characterizes throughput, jitter, and tail latency; uses tracing and profiling tooling (perf, ftrace, eBPF, or equivalent) to localize bottlenecks to a code path.
- Systems-level QE: designs and owns automated test infrastructure, performance and regression harnesses, and reproduces kernel-level races and corner cases. Not manual or UI QA.
- Enterprise Linux distribution environment (RHEL or equivalent): version matrices, backports, customer-grade triage.
- Principal scope: owns test or validation strategy for a subsystem; sets technical direction; mentors.
Full description
Introducing Cedana
The Problem
Inference is increasingly the dominant GPU workload and GPUs remain scarce and expensive. Every cold start, idle replica, and node failure is throughput that reduces revenue per GPU/watt. Data centers are bottlenecked by power requirements and GPU availability. Inference providers are measured on tokens per GPU, latency under load, and uptime, yet the complexity of models, hardware, and operations is a continuous and evolving challenge. Cluster productivity directly determines margin, research output, and productivity per MW.
Cedana’s Solution
Cedana mission is to accelerate and expand the ROI of every GPU. We maximize inference performance, reliability, and throughput per GPU with a breakthrough form of real-time GPU virtualization. We capture the full state of a running GPU workload and can manipulate it in real-time to enabling full workload migration, throughput and power optimization, and more. The result is more tokens from the same GPUs, faster time to first token, higher availability, higher power efficiency. Our system operates at the kernel and CUDA driver level, requires no code or config changes, and works seamlessly with Kubernetes, SLURM, and NVIDIA Dynamo. We work with the leading inference providers, and we deploy into neoclouds, enterprises, and research clusters.
The Team
Cedana's founding team has spent over a decade making computation run fast, productively, and reliably for AI. Our team has PhDs and MS from Caltech, CMU, Cambridge University, and have published across NeurIPS, CVPR, and Nature. We published some of the earliest formal methods for guaranteeing convergence in distributed training. At Shopify, we've developed a control plane for robotics fleets used in warehouse automation. We bring repeat founder experience, having built and exited a Series B healthcare AI company.
Backed by Y Combinator, Initialized Capital, Pebblebed (founders of OpenAI and Facebook AI Research), Keith Adams (engineer #20 at VMware, founded HHVM and FAIR at Facebook, Chief Architect at Slack), Venture Guides, Garry Tan, and Gokul Rajaram.
More about Cedana
Cedana has secured partnerships with established AI / HPC systems integrators and OEMs, is in production with leading inference providers, research institutions and enterprises. Check out our co-published work with NVIDIA and measured results on 3rd party benchmarks.
The Role
What you’ll own
You will own the core components of our inference virtualization stack including production, test and validation for kernel development. This includes measurement and optimization. Over time, you will contribute to critical core components of our system, with a focus on new capabilities, performance, and reliability.