AI Inference Platform Engineer
Build, run, and optimize DRW's firmwide AI inference platform serving LLM, vision, multimodal, and embedding models on NVIDIA GPUs. Day to day: onboarding new models, tuning inference runtimes (e.g., TensorRT-LLM, vLLM, SGLang), optimizing KV cache and distributed serving, and managing multi-tenant GPU scheduling for latency, throughput, and cost.
- Compensation
- $200K–$250K / yr
- Location
- Chicago
- Work setup
- Not specified
- Seniority
- Not specified
- Employment
- Not specified
- Experience
- Not specified
- Category
- Devops
- Posted
Skills
Must have
- Hands-on experience serving LLMs on NVIDIA GPUs, including Hopper/Blackwell architectures, HBM, Tensor Cores, NVLink/NVSwitch, and compute/memory bottlenecks
- Deep expertise in at least one modern inference runtime such as TensorRT-LLM, vLLM, or SGLang
- Practical knowledge of inference optimization techniques: continuous batching, scheduling, chunked prefill, speculative decoding, quantization, CUDA Graphs, paged attention
- Understanding of KV cache architecture: prefix caching, block management, sizing, eviction, quantization, cache-aware routing, multi-tier caching
- Experience measuring model quality equivalence across serving configurations, including evaluation harnesses and regression detection
- Experience designing and tuning distributed inference systems: tensor parallelism, multi-node deployments, disaggregated prefill and decode
- Experience with multi-tenant GPU scheduling, workload isolation, and QoS across concurrent inference workloads
- Proficiency with GPU performance and observability tooling such as Nsight, DCGM, OpenTelemetry, Prometheus, and Grafana
- Strong Linux and systems performance fundamentals, diagnosing bottlenecks across hardware, drivers, runtimes, networking, and application layers
- Production experience with model serving infrastructure, including CI/CD, automated testing, observability, and production readiness
Nice to have
- Measurement-driven approach to performance optimization
- Ability to evaluate unfamiliar models, runtimes, and hardware quickly and make sound engineering decisions with limited guidance
- Clear communication of complex performance tradeoffs across engineering teams
Full description
DRW is a diversified trading firm with over 3 decades of experience bringing sophisticated technology and exceptional people together to operate in markets around the world. We value autonomy and the ability to quickly pivot to capture opportunities, so we operate using our own capital and trading at our own risk.
Headquartered in Chicago with offices throughout the U.S., Canada, Europe, and Asia, we trade a variety of asset classes including Fixed Income, ETFs, Equities, FX, Commodities and Energy across all major global markets. We have also leveraged our expertise and technology to expand into three non-traditional strategies: real estate, venture capital and cryptoassets.
We operate with respect, curiosity and open minds. The people who thrive here share our belief that it’s not just what we do that matters–it's how we do it. DRW is a place of high expectations, integrity, innovation and a willingness to challenge consensus.
About the Role
We're looking for an AI Inference Platform Engineer to build, operate, and optimize the systems that serve large language, vision, multimodal, and embedding models across DRW. This role provides DRW's firmwide interface to modern AI models, from early evaluation through reliable production use.
You'll work across inference runtimes, distributed systems, and production platform engineering, with deep GPU literacy. You'll own the serving platform end-to-end: onboarding newly released models, measuring quality and performance equivalence across serving configurations, scheduling workloads across tenants, and continuously improving latency, throughput, utilization, reliability, and cost across the inference fleet.
What You'll Do
- Optimize LLM inference performance across modern NVIDIA GPU architectures and inference runtimes.
- Build end-to-end performance profiling and observability to identify bottlenecks from individual GPU kernels through multi-node inference systems.
- Design and optimize KV cache and distributed inference architectures, including caching, routing, memory tiering, and prefill/decode strategies.
- Own day-0 model onboarding, determining the appropriate runtime, precision, sharding, memory, batching, cache policy, and serving configuration for new models.
- Maintain validated performance profiles for important model and hardware combinations, including performance and quality regression testing.
- Measure and monitor quality equivalence across serving configurations, including KV cache quantization, speculative decoding acceptance thresholds, precision choices, and model routing, so in-house serving can be trusted to match reference-model quality on production workloads.