GPU Infrastructure Engineer
Owns scaling PolarGrid's distributed GPU inference network: bringing new sites and nodes online, running the global request-routing layer, and hitting availability targets. Heavy focus on bare-metal GPU fleets, LLM inference engines (vLLM, SGLang, TensorRT-LLM), networking (BGP, DNS, InfiniBand), observability, and Go/Rust automation tooling.
- Compensation
- CA$150K–CA$190K / yr
- Location
- Ottawa
- Work setup
- Not specified
- Seniority
- Not specified
- Employment
- Not specified
- Experience
- Not specified
- Category
- Devops
- Posted
Skills
Must have
- Experience operating distributed systems across multiple sites or regions with high availability targets
- Strong networking fundamentals: BGP, DNS, TCP/TLS and load balancing
- Hands-on experience running bare-metal GPU fleets: provisioning, burn-in, firmware and node lifecycle
- InfiniBand or RDMA networking experience
- Experience running LLM inference engines such as vLLM, SGLang or TensorRT-LLM, including multi-node serving, cold starts and TTFT
- Deep observability skills focused on p95/p99 latency and gray failures, not just uptime
- Experience designing failover, graceful degradation and automated recovery
- Strong automation and infrastructure-as-code mindset, including CI/CD for infrastructure
- Ability to plan capacity on constrained, uneven hardware supply
- Experience building internal tooling or control planes, ideally in Go or Rust
Nice to have
- DDoS mitigation experience
- Running DNS infrastructure at scale
- Hardware security: TPM, measured boot or confidential computing
- Real-time streaming inference, including voice and audio
- Operating remote datacenter sites with on-site technicians
- Certificate automation across many domains
- Deterministic simulation testing
Full description
Compensation: $150k – $190k CAD
About PolarGrid
PolarGrid is an early-stage Canadian AI infrastructure company building a distributed inference network. We run LLM and voice inference workloads on GPU nodes at the edge, with supporting backend services hosted on AWS.
Our goal is to make AI inference fast, reliable, and secure for customers running production workloads.
The Role
This role is responsible for scaling PolarGrid's inference network from a handful of sites to a geographically distributed fleet of GPU clusters. The GPU Infrastructure Engineer will own how new sites and nodes come online, from provisioning and burn-in through validation and a staged move into production. They will define when a site is ready for customer traffic, and make sure each new site joins the network without hurting latency or reliability.
A core responsibility is running the global routing layer that sends each request to the best place to serve it. The GPU Infrastructure Engineer will work with live signals such as network latency, capacity and health, and tune routing against real traffic to keep latency low and predictable.
The role also owns reliability across the network. The GPU Infrastructure Engineer will set and track availability targets, build the health checks and automation that detect and route around failing hardware and models, keep recovery fast, and test failure handling regularly.
The GPU Infrastructure Engineer will build the observability, release and network systems needed to run a distributed GPU fleet without operations work growing with every new site. This includes end-to-end tracing and latency alerting, safe staged rollouts, network configuration as code, and security and DDoS protection at the edge.
Requirements
-
Experience operating distributed systems across multiple sites or regions with high availability targets
-
Strong networking fundamentals: BGP, DNS, TCP/TLS and load balancing
-
Hands-on experience running bare-metal GPU fleets: provisioning, burn-in, firmware and node lifecycle
-
InfiniBand or RDMA networking experience
-
Experience running LLM inference engines such as vLLM, SGLang or TensorRT-LLM, including multi-node serving, cold starts and TTFT
-
Deep observability skills focused on p95/p99 latency and gray failures, not just uptime
-
Experience designing failover, graceful degradation and automated recovery
-Strong automation and infrastructure-as-code mindset, including CI/CD for infrastructure