Backend Engineer for Inference Optimization
Backend engineer optimizing model inference for Cohere's enterprise API — building low-latency, high-throughput serving infrastructure and batching strategies. Core stack: Go, Rust, or C++, with gRPC, REST APIs, and microservices.
- Compensation
- Not disclosed
- Location
- Toronto, Canada
- Work setup
- Remote
- Seniority
- Not specified
- Employment
- Not specified
- Experience
- 5+ years
- Category
- Backend
- Posted
Skills
Must have
- 5+ years of relevant backend engineering experience
- Proficiency in Go, Rust, or C++
- Experience with low-latency and high-throughput systems
- Knowledge of model serving optimization
- Familiarity with gRPC, REST APIs, and microservices
Full description
Cohere is looking for a Backend Engineer to specialize in optimizing model inference systems. In this remote-first role, you will create low-latency serving infrastructure for advanced backend services.
Candidates for this position must have over five years of backend engineering experience and strong programming skills in Go, Rust, or C++. You'll work on building backend services that enhance Cohere's enterprise API, focusing on batching strategies and performance optimization.
Key Responsibilities:
- Establish low-latency serving infrastructure for model inference
- Develop backend services for the enterprise-level API
- Implement batching strategies to maximize throughput efficiency
- Optimize model serving and inference processes
- Collaborate within a remote-first culture
Requirements:
- 5+ years of relevant backend engineering experience
- Proficiency in Go, Rust, or C++
- Experience with low-latency and high-throughput systems
- Knowledge of model serving optimization
- Familiarity with gRPC, REST APIs, and microservices
Utilize your expertise in backend engineering to drive innovations in model inference at Cohere.