Backend Engineer LLM Inference

You will own backend architecture and build distributed services in Go, TypeScript or JavaScript, and Python where appropriate. You will develop OpenAI-compatible streaming APIs, route inference requests, optimize service performance, design data systems, implement authentication and billing integrations, test production systems, and mentor engineers.

Responsibilities

• Design, build, and maintain backend services in Go, TypeScript or JavaScript, and Python
• Build and scale reliable distributed systems
• Own features from design documentation through implementation, testing, and release
• Identify technical debt, bottlenecks, and reliability risks
• Design and maintain OpenAI-compatible APIs for chat completions, streaming, and tool calls
• Build token streaming over SSE and WebSockets
• Build request routing and load balancing across inference backends
• Implement backpressure, queueing, and admission control
• Design REST and gRPC interfaces between internal services
• Design and optimize PostgreSQL data models
• Use Redis or Valkey for caching, state, and high-performance workloads
• Integrate S3-compatible storage and Cloudflare R2
• Build authentication and authorization flows with Auth0 and JWT
• Build rate limiting and usage metering for multi-tenant APIs
• Build reliable Stripe and third-party integrations
• Optimize latency, throughput, and resource usage
• Write automated tests
• Mentor engineers and contribute to code reviews and design discussions

Requirements

• 5+ years of professional backend engineering experience
• Commercial experience with Go and TypeScript or JavaScript using Node.js
• Experience building and operating distributed systems
• Understanding of consistency, fault tolerance, and service-to-service communication
• Deep understanding of Go concurrency and the Node.js event loop and async model
• Strong PostgreSQL knowledge including schema design, indexing, query optimization, transactions, isolation levels, and locking
• Experience with Redis, Valkey, or similar in-memory stores
• Experience with LLM inference backends or understanding of token streaming and long-running requests
• Experience building streaming APIs using SSE or WebSockets
• Understanding of OpenAI-compatible API design
• Experience with request routing, load balancing, backpressure, and admission control
• Experience designing REST or gRPC APIs
• Experience building authentication systems with JWT, Auth0, or similar
• Experience leading systems or major features end to end
• Experience resolving complex production issues