Member of Technical Staff, Inference

Mount Thor is hiring a founding inference engineer to build a production inference platform on Apple Silicon. You will own the stack from GPU kernels and model execution to distributed scheduling, networking, and customer-facing serving systems.

Who We Are

Mount Thor makes Apple hardware—macOS and Apple Silicon—available at datacenter scale for AI workloads. We build the infrastructure around that hardware to deliver elastic compute through developer-friendly interfaces.

We are building an inference platform that gives developers a straightforward way to deploy and serve models on Apple Silicon, backed by deep engineering across hardware, runtimes, and distributed systems.

The Role

You will establish the technical foundation for inference at Mount Thor. You will define the architecture, write the performance-critical code, and take the system through deployment and production operation.

Your scope spans the complete path from an incoming inference request to execution on the hardware and a response back to the customer. You will work across kernels, compilers, inference engines, memory management, model parallelism, networking, scheduling, and serving APIs. You will make decisions about where to extend existing open-source systems and where to build new components.

You will work closely with Platform, Fleet, and Network Infrastructure engineering to turn individual machines into a reliable inference service. You will own the tradeoffs among latency, throughput, model quality, reliability, and cost, and use measurements to guide the work.

In this role you will

• Build and optimize the inference engine. Implement model execution, continuous batching, prefill and decode scheduling, KV-cache management, prefix caching, and memory allocation. Bring new model architectures into production and implement optimizations such as quantization, speculative decoding, and chunked prefill.

• Write performance-critical kernels and runtime code. Implement and optimize operations such as matrix multiplication, attention, and mixture-of-experts execution. Work across Metal, MLX, and compiler and runtime internals to improve memory access, operator fusion, synchronization, and CPU/GPU execution. Validate numerical correctness alongside performance.

• Design for Apple Silicon. Build around unified memory, memory bandwidth, compute capabilities, and operating-system behavior. Profile execution on real hardware, diagnose bottlenecks, and choose model representations and execution strategies based on measured results.

• Build distributed inference and its communication layer. Implement model sharding and parallel execution across machines. Develop and optimize collective communication, data movement, and the overlap between computation and communication. Work with network engineers on transport performance, topology, and failure handling.