Member of Technical Staff (Model Quantization & Inference)
ML engineer/researcher at Lamb Labs developing extreme low-bit (1–2 bit) quantization and inference methods for large language models — covering weight/activation quantization, KV-cache and attention optimization, and QAT/PTQ/distillation — to make models run efficiently on the company's custom AI chip (MPU), working directly with FPGA/ASIC engineers. Core tools: Python and PyTorch.
- Compensation
- Not disclosed
- Location
- GB
- Work setup
- Onsite
- Seniority
- Not specified
- Employment
- Not specified
- Experience
- Not specified
- Category
- Software engineering
- Posted
Skills
Must have
- Deep understanding of modern transformer-based language models and their inference behavior
- Strong experience with model quantization, compression or low-precision numerical computation
- Hands-on experience working with sub-4-bit models, ideally including 1–2-bit quantization
- Understanding of weight and activation distributions, quantization error, outliers and numerical stability
- Strong Python and PyTorch skills, including ability to modify models, training procedures and inference implementations
- Experience designing and running rigorous experiments, diagnosing model quality regressions and evaluating trade-offs
- Ability to read research papers critically, develop original ideas and turn them into working implementations
- High degree of independence, curiosity and ownership
Nice to have
- PhD or equivalent depth of research and engineering experience in ML, computer architecture, numerical optimization or related field
- Experience with 1-bit, 1.58-bit, binary, ternary or other extreme low-bit language models
- Research or implementation experience with BitNet, GPTQ, AWQ, QuIP#, AQLM, HQQ or related quantization methods
- Quantization-aware training, post-training quantization, knowledge distillation or low-bit fine-tuning
- KV-cache quantization, compressed attention or long-context inference optimization
- Activation quantization, outlier mitigation, mixed-precision methods or alternative numerical formats
- Developing or modifying transformer architectures for efficient inference
- Custom inference kernels using CUDA, Triton or similar low-level frameworks
- Experience with inference systems such as vLLM, SGLang or llama.cpp
- Hardware-aware ML, accelerator-aware quantization or collaboration with FPGA/ASIC teams
- Publications, open-source projects or substantial original technical contributions in these areas
Full description
About Lamb Labs
At Lamb Labs, we're building a new AI chip: the Model Processing Unit (MPU).
Modern AI inference is constrained by the cost of moving model weights between memory and compute. We're taking a fundamentally different approach: aggressively compressing and optimizing language models so that their weights and computation can be encoded directly into silicon.
We're developing extremely low-bit language models and custom hardware architectures together, with the goal of making inference orders of magnitude faster and more energy-efficient.
We're a small, research-driven team working at the intersection of machine learning, compilers and computer architecture. We care about fundamental ideas, rigorous experimentation and turning research into working systems.
The Role
We're looking for an exceptional ML engineer or researcher to work on some of the hardest problems in low-bit language model inference.
You'll help develop the models and algorithms that make our hardware architecture possible.
Our focus is on pushing language models to extremely low precision, particularly 1–2-bit weights, while preserving as much model quality as possible. But this goes well beyond weight quantization.
You'll work across activation quantization, KV-cache optimization, attention, numerical representations, training and fine-tuning, and the broader challenges of running highly compressed language models efficiently.
We're not looking for someone who simply knows how to use existing quantization libraries. We want someone who understands these methods deeply, can identify their limitations, and can develop better ones.
You'll have broad ownership of the problem, from understanding the mathematical and numerical behavior of models to implementing new techniques, running experiments and working directly with our hardware team.
At Lamb Labs, the model and the chip are designed together. Your research will directly influence how we build our silicon.
What You'll Work On
Extreme low-bit quantization
- Develop and improve methods for quantizing large language models to sub-4-bit precision, particularly 1–2 bits per weight.
- Investigate binary, ternary, mixed-precision and other hardware-efficient representations.
- Understand and overcome the quality degradation caused by aggressive quantization.
Activations, attention and KV-cache
- Explore low-bit activation quantization, including outlier handling, scaling, dynamic ranges and numerical stability.
- Develop methods for reducing KV-cache memory requirements while preserving attention quality and long-context capabilities.