Member of Technical Staff, Post-Training
Early, high-ownership role at Handshake Labs defining its post-training practice: designing SFT, RL, and preference-optimization methodologies, evaluation frameworks, and data pipelines for frontier models, then turning research insights into reusable AI products and platforms. Core stack is Python and PyTorch, based in San Francisco/Mountain View.
- Compensation
- $200K–$350K / yr
- Location
- San Francisco, CA
- Work setup
- Hybrid
- Seniority
- Not specified
- Employment
- Full time
- Experience
- 3+ years
- Category
- Software engineering
- Posted
Skills
Must have
- 3+ years of demonstrated strength in post-training, fine-tuning, or model-evaluation work (RL, SFT, LoRA/PEFT, full fine-tuning, RLHF, DPO, PPO, reward modeling, or training environments)
- Strong Python skills and ability to write clean, efficient, scalable software
- Hands-on experience with modern ML tooling, particularly PyTorch and large-scale data, training, or evaluation workflows
- Sound experimental judgment: form hypotheses, choose meaningful metrics, diagnose failures, distinguish signal from noise
- Experience designing systems, including tradeoffs around quality, scale, reliability, and reuse
- Comfort operating in an ambiguous, fast-moving environment with substantial ownership
- Collaborative, low-ego communication with researchers, engineers, domain experts, and customers
Nice to have
- Building or operating large-scale ML training, inference, data, or evaluation systems
- Developing LLM/agent benchmarks, evaluation methodologies, annotation systems, or data-quality frameworks
- Research or applied work on reinforcement learning, alignment, model behavior, synthetic data, or human-in-the-loop systems
- Published research, meaningful open-source contributions, or evidence of technical leadership in ML systems or AI research
- Experience productizing research or repeated customer work into robust, reusable platforms
Full description
About Handshake
Handshake's mission is to organize expert human knowledge to advance the AI economy. Handshake AI works directly with frontier labs on their most consequential data, evaluation, and post-training challenges, building the systems that turn expert human knowledge into the data and evaluations that make frontier models better.
You will work alongside engineers, researchers, operators, and builders from organizations including Scale AI, Meta, Google, Amazon, xAI, Notion, and Palantir—and help build the systems that make expert human knowledge useful for advancing AI.
About Handshake Labs
Handshake Labs is building external AI products, research platforms, and customer-facing AI systems. We are evolving work that is often custom-built for an individual partner into reusable products and platforms that improve with every deployment.
Our work spans the full post-training loop: designing evaluations and training environments, building high-quality data and feedback systems, running experiments, and turning what works into durable infrastructure. For example, we are developing agents that can analyze long, complex coding-agent sessions in days rather than weeks—with expert review and calibration built into the system.
##
The Role
We are hiring a Member of Technical Staff, Post-Training to help define and build this new organization. This is a broad, high-ownership role for researchers who build. You may come from research science, research engineering, machine learning engineering, or a closely related background; what matters is the ability to reason deeply about model improvement and turn that reasoning into reliable systems.
You will partner with researchers, domain experts, and customers to turn ambiguous post-training questions into experiments, evaluation frameworks, data pipelines, and products. Early members of the team will have unusual influence over our technical direction, operating culture, and the reusable systems we build.
We care more about demonstrated research capability, technical judgment, and a builder’s mindset than a specific title, degree, or career path.
Location: San Francisco & Mountain View preferred; we are open to exceptional candidates in other locations.
##
What you’ll do
- Design post-training systems and methodologies for frontier models, including supervised fine-tuning, reinforcement learning, preference optimization, reward modeling, and related approaches.
- Translate open-ended research or partner needs into clear hypotheses, experiments, evaluation plans, and production-quality implementations.
- Build and improve evaluation frameworks, benchmarks, training environments, data-processing pipelines, and quality-control systems.