Creatify
Member of Technical Staff, Applied Research
Applied-research engineer at an AI startup building environments, evals, and datasets for frontier AI labs: designs and runs post-training workflows (SFT, preference optimization, GRPO), builds agent environments, reward/eval systems, and distributed training infrastructure. Core stack: Python, PyTorch, FSDP, FastAPI, GPU clusters.
- Compensation
- Not disclosed
- Location
- New York, NY
- Work setup
- Hybrid
- Seniority
- Not specified
- Employment
- Not specified
- Experience
- 3+ years
- Category
- Software engineering
- Posted
Skills
Must have
- Masters/PhD-level knowledge of machine learning fundamentals (linear algebra, optimization, SGD)
- Hands-on post-training experience: LLM fine-tuning, preference data, reward modeling, or RL for language models
- Interest/instinct for agent environments, multi-turn tool use, sandboxed tasks, and eval harnesses
- Strong Python fluency; comfort with PyTorch, FastAPI, and modern ML infrastructure
- Strong engineering judgment; able to move between research ambiguity and production constraints
Nice to have
- Roughly 3-5 years of experience (no hard rule)
Full description
About Us
Chakra Labs' mission is to encode human taste into intelligence. We build high-fidelity environments, evals, and datasets for frontier AI research, working with several of the top labs.
Our work sits at the frontier of post-training, agent environments, data quality, and research infrastructure. We care about building systems that make models better in ways that are measurable, useful, and hard to fake.
What You’d Work On
- Post-training loops. You’d help design and run model improvement workflows across supervised fine-tuning, preference optimization, and reinforcement learning approaches like GRPO. The work is not just launching training jobs; it’s figuring out what signal matters, how to collect it, and whether the model actually improved.
- Environment and task design. We build environments that feel real and scenarios that push agents past static benchmark behavior. You’d design tasks, tools, validators, reward signals, and evaluation harnesses that test meaningful capabilities instead of whatever is easiest to measure.
- High-fidelity trajectories. You’d create, inspect, and improve the data that teaches models how to behave. That means caring about taste, correctness, edge cases, and whether a trajectory would actually help a frontier model learn.
- Reward and evaluation systems. You’d work on reward functions, rubrics, validators, and analysis tools that turn messy model behavior into useful training signal. You should be interested in where evals lie, where rewards get hacked, and how to make measurements more robust.
- Training and research infrastructure. You’d run experiments across distributed GPU clusters, work with PyTorch and FSDP, and build the infrastructure needed to support model training, evaluation, and data generation at scale.
- Customer research problems. You’d work with frontier AI labs to translate ambiguous research goals into concrete environments, datasets, experiments, and deliverables.
About You
- Machine learning fundamentals. You have Masters / PhD-level knowledge of machine learning fundamentals. You understand linear algebra, optimization, stochastic gradient descent, and can reason from first principles when a model or training run behaves unexpectedly.
- Hands-on post-training experience. You have worked with or deeply understand LLM fine-tuning, preference data, reward modeling, or reinforcement learning for language models. You know the difference between reproducing a recipe and understanding why it works.
- Environment-building instinct. You are excited by agent environments, multi-turn tool use, sandboxed tasks, eval harnesses, and the question of how to test capabilities that do not fit neatly into a benchmark.