MLPerf Senior Software Engineer
Senior contractor role at MLCommons designing and building the open-source infrastructure and automation behind the MLPerf AI benchmarks — submission, validation, review, reproducibility, and results visualization. Core stack: Python, Bash, GitHub Actions CI/CD, and Docker; fully remote, working with member companies.
- Compensation
- Not disclosed
- Location
- California
- Work setup
- Remote
- Seniority
- Senior
- Employment
- Contract
- Experience
- 4+ years
- Category
- Software engineering
- Posted
Skills
Must have
- Demonstrated experience designing systems and infrastructure — architecture decisions, trade-off analysis, building for scale and reliability
- 4+ years of post-academic software engineering (or equivalent impact via open source / industry)
- Proficiency in Python and Bash; strong automation mindset
- Hands-on experience with GitHub Actions (or equivalent CI/CD) and artifact/version management
- Working knowledge of containers (Docker) and running workloads reproducibly
- Familiarity with ML concepts (inference/training basics) and performance benchmarking or optimization
Nice to have
- Experience designing distributed systems, data pipelines, or large-scale automation infrastructure
- Infrastructure-as-code (e.g., Terraform) and cloud platforms
- Experience with MLPerf (reference or vendor implementations), LoadGen, and logging schemas
- ML frameworks: PyTorch, TensorFlow, ONNX Runtime
- Understanding of hardware for ML (CPUs/GPUs/TPUs/accelerators) and performance profiling
- Data validation, schema management, and results visualization experience
- Prior open-source contributions in benchmarking tools and communities
Full description
Compensation: No equity
About MLCommons Association
MLCommons is a new kind of organization for the AI era. We are a non-profit that builds the standards that define AI through open, state-of-the art, industry-standard benchmarks to measure quality, performance, and risk. We aren’t like standards organizations that slowly negotiate over specifications - we bring the community together to work on cutting edge problems and build the future with code.
We’re a global consortium supported by 125+ members and affiliates, including startups, leading companies, and academics. We are a fully remote organization that supports a flexible working environment; this requires mature communication, accountability, and transparent collaboration by all team members.
About MLPerf
The MLPerf family of benchmarks is the industry standard for measuring performance and efficiency of AI on everything from smartphones and laptops to the world’s largest supercomputers. Our open, fair, and trusted benchmarks help bring together the broader community, including researchers, engineers, purchasers, and vendors to enable more capable and efficient AI for the entire world.
The MLPerf team includes product managers, program managers, and engineers who work collaboratively with member organizations. Together, they build, operate, and sustain the MLPerf benchmarks, AI workload evaluation harnesses, and infrastructure that serve our members, their stakeholders, and the entire AI ecosystem.
You can read about MLPerf here:
IEEE Spectrum: Machine Learning Tests Keep Getting Bigger
AMD: AMD Delivers Breakthrough MLPerf Inference 6.0 Results
NVIDIA: NVIDIA Platform Delivers Lowest Token Cost Enabled by Extreme Co-Design
Scope of MLPerf Software Engineer
As an MLPerf Software Engineer, you will work in the open-source space, designing and building the systems and infrastructure that power the MLPerf benchmarks — the automation behind the benchmarks and the critical processes around submission, review, publication, and reproducibility, as well as the visualization of benchmarking results.
This is a role for an engineer who thinks in systems: someone who can own architecture and technical direction, weigh trade-offs, and design infrastructure that scales — not just implement against a spec. You will work alongside the MLPerf team and experts in performance optimization, benchmarking, and ML at our member organizations to shape the evolution and success of the MLPerf benchmarking suite.
Responsibilities
Architect and design benchmarking systems, infrastructure, and tooling — owning technical decisions from concept through implementation.