Senior Manager - Data Quality and Evaluation
- Compensation
- Not disclosed
- Location
- Not listed
- Work setup
- Remote
- Seniority
- Senior
- Employment
- Not specified
- Experience
- 5+ years
- Category
- Not specified
- Posted
Skills
Must have
- 5+ years of experience in quality operations, data operations, AI data services, localization quality, annotation quality, evaluation operations, trust and safety quality, or a related field.
- Experience managing quality programs for complex customer accounts or high-volume operational delivery.
- Strong understanding of QA methodologies, calibration, sampling, adjudication, error analysis, and performance reporting.
- Experience working cross-functionally with delivery, operations, supply chain, sales, and customer-facing teams.
- Strong analytical skills and the ability to turn quality data into clear operational improvements.
- Excellent written and verbal communication skills, including the ability to communicate quality issues clearly to customers and senior stakeholders.
- Comfort operating in fast-moving, ambiguous environments where processes are still being built.
- Strong people leadership skills with experience coaching quality specialists, reviewers, annotators, or operational teams.
Nice to have
- Experience with AI data, RLHF, model evaluation, LLM evaluation, speech/audio evaluation, transcription, coding evaluation, multilingual evaluation, or expert review programs.
- Experience designing rubrics, annotation guidelines, evaluation instructions, reviewer training, calibration workflows, or quality scorecards.
- Experience supporting AI labs, enterprise AI teams, research teams, or technical customers.
- Familiarity with human-in-the-loop data workflows, annotation platforms, QA tooling, dashboards, and data labeling operations.
- Experience working with expert contributors, linguists, annotators, domain specialists, technical reviewers, or distributed talent networks.
- Knowledge of multilingual evaluation, speech/audio QA, cultural appropriateness, or language-specific quality risks.
Full description
This is a remote position.
Senior Manager, Data Quality & Evaluation
Remote
About Our Client
Our Client is a global language, data, and AI services company helping leading organizations build, evaluate, and improve AI systems.
Their Data Services team partners with AI labs, technology companies, and enterprise AI teams on complex human data programs across multilingual evaluation, speech and audio, model response evaluation, expert review, annotation, and emerging agentic workflows.
As AI systems become more capable and more complex, high-quality human evaluation is becoming a critical part of how models are trained, tested, and improved. They are building a Data Services organization focused on quality, scalability, operational excellence, and customer trust.
Summary of the role
The Senior Manager, Data Quality helps build the quality engine behind Our Client’s AI Data Services business.
In this role, you will define how we evaluate, calibrate, measure, and scale high-quality human data programs for leading AI companies. You will design quality frameworks for data collection, annotation, evaluation, review, calibration, adjudication, and customer reporting.
You will collaborate closely with Program Management, Supply Chain, Solutions, Sales, and customer-facing teams to turn customer requirements into clear, scalable quality workflows. You will help ensure that every program has the right evaluation methodology, task instructions, reviewer calibration, sampling approach, escalation process, and performance reporting in place.
This is a high-impact role for someone who enjoys building systems, improving quality, working cross-functionally, and operating in a fast-moving AI services environment.
ResponsibilitiesBuild quality systems for AI data programs
- Design and manage quality frameworks for AI data and evaluation programs.
- Translate customer requirements into clear quality standards, rubrics, acceptance criteria, review processes, and KPIs.
- Build quality workflows that are practical, scalable, and trusted by customers as programs move from pilot to production.
- Identify quality risks early and work with delivery teams to resolve issues before they impact timelines, customer confidence, or program outcomes.
- Create repeatable quality processes across calibration, QA sampling, adjudication, reviewer performance tracking, and customer reporting.
Lead evaluation, calibration, and QA processes
- Support quality operations across multilingual evaluation, speech/audio QA, transcription, data annotation, human preference evaluation, expert review, model response evaluation, coding evaluation, tool-use evaluation, and agent workflow evaluation.