Apple
AI/LLM Quality Engineer - Siri
- Compensation
- Not disclosed
- Location
- Beijing
- Work setup
- Not specified
- Seniority
- Not specified
- Employment
- Not specified
- Experience
- Not specified
- Category
- Ai engineering
- Posted
Skills
Must have
- BS/MS in Computer Science, Computer Engineering, or equivalent practical experience.
- Strong proficiency in at least one modern programming or scripting language (e.g., Python, Swift, Java, Go), with solid fundamentals in data structures, databases, SQL, and OS environments (Unix/Linux
- Hands-on experience architecting and implementing test automation frameworks for client-side applications, backend services, and end-to-end integration flows.
- Experience with CI/CD pipelines, containerization, and modern cloud/distributed infrastructure.
- Solid understanding of Generative AI / LLM concepts, with proven ability to build tools, scripts, or workflows utilizing model APIs.
- Strong analytical, logical, and debugging skills: capable of breaking down complex, ambiguous problems into testable hypotheses and investigating issues across diverse technical stacks.
- High adaptability and resilience to navigate rapidly shifting priorities in a fast-paced environment.
- Strong communication and interpersonal skills, with the ability to collaborate effectively across multi-functional and geographically distributed teams.
- Professional working proficiency in both written and spoken English and Mandarin.
Nice to have
- LLM-Based Lifecycle QA & Evaluation Pipelines: Proven track record in establishing an LLM-driven, full-lifecycle quality assurance system and automated evaluation pipelines (e.g., automated test case
- Model Quality Metrics & Bad Case Closed-Loop: Demonstrated ability to define model quality metrics and dashboards; systematically collect, categorize, and analyze bad cases (distinguishing hallucinati
- Evaluation & Benchmark Design: Experience designing evaluation schemes, curated benchmark datasets, adversarial/negative samples, and edge cases across conversational AI, RAG, tool-use/agents, and mul
- AI Productivity Tooling: Hands-on experience developing lightweight AI-driven developer productivity tools (e.g., agentic workflows, LLM-assisted code or test generation).
- Safety & Compliance: Experience in content safety, compliance, privacy, and responsible AI risk evaluation (e.g., bias, harmful content generation).
- Apple Ecosystem: Experience testing on Apple platforms (iOS, iPadOS, macOS, watchOS and etc).
- Soft Skills & Mindset: Self-motivated and proactive, with demonstrated creative and critical thinking capabilities.
- Language Skills: Fluency or proficiency in Cantonese is a strong plus.
Full description
We are looking for an exceptional Software Quality Engineer with hands-on experience and a strong technical background to elevate the quality bar for Siri features and intelligent services in Mainland China. In this role, you will champion product quality across the entire lifecycle, blend traditional QA rigor with cutting-edge Generative AI and LLM evaluation methodologies, and build innovative automation tools that push testing, evaluation, and operational efficiency to the next level.
As a Siri Quality Engineer, you will work at the intersection of AI, speech/language technologies, and customer experience. Your key responsibilities include:
Minimum Qualifications
- BS/MS in Computer Science, Computer Engineering, or equivalent practical experience.
- Strong proficiency in at least one modern programming or scripting language (e.g., Python, Swift, Java, Go), with solid fundamentals in data structures, databases, SQL, and OS environments (Unix/Linux/macOS).
- Hands-on experience architecting and implementing test automation frameworks for client-side applications, backend services, and end-to-end integration flows.
- Experience with CI/CD pipelines, containerization, and modern cloud/distributed infrastructure.
- Solid understanding of Generative AI / LLM concepts, with proven ability to build tools, scripts, or workflows utilizing model APIs.
- Strong analytical, logical, and debugging skills: capable of breaking down complex, ambiguous problems into testable hypotheses and investigating issues across diverse technical stacks.
- High adaptability and resilience to navigate rapidly shifting priorities in a fast-paced environment.
- Strong communication and interpersonal skills, with the ability to collaborate effectively across multi-functional and geographically distributed teams.
- Professional working proficiency in both written and spoken English and Mandarin.
Preferred Qualifications
- LLM-Based Lifecycle QA & Evaluation Pipelines: Proven track record in establishing an LLM-driven, full-lifecycle quality assurance system and automated evaluation pipelines (e.g., automated test case generation, offline model evaluation, self-healing E2E test automation, continuous online monitoring, and A/B experimentation quality assessment).
- Model Quality Metrics & Bad Case Closed-Loop: Demonstrated ability to define model quality metrics and dashboards; systematically collect, categorize, and analyze bad cases (distinguishing hallucinations, tool-invocation errors, retrieval failures, and prompt deficiencies); and partner closely with cross functional teams to drive closed-loop fixes.
- Evaluation & Benchmark Design: Experience designing evaluation schemes, curated benchmark datasets, adversarial/negative samples, and edge cases across conversational AI, RAG, tool-use/agents, and multimodal scenarios.