Quality Assurance Automation Engineer - AI/ML Testing

Role Overview :

We are seeking a seasoned QA Automation Engineer with a specialized focus on AI/ML testing to join our high-growth SaaS engineering team. In this role, you will be responsible for architecting and executing robust automation frameworks that ensure the reliability and performance of complex AI-driven features. You will collaborate closely with data scientists, product managers, and backend developers to bridge the gap between traditional software quality and the non-deterministic nature of machine learning models. By implementing advanced testing strategies, you will directly influence the quality of our product releases, ensuring that our AI solutions deliver consistent, high-value outcomes for our global enterprise clients.

Key Responsibilities :

• Design and maintain scalable automation frameworks using Playwright and Python to validate complex SaaS workflows and UI components.

• Develop specialized test suites for AI/ML models to evaluate model accuracy, bias, and performance, ensuring our intelligent features meet rigorous production standards.

• Collaborate with cross-functional engineering teams to integrate automated testing into CI/CD pipelines, reducing manual intervention and accelerating deployment cycles.

• Analyze complex system behaviors and edge cases to proactively identify potential failures, safeguarding the user experience against regressions.

• Mentor junior team members on best practices in automation and AI testing methodologies to foster a culture of technical excellence and continuous improvement.

Required Skillset :

• Demonstrated expertise in building end-to-end automation frameworks using Playwright and Python within a fast-paced SaaS environment.

• Proven ability to design and execute test strategies for AI/ML-based applications, including data validation and model output verification.

• Strong analytical mindset with the ability to communicate complex technical quality metrics to both technical stakeholders and product leadership.

• Exceptional problem-solving skills and the ability to thrive in a remote-first work environment, maintaining high levels of autonomy and collaboration across distributed teams.

• A Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field, supported by 5 - 10 years of professional experience in software quality assurance.