Advance your career with EY by becoming an AI QA Senior, a pivotal role in ensuring the quality, reliability, and safety of cutting-edge AI systems. This position is ideal for seasoned professionals with a deep understanding of AI testing, validation frameworks, and enterprise AI solution assurance. You'll be instrumental in validating AI systems, Agentic AI workflows, RAG applications, and conversational AI solutions.

This role demands practical experience in relevant AI technologies, with personal projects serving as a valuable supplement to professional application development. Candidates must be prepared to articulate their direct contributions to at least two significant AI implementations, detailing the business context, technical methodology, and achieved outcomes.

Spearhead the design, development, and execution of comprehensive testing strategies for AI, GenAI, Agentic AI, and RAG systems. Validate Large Language Model outputs for accuracy, consistency, safety, and compliance. Create robust test frameworks and automation suites for AI services, copilots, and enterprise AI applications. Design and implement evaluation frameworks for RAG systems, retrieval pipelines, and knowledge-based AI applications. Conduct diverse testing, including functional, integration, regression, performance, reliability, and scalability. Develop automated testing frameworks using Python to validate AI workflows, APIs, models, and orchestration. Assess AI model testing across conversational AI, multimodal AI, and intelligent automation. Perform crucial security validations like penetration testing, adversarial testing, and prompt injection testing. Evaluate AI solutions for bias, hallucinations, content quality, and responsible AI compliance. Ensure seamless integrations between AI systems, enterprise applications, APIs, databases, and cloud platforms. Utilize RAG evaluation frameworks like Ragas to measure retrieval quality and response effectiveness. Develop testing dashboards, quality metrics, and continuous improvement processes. Collaborate with AI engineers and architects to mitigate quality risks and enhance operational reliability.

A minimum of 5 to 9 years of overall experience is required, with at least 3 years dedicated to hands-on work with AI/ML, applied AI, or GenAI systems. Demonstrate proven experience in building production, pre-production, or enterprise pilot AI applications. Exhibit strong depth in at least three of the following areas, with significant ownership in at least two: LLM application development, retrieval pipelines, agent/tool orchestration, evaluation/guardrails, model adaptation/fine-tuning, AI application cloud deployment, conversational AI, multimodal AI, or backend engineering for AI systems. Possess a strong software engineering foundation for building reliable and maintainable AI services. Ability to clearly articulate direct contributions to delivered AI implementations is essential. An educational background including a Bachelor's or Master's degree in Computer Science, Data Science, Engineering, or a related field is necessary.