Join EY's Regulatory Compliance team as a Senior AI Model Evaluation Consultant. This role focuses on validating AI/ML models, Generative AI, and multi-agent systems within GxP-regulated environments. You will ensure AI solutions are reliable, fair, robust, and compliant, working with cross-functional teams to deliver production-ready, inspection-ready AI systems.

Your expertise will be crucial in leading AI model evaluation across the entire lifecycle, from development to post-deployment monitoring. You'll embed Responsible AI, quality-by-design, and governance-by-design principles to ensure the safe and trustworthy use of AI in regulated industries.

Key responsibilities include supporting AI/ML and Agentic AI strategy development, performing comprehensive model evaluation and validation, and assessing model performance using various metrics. You will analyze model behavior across datasets to identify bias and instability, and conduct hands-on evaluation of GenAI/LLM systems. Designing and executing testing strategies for AI agents and multi-agent workflows, utilizing tools like LangChain, LangGraph, and Langfuse, is essential. You will also collaborate with data scientists to identify improvements and document findings in an audit-ready format.

The ideal candidate possesses a Bachelor's or Master's degree in Life Sciences, Engineering, or a related field, with over 5 years of experience in AI/ML, GenAI, software testing, validation, or quality engineering. Experience with cloud ecosystems (Azure, AWS, GCP) and client-facing roles is preferred. Strong skills in ML models, LLMs, GenAI evaluation techniques, LangChain, LangGraph, and Langfuse are mandatory. Excellent documentation, analytical, and stakeholder communication skills are required. Familiarity with GxP regulations, GAMP 5, SDLC, data integrity, and change control processes is also necessary.