Join EY and build a remarkable career within our global network. As a Senior AI Model Evaluation Consultant within the Enterprise Risk Regulatory Compliance team, you will guide clients in understanding their needs and delivering innovative solutions. You will foster strong internal and external relationships, ensuring high-quality deliverables and operational efficiency.

This pivotal role involves leading AI model evaluation and validation throughout its lifecycle, from development to post-deployment monitoring. You will champion Responsible AI, quality-by-design, and governance-by-design, enabling safe, trustworthy, and explainable AI solutions in life sciences and pharmaceutical sectors.

We are seeking a detail-oriented professional with deep expertise in validating AI/ML models, Generative AI, and multi-agent workflows in GxP-regulated environments. Your work will ensure the reliability, fairness, robustness, and compliance of AI solutions, supporting cross-functional teams in creating production-ready, inspection-ready AI systems.

Support the development of AI/ML and agentic AI strategies by providing insightful analysis, validation, and readiness assessments.

Conduct comprehensive AI/ML model evaluation and validation across all project phases, including development, training, testing, and deployment.

Assess model performance using key metrics like precision, recall, accuracy, F1-score, and confusion matrices.

Evaluate model behavior across diverse datasets and edge cases to identify potential bias, drift, and instability.

Design and execute testing strategies for AI agents and multi-agent workflows, focusing on orchestration, tool invocation, and output validation.

Utilize LangChain, LangGraph, and Langfuse for robust agent workflows, observability, traceability, and comprehensive evaluation logging.

Document all evaluation findings, limitations, risks, and acceptance criteria in a clear, audit-ready format.

A Bachelor's or Master's degree in Life Sciences, Engineering, or a related field is required.

Possess at least 5 years of experience in AI/ML, Generative AI, software testing, validation, or quality engineering.

Demonstrated experience in creating user stories, test scripts, validation plans, or AI solution prototypes.

Familiarity with cloud ecosystems such as Azure, AWS, or GCP is beneficial.

Consulting or client-facing experience is highly preferred.

Possess excellent communication, documentation, and stakeholder management skills.

Strong understanding of ML models, LLMs, GenAI workflows, agentic systems, and evaluation techniques.

Practical experience with LangChain, LangGraph, and Langfuse is essential.

Solid understanding of GxP regulations (FDA 21 CFR Part 11, EU Annex 11) and GAMP 5 methodology is expected.