Seeking a skilled AI QA Engineer with expertise in Agentic AI and Generative AI systems, particularly those leveraging large language models (LLMs). This role focuses on ensuring the quality and reliability of advanced AI applications through rigorous testing and automation.

This position requires a strong foundation in software quality assurance, combined with a passion for the cutting edge of AI development. You will be instrumental in building and maintaining robust testing strategies for complex AI solutions.

Develop and maintain Python-based automation frameworks, including Pytest, to streamline testing processes.

Critically evaluate LLM outputs for accuracy, relevance, and the presence of hallucinations.

Validate the functionality and performance of multi-agent workflows and data pipelines.

Design and implement evaluation metrics, such as F1 scores, precision, and recall, to quantify AI performance.

Conduct comprehensive API and integration testing to ensure seamless system interaction.

Strive for high test coverage and integrate testing seamlessly into CI/CD pipelines for efficient deployment.

Possess strong proficiency in Python programming and experience with automation frameworks like Pytest or unittest.

Demonstrate a solid understanding of Generative AI principles and Large Language Models.

Exhibit expertise in API testing, debugging, and troubleshooting complex issues.

Familiarity with CI/CD practices and version control systems is essential.

Requires a minimum of 7 years of experience in QA, with demonstrated exposure to GenAI or ML systems.