Join our Technology & Transformation practice as a Senior Quality Engineer, specializing in Automation Testing. This role is pivotal in ensuring the quality, safety, and reliability of our advanced AI-powered applications. You will drive the quality strategy, develop cutting-edge automation frameworks, and contribute to insightful decision-making through data-driven solutions.

This is an opportunity to work with a global network, leveraging strategic guidance and implementation services to transform complex data into actionable intelligence. Our practice focuses on business intelligence, data management, performance management, and next-generation analytics, including big data, cloud, cognitive, and machine learning technologies.

As a Senior Quality Engineer, you will own the entire test strategy, from automation roadmaps to release readiness. Your responsibilities include designing and maintaining robust automation frameworks using Playwright with Python and Behave for BDD. You'll automate various test types, including UI, API, integration, and regression testing, while also supporting UAT automation and existing regression suites.

A key focus will be on validating AI/LLM systems for accuracy, relevance, bias, and safety. You will test for vulnerabilities like prompt injection, handle harmful prompts, and ensure AI agents integrate effectively. Building LLM evaluation frameworks, benchmarking, and scoring systems will be integral. Furthermore, you'll integrate tests into CI/CD pipelines, conduct basic performance testing, debug complex issues using logs and traces, and define quality dashboards for auditability and compliance. Mentoring junior QA engineers and collaborating with cross-functional teams like Product, Engineering, and Data Science are also core aspects of this role.

We are seeking a seasoned Quality Engineering professional with over 9 years of experience in test automation. Essential hands-on expertise includes Playwright with Python, alongside experience building scalable automation frameworks utilizing Python, Pytest, and Behave. A strong background in testing AI, Generative AI, or LLM-powered applications is mandatory.

Proficiency in API testing using tools like Playwright, Postman, or REST-assured is required. Experience with BDD, Git, CI/CD, and Agile methodologies is expected. Knowledge of AI evaluation techniques, prompt testing, RAG validation, hallucination testing, and Responsible AI principles is crucial. Excellent problem-solving, mentoring, communication, and stakeholder management skills are vital for success in this role.