Join our ASCI team as a Sr. Prod Compliance Associate, a role focused on enhancing AI model quality and customer safety across Alexa experiences. You will leverage your judgment within established Standard Operating Procedures (SOPs) and utilize AI tools to support various workstreams. Working in collaboration with AI automation like LLM judges and AI agents, you will act as a human-in-the-loop, ensuring adherence to quality and safety benchmarks.

AI-powered tools, including LLM assistants and summarizers, are expected to be utilized in daily tasks. Exceptions apply only when handling sensitive customer data, which requires strict adherence to security protocols within a designated environment.

Key responsibilities include AI data annotation, quality auditing, and model improvement through multi-domain data curation and LLM judge output validation. You will also execute red teaming across various Alexa experiences, identifying and categorizing policy violations and safety gaps. Additionally, you will triage and analyze customer feedback, monitor for policy violations, and provide incident escalation support.

Further duties involve contributing to process improvements, mentoring junior associates, and participating in User Acceptance Testing (UAT) for new AI tools. A typical day involves incident alert prioritization, LLM judge output validation, dataset curation, and handling ambiguous customer feedback cases, rotating through red teaming, incident summary drafting, and process improvements.

A Bachelor's degree is required, complemented by over two years of experience in content moderation, data annotation, AI/ML operations, or compliance operations. Familiarity with AI annotation tools, LLM outputs, or model evaluation frameworks is preferred.

Additional preferred qualifications include experience with red teaming, content safety, or responsible AI operations. Exposure to incident management or escalation handling workflows will be beneficial for this role. We are committed to fostering an inclusive culture and provide workplace accommodations for individuals with disabilities.