LLM Red-Teamer

About this role

The company is engaging LLM Red-Teamers to contribute to a high-impact customer project focused on the evaluation and improvement of frontier language models. In this role, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world input. No prior experience in AI is required — your domain knowledge is what matters.

Skills

Key responsibilities

• Develop complex, adversarial multi-turn conversations and task-based scenarios aligned with detailed project specifications.
• Author clear, precise evaluation rubrics to rigorously assess model responses against defined behavioral targets.
• Iteratively test conversations and tasks against frontier LLMs, escalating difficulty and nuance until the desired quality threshold is achieved.
• Deliver comprehensive task packages, including transcripts, target behaviors, binary rubrics, and supporting rationale or evidence.
• Validate LLM outputs, documenting model strengths and failure modes relative to the project specification.
• Maintain calibration with team leads and quality control contacts as project requirements evolve.
• Contribute independently, producing high-quality deliverables at a steady and consistent pace.

Preferred qualifications

• Exceptional written English skills, with clarity, precision, and strong structural organization.
• Prior experience in AI human data environments (RLHF, SFT, evaluations, annotation, or prompt engineering).
• Deep familiarity with large language models, including the ability to anticipate and identify common failure patterns.
• Demonstrated ability to work autonomously, interpreting and executing complex specifications with minimal oversight.
• Proven critical thinking and meticulous attention to detail.
• Experience designing evaluation items or rubrics is advantageous.
• Background in writing-intensive or analysis-centric fields such as research, editorial, technical writing, or quality assurance is a plus.