Want smarter insights in your inbox? Sign up for our weekly newsletters to get only what matters to enterprise AI, data, and security leaders.<\/em> Subscribe Now<\/em><\/p>\n\n\n\n When models attempt to get their way or become overly accommodating to the user, it can mean trouble for enterprises. That is why it\u2019s essential that, in addition to performance evaluations, organizations conduct alignment testing.<\/p>\n\n\n\n However, alignment audits often present two major challenges: scalability and validation. Alignment testing requires a significant amount of time for human researchers, and it\u2019s challenging to ensure that the audit has caught everything.\u00a0<\/p>\n\n\n\n In a paper, Anthropic researchers said they developed auditing agents that achieved \u201cimpressive performance at auditing tasks, while also shedding light on their limitations.\u201d The researchers stated that these agents, created during the pre-deployment testing of Claude Opus 4, enhanced alignment validation tests and enabled researchers to conduct multiple parallel audits at scale. Anthropic also released a replication of its audit agents on GitHub.\u00a0<\/p>\n\n\n\n New Anthropic research: Building and evaluating alignment auditing agents.<\/p> We developed three AI agents to autonomously complete alignment auditing tasks.<\/p> In testing, our agents successfully uncovered hidden goals, built safety evaluations, and surfaced concerning behaviors. pic.twitter.com\/HMQhMaA4v0<\/a><\/p>\u2014 Anthropic (@AnthropicAI) July 24, 2025<\/a><\/blockquote>\n<\/div><\/figure>\n\n\n\n \u201cWe introduce three agents that autonomously complete alignment auditing tasks. We also introduce three environments that formalize alignment auditing workflows as auditing games, and use them to evaluate our agents,\u201d the researcher said in the paper.\u00a0<\/p>\n\n\n\n The AI Impact Series Returns to San Francisco – August 5<\/strong><\/p>\n\n\n\n The next phase of AI is here – are you ready? Join leaders from Block, GSK, and SAP for an exclusive look at how autonomous agents are reshaping enterprise workflows – from real-time decision-making to end-to-end automation.<\/p>\n\n\n\n Secure your spot now – space is limited: https:\/\/bit.ly\/3GuuPLF<\/p>\n\n\n\n The three agents they explored were:<\/p>\n\n\n\n \u201cOverall, our agents demonstrate promise across multiple alignment auditing tasks. Additionally, our approach to validating agents with auditing games yields insights into agent capabilities, limitations, and key affordances. With further work, automated auditing could significantly help scale human oversight over AI systems,\u201d the paper said.<\/p>\n\n\n\n Anthropic said the first environment it developed tests an agent\u2019s ability to complete an alignment audit for an intentionally misaligned model. For this test, the researchers brought the investigator agent to the environment and equipped it with similar tools that human teams use. The researchers instructed the agent to identify the root cause of the misalignment and determine the specific behaviors that were misaligned.\u00a0<\/p>\n\n\n\n According to the researchers, the investigator agent successfully found the root cause of the issues 10-13% of the time. Still, this performance improved to 42% \u201cwhen using a super-agent approach that aggregates findings across multiple investigations.\u201d<\/p>\n\n\n\n The next step in most alignment audits is typically a behavioral evaluation, which determines whether the model consistently misbehaves in various settings. To test its evaluation agent, the researchers designed 10 models \u201cgiven system prompt instructions to exhibit a specific behavior, such as excessive deference to the user.\u201d<\/p>\n\n\n\n They ran the agent five times per model and saw that the agent correctly finds and flags at least one quirk of the model. However, the agent sometimes failed to identify specific quirks consistently. It had trouble evaluating subtle quirks, such as self-promotion and research-sandbagging, as well as quirks that are difficult to elicit, like the Hardcode Test Cases quirk.<\/p>\n\n\n\n The last test and agent concern behavioral red-teaming to find the prompts that elicit \u201cconcerning\u201d behaviors. The breadth-first red-teaming agent converses with the target model (in Anthropic\u2019s case, it was with Claude Opus 4), and this chat is then rated for alignment-relevant properties.\u00a0<\/p>\n\n\n\n The agent identified seven of the ten system quirks, but it also struggled with the same issues as the evaluator agent.<\/p>\n\n\n\n Alignment became a significant topic in the AI world after users noticed that ChatGPT was becoming overly agreeable. OpenAI rolled back some updates to GPT-4o to address this issue, but it showed that language models and agents can confidently give wrong answers if they decide this is what users want to hear.\u00a0<\/p>\n\n\n\n To combat this, other methods and benchmarks were developed to curb unwanted behaviors. The Elephant benchmark, developed by researchers from Carnegie Mellon University, the University of Oxford, and Stanford University, aims to measure sycophancy. DarkBench categorizes six issues, such as brand bias, user retention, sycophancy, anthromorphism, harmful content generation, and sneaking. OpenAI also has a method where AI models test themselves for alignment.\u00a0<\/p>\n\n\n\n Alignment auditing and evaluation continue to evolve, though it is not surprising that some people are not comfortable with it.\u00a0<\/p>\n\n\n\n Hallucinations auditing Hallucinations<\/p> Great work team.<\/p>\u2014 spec (@_opencv_) July 24, 2025<\/a><\/blockquote>\n<\/div><\/figure>\n\n\n\n However, Anthropic said that, although these audit agents still need refinement, alignment must be done now.\u00a0<\/p>\n\n\n\n \u201cAs AI systems become more powerful, we need scalable ways to assess their alignment. Human alignment audits take time and are hard to validate,\u201d the company said in an X post.\u00a0<\/p>\n\n\n\n As AI systems become more powerful, we need scalable ways to assess their alignment.<\/p> Human alignment audits take time and are hard to validate. <\/p> Our solution: automating alignment auditing with AI agents.<\/p> Read more: https:\/\/t.co\/CqWkQSfBIG<\/p>\u2014 Anthropic (@AnthropicAI) July 24, 2025<\/a><\/blockquote>\n<\/div><\/figure>\n Daily insights on business use cases with VB Daily<\/strong><\/p>\n If you want to impress your boss, VB Daily has you covered. We give you the inside scoop on what companies are doing with generative AI, from regulatory shifts to practical deployments, so you can share insights for maximum ROI.<\/p>\n \n\t\t\t\t\tThanks for subscribing. Check out more VB newsletters here.\n\t\t\t\t<\/p>\n An error occured.<\/p>\n<\/p><\/div>\n
\n<\/div>
\n\n\n\n
\n<\/div>\n
Auditing agents in action<\/h2>\n\n\n\n
![\"\"\/](\"https:\/\/lh7-rt.googleusercontent.com\/docsz\/AD_4nXcW1XtsOKBXA6qJ38w5DENejlWdBHmUizradX0vtmldDWP1XCvKhh5gBgHyTwG_BxHnruDUEbZ_DQk3YAWb-GXcS0biwy3emGjwmpXnyFCQByI9CHlvohfL83Xy0mCqKw9h-kn8Fg?key=08PpMIdS2W6zYhED38G-qQ\")
Alignment and sycophany problems<\/h2>\n\n\n\n
![\"\"\/](\"https:\/\/venturebeat.com\/wp-content\/themes\/vb-news\/brand\/img\/vb-daily-phone.png\")
\n\t\t\t\t<\/div>\n<\/p><\/div>\n<\/div>\t\t\t<\/div><\/script>\r\n