Want smarter insights in your inbox? Sign up for our weekly newsletters to get only what matters to enterprise AI, data, and security leaders.<\/em> Subscribe Now<\/em><\/p>\n\n\n\n In case you missed it, OpenAI yesterday debuted a powerful new feature for ChatGPT and with it, a host of new security risks and ramifications.<\/p>\n\n\n\n Called the \u201cChatGPT agent,\u201d this new feature is an optional mode that ChatGPT paying subscribers can engage by clicking \u201cTools\u201d in the prompt entry box and selecting \u201cagent mode,\u201d at which point, they can ask ChatGPT to log into their email and other web accounts; write and respond to emails; download, modify, and create files; and do a host of other tasks on their behalf, autonomously, much like a real person using a computer with their login credentials.<\/p>\n\n\n\n Obviously, this also requires the user to trust the ChatGPT agent not to do anything problematic or nefarious, or to leak their data and sensitive information. It also poses greater risks for a user and their employer than the regular ChatGPT, which can\u2019t log into web accounts or modify files directly. <\/p>\n\n\n\n Keren Gu, a member of the Safety Research team at OpenAI, commented on X that \u201cwe\u2019ve activated our strongest safeguards for ChatGPT Agent. It\u2019s the first model we\u2019ve classified as High capability in biology & chemistry under our Preparedness Framework. Here\u2019s why that matters\u2013and what we\u2019re doing to keep it safe.\u201d<\/p>\n\n\n\n The AI Impact Series Returns to San Francisco – August 5<\/strong><\/p>\n\n\n\n The next phase of AI is here – are you ready? Join leaders from Block, GSK, and SAP for an exclusive look at how autonomous agents are reshaping enterprise workflows – from real-time decision-making to end-to-end automation.<\/p>\n\n\n\n Secure your spot now – space is limited: https:\/\/bit.ly\/3GuuPLF<\/p>\n\n\n\n So how did OpenAI handle all these security issues? <\/p>\n\n\n\n Looking at OpenAI\u2019s ChatGPT agent system card, the \u201cread team\u201d employed by the company to test the feature faced a challenging mission: specifically, 16 PhD security researchers who were given 40 hours to test it out. <\/p>\n\n\n\n Through systematic testing, the red team discovered seven universal exploits that could compromise the system, revealing critical vulnerabilities in how AI agents handle real-world interactions.<\/p>\n\n\n\n What followed next was extensive security testing, much of it predicated on red teaming. The Red Teaming Network submitted 110 attacks, from prompt injections to biological information extraction attempts. Sixteen exceeded internal risk thresholds. Each finding gave OpenAI engineers the insights they needed to get fixes written and deployed before launch.<\/p>\n\n\n\n The results speak for themselves in the published results in the system card. ChatGPT Agent emerged with significant security improvements, including 95% performance against visual browser irrelevant instruction attacks and robust biological and chemical safeguards.<\/p>\n\n\n\n OpenAI\u2019s Red Teaming Network was comprised 16 researchers with biosafety-relevant PhDs who topgether submitted 110 attack attempts during the testing period. Sixteen exceeded internal risk thresholds, revealing fundamental vulnerabilities in how AI agents handle real-world interactions. But the real breakthrough came from UK AISI\u2019s unprecedented access to ChatGPT Agent\u2019s internal reasoning chains and policy text. Admittedly that\u2019s intelligence regular attackers would never possess.<\/p>\n\n\n\n Over four testing rounds, UK AISI forced OpenAI to execute seven universal exploits that had the potential to compromise any conversation:<\/p>\n\n\n\n Attack vectors that forced OpenAI\u2019s hand<\/strong><\/p>\n\n\n\n FAR.AI\u2019s assessment was openly critical of OpenAI\u2019s approach. Despite 40 hours of testing revealing only three partial vulnerabilities, they identified that current safety mechanisms relied heavily on monitoring during reasoning and tool-use processes, which the researchers considered a potential single point of failure if compromised.<\/p>\n\n\n\n OpenAI\u2019s response to the red team\u2019s results redefined entire segments of the ChatGPT agent architecture. One of the many initiatives undertaken including building a dual-layer inspection architecture that monitors 100% of production traffic in real-time, achieving these measurable improvements:<\/p>\n\n\n\n Security improvements after red team discoveries<\/strong><\/p>\n\n\n\n The architecture works like this:<\/p>\n\n\n\n But the technical defenses tell only part of the story. OpenAI made difficult security choices that acknowledge some AI operations require significant restrictions for safe autonomous execution.<\/p>\n\n\n\n Based on the vulnerabilities discovered, OpenAI implemented the following countermeasures across their model:<\/p>\n\n\n\n During pre-launch testing alone, this system identified and resolved 16 critical vulnerabilities that red teamers had discovered.<\/p>\n\n\n\n Red teamers revealed the potential that the ChatGPT Agent could be comprimnised and lead to greater biological risks. Sixteen experienced participants from the Red Teaming Network, each with biosafety-relevant PhDs, attempted to extract dangerous biological information. Their submissions revealed the model could synthesize published literature on modifying and creating biological threats.<\/p>\n\n\n\n In response to the red teamers\u2019 findings, OpenAI classified ChatGPT Agent as \u201cHigh capability\u201d for biological and chemical risks, not because they found definitive evidence of weaponization potential, but as a precautionary measure based on red team findings. This triggered:<\/p>\n\n\n\n The 110 attack submissions revealed patterns that forced fundamental changes in OpenAI\u2019s security philosophy. They include the following: <\/p>\n\n\n\n Persistence over power<\/strong>: Attackers don\u2019t need sophisticated exploits, all they need is more time. Red teamers showed how patient, incremental attacks could eventually compromise systems.<\/p>\n\n\n\n Trust boundaries are fiction<\/strong>: When your AI agent can access Google Drive, browse the web, and execute code, traditional security perimeters dissolve. Red teamers exploited the gaps between these capabilities.<\/p>\n\n\n\n Monitoring isn\u2019t optional<\/strong>: The discovery that sampling-based monitoring missed critical attacks led to the 100% coverage requirement.<\/p>\n\n\n\n Speed matters<\/strong>: Traditional patch cycles measured in weeks are worthless against prompt injection attacks that can spread instantly. The rapid remediation protocol patches vulnerabilities within hours.<\/p>\n\n\n\n OpenAI is helping to create a new security baseline for Enterprise AI<\/strong><\/p>\n\n\n\n For CISOs evaluating AI deployment, the red team discoveries establish clear requirements:<\/p>\n\n\n\n UK AISI\u2019s testing proved particularly instructive. All seven universal attacks they identified were patched before launch, but their privileged access to internal systems revealed vulnerabilities that would eventually be discoverable by determined adversaries.<\/p>\n\n\n\n \u201cThis is a pivotal moment for our Preparedness work,\u201d Gu wrote on X. \u201cBefore we reached High capability, Preparedness was about analyzing capabilities and planning safeguards. Now, for Agent and future more capable models, Preparedness safeguards have become an operational requirement.\u201d<\/p>\n\n\n\n The seven universal exploits discovered by researchers and the 110 attacks from OpenAI\u2019s red team network became the crucible that forged ChatGPT Agent. <\/p>\n\n\n\n By revealing exactly how AI agents could be weaponized, red teams forced the creation of the first AI system where security isn\u2019t just a feature. It\u2019s the foundation.<\/p>\n\n\n\n ChatGPT Agent\u2019s results prove red teaming\u2019s effectiveness: blocking 95% of visual browser attacks, catching 78% of data exfiltration attempts, monitoring every single interaction. <\/p>\n\n\n\n In the accelerating AI arms race, the companies that survive and thrive will be those who see their red teams as core architects of the platform that push it to the limits of safety and security.<\/p>\n Daily insights on business use cases with VB Daily<\/strong><\/p>\n If you want to impress your boss, VB Daily has you covered. We give you the inside scoop on what companies are doing with generative AI, from regulatory shifts to practical deployments, so you can share insights for maximum ROI.<\/p>\n \n\t\t\t\t\tThanks for subscribing. Check out more VB newsletters here.\n\t\t\t\t<\/p>\n An error occured.<\/p>\n<\/p><\/div>\n
\n<\/div>
\n\n\n\n
\n<\/div>![\"\"](\"https:\/\/venturebeat.com\/wp-content\/uploads\/2025\/07\/image_2a3039.png\")
<\/figure>\n\n\n\nThe red team\u2019s mission<\/h2>\n\n\n\n
Red teams exposed seven universal exploits <\/h2>\n\n\n\n
Attack Type<\/strong><\/td> Success Rate (Pre-Fix)<\/strong><\/td> Target<\/strong><\/td> Impact<\/strong><\/td><\/tr><\/thead> Visual Browser Hidden Instructions<\/td> 33%<\/td> Web pages<\/td> Active data exfiltration<\/td><\/tr> Google Drive Connector Exploitation<\/td> Not disclosed<\/td> Cloud documents<\/td> Forced document leaks<\/td><\/tr> Multi-Step Chain Attacks<\/td> Variable<\/td> Cross-site actions<\/td> Complete session compromise<\/td><\/tr> Biological Information Extraction<\/td> 16 submissions exceeded thresholds<\/td> Dangerous knowledge<\/td> Potential weaponization<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n How red teaming helped turn ChatGPT vulnerabilities into a fortress<\/h2>\n\n\n\n
Defense Metric<\/strong><\/td> Previous Models<\/strong><\/td> ChatGPT Agent<\/strong><\/td> Improvement<\/strong><\/td><\/tr><\/thead> Irrelevant Instructions (Visual Browser)<\/td> 82%<\/td> 95%<\/td> +13%<\/td><\/tr> In-Context Data Exfiltration<\/td> 75%<\/td> 78%<\/td> +3%<\/td><\/tr> Active Data Exfiltration<\/td> 58%<\/td> 67%<\/td> +9%<\/td><\/tr> System Reliability<\/td> Sampling-based<\/td> 100% coverage<\/td> Complete monitoring<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n \n
\n
A biological risk wake-up call<\/h2>\n\n\n\n
\n
What red teams taught OpenAI about AI security<\/h2>\n\n\n\n
\n
![\"\"](\"https:\/\/venturebeat.com\/wp-content\/uploads\/2025\/07\/image_adecd1.png\")
<\/figure>\n\n\n\nRed teams are core to building safer, more secure AI models <\/h2>\n\n\n\n
![\"\"\/](\"https:\/\/venturebeat.com\/wp-content\/themes\/vb-news\/brand\/img\/vb-daily-phone.png\")
\n\t\t\t\t<\/div>\n<\/p><\/div>\n<\/div>\t\t\t<\/div>\r\n