Source: Fortune
Summary
Gray Swan, an AI security startup, is updating its red-teaming platform, Arena, to address security challenges posed by AI agents. The company launched Arena 2.0 with a new competition called Indirect Prompt Injection (IPI) Black, offering $300,000 in prizes. The platform helps test AI models by simulating real-world attacks. CEO Matt Fredrikson said the platform now includes beginner-friendly challenges and training. Gray Swan also offers commercial products like Shade and Cygnal, which test and block AI vulnerabilities. The company has grown from 15 to 66 employees since the start of the year.
Our Reading
The announcement sounds familiar.
Gray Swan is rebranding its platform for a new AI era.
It’s running a big competition with major sponsors.
The company is expanding its workforce and products.
Security challenges are getting more complex, but the goal stays the same.
Author: Evan Null
Gray Swan’s New Arena
Gray Swan, an AI security startup, has launched a redesigned version of its red-teaming platform called Arena. The new version, Arena 2.0, is aimed at testing AI agents, which are more complex than traditional chatbots. The platform allows participants to compete by trying to trick AI models into violating their guardrails. The company’s CEO, Matt Fredrikson, said the platform is evolving to keep up with the changing AI landscape.
Indirect Prompt Injection Competition
Gray Swan is running its largest competition yet, called Indirect Prompt Injection (IPI) Black, with $300,000 in prizes. The competition is sponsored by major companies like Anthropic, Meta, Google DeepMind, and Reflection AI. The goal is to test how well AI agents can resist hidden instructions embedded in content like web pages or forum posts. The competition will run through November 2.
Testing AI Agents
The new Arena includes sandbox environments where tests can be run safely. These environments are designed to simulate real-world scenarios where AI agents might be exposed to malicious content. Fredrikson said that the platform is used by both public and private clients, with some tests using unguarded models. The company also monitors AI agents to detect if they break out of the sandbox environment.
Challenges in Red-Teaming
As AI models have become more robust, the pool of people capable of red-teaming them has shrunk. Fredrikson said that the original Arena had a community of about 15,000 participants. While the number hasn’t dropped, maintaining engagement has required significant effort. The new Arena is designed to be more accessible, with beginner-friendly challenges and training programs to help new users get started.
Gray Swan’s Growth
Gray Swan has grown from 15 to 66 employees since the start of the year. The company initially focused on working with frontier AI labs but has now expanded to include large enterprises. Its commercial products, Shade and Cygnal, are used to test and block AI vulnerabilities. Fredrikson said that while AI attackers are faster at finding attacks, humans still have an edge in developing new jailbreaking tactics.
