Gray Swan AI
Adversarial red-teaming arenas and safety evals for frontier models
Gray Swan AI, founded by CMU professors Zico Kolter and Matt Fredrikson with Andy Zou, runs adversarial red-teaming arenas, jailbreak competitions, and security evaluations used by frontier labs including OpenAI and Anthropic. Its environments stress-test agent robustness and safety.
Co-founder Zico Kolter chairs OpenAI's safety committee; ran red-teaming arenas for GPT and Claude launches
| Website | grayswan.ai |
|---|---|
| Domains | Cybersecurity, Alignment |
| Location | Pittsburgh |
| Team size | 11-25 |
| Founded | 2023 |
| Funding | ~$40M (reported) |
| Founders | Zico Kolter @zicokolter, Matt Fredrikson, Andy Zou |
Similar startups
Also listed under Cybersecurity, Alignment.
| Name | Domains | Location | Team | Funding | Raising |
|---|---|---|---|---|---|
| Rise Data Labs US-based expert human data and custom RL environments for enterprise AI | Computer Use, Coding, Finance, Cybersecurity, Legal, Data Labeling, Enterprise, Multi-Domain, RLHF, Custom Environments | United States | - | - | Yes |
| Andon Labs Long-horizon autonomy benchmarks like Vending-Bench | Long Horizon, Alignment | San Francisco | 1-10 | Seed (Y Combinator) | - |
| ARIMLABS Security and long-horizon environments for agentic AI | Cybersecurity, Long Horizon | Warsaw | 11-25 | - | - |
| dmodel Alignment-oriented environments focused on reward quality | Machine Learning, Alignment | San Francisco | 11-25 | - | - |
| Incalmo Offensive-security environments for AI agents | Cybersecurity | San Mateo | 11-25 | - | - |
| Invariant Labs Security testing and analysis for AI agents; acquired by Snyk | Cybersecurity, Alignment | Zurich | 1-10 | Acquired by Snyk (June 2025) | No |