Pure-play commercialCatalog available · 3

Gray Swan AI

Adversarial red-teaming arenas and safety evals for frontier models

Gray Swan AI, founded by CMU professors Zico Kolter and Matt Fredrikson with Andy Zou, runs adversarial red-teaming arenas, jailbreak competitions, and security evaluations used by frontier labs including OpenAI and Anthropic. Its environments stress-test agent robustness and safety.

Co-founder Zico Kolter chairs OpenAI's safety committee; ran red-teaming arenas for GPT and Claude launches

UnknownConfidence: unknown·Last verified: Unknown·How verification works

This legacy directory profile is awaiting claim-level source migration. Existing values are retained, not upgraded to verified facts.

Websitegrayswan.ai
DomainsCybersecurity, Alignment
LocationPittsburgh
Team size11-25
Founded2023
Funding~$40M (reported)
FoundersZico Kolter @zicokolter, Matt Fredrikson, Andy Zou

Capability coverage

Focus areas and technical capabilities are shown separately. Missing technical evidence remains Unknown.

Focus areas

Legacy classification; source migration pending

Catalog-evidenced technical capabilities

Unknown. No source-backed catalog capability record is available yet.

Capability catalog

Products and services supported by official company materials. Reviewed 2026-08-16.

Suggest an item
Evaluation Suitecompany reported

Arena

Adversarial model evaluation for model builders.

Commercial
Arena
Evaluation Suitecompany reported

Adversarial Evaluation

Adversarial evaluation services for AI models.

Commercial
Adversarial Evaluation
Evaluation Suitecompany reported

AI Red Teaming

Security red-teaming programs for AI systems.

Commercial
AI Red Teaming

Environments & datasets

Verified public artifacts and documented private commercial inventory.

Add catalog item →
No verified catalog items recorded yet. This does not mean the company has no environments or datasets.

Sources

Structured sources have not yet been migrated for this profile.

Improve this record

Company representatives and researchers can propose sourced changes. Submissions do not directly overwrite editorial data.

Similar startups

Also listed under Cybersecurity, Alignment.

NameDomainsType / catalogLocationTeamFundingRaising
Rise Data Labs
RL environments and tasks across multiple domains, supported by a large expert network
Computer Use, Coding, Finance, Cybersecurity, Legal, Data Labeling, Enterprise, Multi-Domain, RLHF, Custom Environments
Data + environments
3 artifacts
New York, United States11-25-Unknown
Andon Labs
Long-horizon autonomy benchmarks like Vending-Bench
Long Horizon, Alignment
Pure-play commercial
Catalog unknown
San Francisco1-10Seed (Y Combinator)Unknown
ARIMLABS
Security and long-horizon environments for agentic AI
Cybersecurity, Long Horizon
Pure-play commercial
Catalog unknown
Warsaw11-25-Unknown
Incalmo
Offensive-security environments for AI agents
Cybersecurity
Pure-play commercial
Catalog unknown
San Mateo11-25-Unknown
Invariant Labs
Security testing and analysis for AI agents; acquired by Snyk
Cybersecurity, Alignment
Acquired / inactive
Catalog unknown
Zurich1-10Acquired by Snyk (June 2025)Unknown
Quesma
Security-domain RL environments and binary analysis evals
Cybersecurity, Coding
Pure-play commercial
Catalog unknown
Warsaw11-25-Unknown