Gray Swan AI

Gray Swan AI is an AI safety and security company founded in June 2024 by researchers from Carnegie Mellon University. The company was co-founded by Matt Fredrikson (CEO), Zico Kolter (Chief Scientist), and Andy Zou (CTO), all of whom have deep roots in AI safety and security research at CMU. The company launched publicly in July 2024 with the mission of becoming the safety and security provider for the AI era. Gray Swan's founding team was responsible for some of the most influential research in AI adversarial robustness. In 2023, they published GCG (Greedy Coordinate Gradient), the first fully automated method for jailbreaking large language models, which exposed fundamental vulnerabilities in AI safety guardrails and received coverage in the New York Times, CNN, and the Washington Post. They followed this with Circuit Breakers, a novel alignment technique using representation engineering to robustly prevent AI systems from generating harmful content even under powerful unseen attacks. They have also produced widely used benchmarks including HarmBench, WMDP, AgentHarm, and CyBench. The company offers an AI Security Suite combining Cygnal (real-time input/output filtering and continuous agentic monitoring) with Shade (automated vulnerability testing and red-teaming). Gray Swan also operates the Gray Swan Arena, described as the world's largest red-teaming network, which has generated over three million attack attempts from researchers and developers testing frontier model safety. The Arena has hosted major competitions in partnership with the UK AI Safety Institute, the US AI Safety Institute, OpenAI, Anthropic, Google DeepMind, Amazon, and Meta. Gray Swan is headquartered in Pittsburgh, Pennsylvania, operating out of offices on South Craig Street near Carnegie Mellon's campus. The company raised approximately $5.5 million in early funding from friends, family, and organizations focused on AI safety and security. Dan Hendrycks, co-founder of the Center for AI Safety and an early advisor and investor, later divested his equity stake in the company due to perceived conflicts of interest related to his role in supporting California's SB-1047 AI regulation bill, though he continued as an unpaid advisor. As of early 2026, Gray Swan continues to run active red-teaming competitions through the Arena platform, including a Safeguards Challenge with a $140,000 prize pool running through May 2026, and has published research on indirect prompt injection vulnerabilities in agentic AI systems in partnership with Stanford.