Center for AI Safety

The Center for AI Safety (CAIS) is an American nonprofit organization founded in April 2022 by Dan Hendrycks and Oliver Zhang in San Francisco, California. Its mission is to reduce societal-scale risks from artificial intelligence through research, field-building, and advocacy. Dan Hendrycks, the Executive and Research Director, holds a PhD in Computer Science from UC Berkeley, where he was advised by Dawn Song and Jacob Steinhardt. He is known for contributing the GELU activation function, the out-of-distribution detection baseline, and distribution shift benchmarks. His research has been cited over 32,000 times. Oliver Zhang, the Managing Director, previously co-founded ML Alignment Theory Scholars (MATS). CAIS conducts both technical and conceptual AI safety research. On the technical side, the organization creates foundational benchmarks and methods designed to improve the safety of AI systems, publishing at top ML conferences and sharing open datasets and code. Notable research contributions include Circuit Breakers (preventing dangerous AI behaviors), the WMDP Benchmark (assessing hazardous knowledge in AI models), HarmBench (an evaluation framework for automated red teaming adopted by US and UK AI Safety Institutes), and Humanity's Last Exam (a global benchmark initiative with over 1,200 collaborators). Their conceptual research examines AI safety from a multidisciplinary perspective, incorporating insights from safety engineering, complex systems, philosophy, and international relations. Field-building is a major focus. CAIS operates a GPU compute cluster that provides free access to ML safety researchers, supporting approximately 350 researchers and producing over 100 papers with 4,000+ citations. The Philosophy Fellowship is a seven-month research program investigating societal implications of advanced AI. CAIS published the textbook Introduction to AI Safety, Ethics, and Society through Taylor and Francis in 2024 and runs an accompanying online course. The SafeBench competition offers $250,000 in prizes for new benchmarks assessing AI risks. In May 2023, CAIS published the Statement on AI Risk, a single-sentence declaration that mitigating the risk of extinction from AI should be a global priority alongside pandemics and nuclear war. The statement was signed by over 600 AI experts including Turing Award winners Geoffrey Hinton and Yoshua Bengio, as well as leaders of major AI labs such as Sam Altman (OpenAI) and Demis Hassabis (Google DeepMind). This became one of the most widely covered AI safety stories of 2023. CAIS launched its advocacy arm, the Center for AI Safety Action Fund (a separate 501(c)(4)), in 2024 with a presence in Washington, DC. Advocacy achievements include helping secure $10 million in congressional funding for the US AI Safety Institute and co-sponsoring California's SB 1047. As of the 2024 tax filing, CAIS has 25 employees and reported annual revenue of $10.2 million with expenses of $7.2 million. Major funders include Open Philanthropy (which made an $8.5 million grant in 2024), Good Ventures Foundation, Silicon Valley Community Foundation, and Founders Pledge. The Survival and Flourishing Fund recommended $289,000 for CAIS in its 2025 round. In 2022, FTX donated $6.5 million to CAIS before its collapse.