Carnegie Mellon University is a leading private research university in Pittsburgh, Pennsylvania, widely regarded as one of the world's top institutions for AI and computer science research. It hosts multiple AI safety and governance programs spanning technical research, policy, and applied AI security.
Carnegie Mellon University is a leading private research university in Pittsburgh, Pennsylvania, widely regarded as one of the world's top institutions for AI and computer science research. It hosts multiple AI safety and governance programs spanning technical research, policy, and applied AI security.
People– no linked people
Updated 05/18/26Funding Details
Updated 05/18/26- Annual Budget
- $1,800,000,000
- Current Runway
- -
- Funding Goal
- -
- Funding Raised to Date
- -
Org Details
Updated 05/18/26Carnegie Mellon University (CMU) is a private research university located in Pittsburgh, Pennsylvania, founded in 1900 by industrialist Andrew Carnegie as the Carnegie Technical Schools. In 1967, it merged with the Mellon Institute of Industrial Research to become Carnegie Mellon University. Today CMU enrolls approximately 15,900 students across seven colleges, employs roughly 8,000 faculty and staff, and holds a $3.5 billion endowment as of June 2025. CMU consistently ranks #1 in AI research programs (U.S. News) and describes itself as the birthplace of artificial intelligence. Its AI safety and existential risk-relevant work spans numerous centers and initiatives. The Carnegie Mellon AI Safety Initiative (CASI) is a student-led organization that facilitates AI alignment and governance research connections, reading groups, fellowships, and seminars for undergraduates and graduate students. The CMU Safe AI Lab, directed by Professor Ding Zhao at the Robotics Institute, focuses on trustworthy physical AI agents for high-stakes applications. The K&L Gates Initiative in Ethics and Computational Technologies was endowed with $10 million by global law firm K&L Gates in 2016 to support research and doctoral fellowships on the ethical and societal implications of AI and computational technologies. The Block Center for Technology and Society, established in 2019, operates a Responsible AI initiative that influences federal and state policy, collaborated with Pennsylvania's governor on responsible generative AI deployment, and helped operationalize the NIST AI Risk Management Framework. The Software Engineering Institute (SEI), a federally funded research and development center operated by CMU for the Department of Defense (contract renewed 2025-2030), established the first Artificial Intelligence Security Incident Response Team (AISIRT) in 2023. SEI's AI Division addresses AI engineering, safety, reliability, and trustworthiness for national security applications. CyLab, CMU's university-wide security and privacy institute, conducts research on machine learning security including adversarial attacks and AI privacy risks, with over 160 affiliated faculty. In 2024, NIST awarded CMU $6 million to establish a joint AI Measurement Science and Engineering Cooperative Research Center focused on AI risk management and evaluation methods. CMU is also a founding member of the U.S. AI Safety Institute Consortium (AISIC). Professor Zico Kolter, head of the Machine Learning Department, leads OpenAI's external safety review panel and in 2025 joined a $10 million Schmidt Sciences AI Safety Science Program investigating adversarial transfer in AI models. Open Philanthropy has made multiple grants to CMU faculty for technical AI safety research including work on AI unlearning, adversarial robustness, and LLM evaluation benchmarks.
Theory of Change
Updated 05/18/26CMU advances AI safety through a multi-pronged institutional strategy: producing technical research on robustness, alignment, and evaluation methods that inform the field broadly; training the next generation of AI safety researchers through degree programs, fellowships, and CASI's educational pipeline; influencing AI policy and governance frameworks by embedding researchers in federal advisory roles (NIST, DoD, AISIC, OpenAI safety panels); and applying safety research to high-consequence national security domains through the SEI. The underlying theory is that a research university with top-ranked AI programs can simultaneously advance technical safety methods, shape industry and government norms, and seed the broader AI safety talent pipeline at scale.
Grants Received
Updated 05/18/26Projects– no linked projects
Updated 05/18/26Discussion
Key risk: As a massive, well-funded university with mixed incentives, marginal donations (likely routed via Conitzer’s program) face high fungibility and may prioritize robustness/governance or speculative multi-agent coordination over core alignment, yielding limited counterfactual x-risk reduction.
Case for funding: CMU’s integrated AI safety/security ecosystem—SEI’s AISIRT and DoD ties, CyLab’s ML security work, the NIST-funded AI Measurement Center, AISIC participation, and leadership like Zico Kolter—uniquely positions it to translate rigorous robustness/evaluation research into widely adopted standards while scaling a high-quality safety talent pipeline.