AI Safety Projects
Safety Research
Google DeepMind's AI safety research program, working to ensure that advanced AI and AGI systems are safe, aligned with human values, and do not pose catastrophic risks.
Existential Risk Laboratory
An interdisciplinary research lab at the University of Chicago dedicated to analyzing and mitigating existential risks, with a focus on AI safety, nuclear security, biosecurity, and extreme climate change.
Online Team
The Online Team at CEA builds and maintains the EA Forum, effectivealtruism.org, and the EA Newsletter, providing the primary digital infrastructure for discussion, research, and coordination within the effective altruism community.
Community Health and Special Projects
A team within the Centre for Effective Altruism that works to strengthen the EA community's ability to have impact by addressing interpersonal harm, advising on risk management, and supporting healthy organizational cultures.
Research Scholars Programme
A two-year research fellowship at Oxford's Future of Humanity Institute that gave early-career researchers salaried positions to explore questions critical to humanity's long-term future, including AI safety, existential risk, and macrostrategy.
Global and Emerging Risks
RAND Global and Emerging Risks is a research division of RAND Corporation delivering rigorous, objective public policy research on catastrophic and existential risks to civilization, including AI, synthetic biology, climate, and nuclear threats.
Talos Fellowship
Talos Network runs the Talos Fellowship, a program that trains and places early-career professionals into European AI policy roles to help ensure safe and responsible AI deployment.
AIES 2022 organizers (supported by AAAI and ACM)
The organizing committee of the Fifth AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES 2022), a peer-reviewed academic conference held August 1-3, 2022 at Keble College, Oxford, UK.
AAAI/ACM Conference on AI, Ethics, and Society (AIES 2020)
AIES 2020 was the third annual AAAI/ACM Conference on AI, Ethics, and Society, a peer-reviewed academic conference addressing the ethical and societal implications of artificial intelligence, held February 7-8, 2020 in New York City.
AISafety.com: Self-study
The self-study section of AISafety.com curates courses, textbooks, and reading lists for independent learning in AI safety, covering both technical alignment and AI governance.
AISafety.com: Media Channels
A curated directory of AI safety podcasts, newsletters, YouTube channels, blogs, books, and forums maintained by AISafety.com. It helps newcomers and practitioners stay informed about rapid developments in the AI safety field.
AISafety.com: Funding
A curated directory of 60+ funding sources for AI safety work, maintained by AISafety.com as part of its broader resource hub for the AI safety ecosystem.
AISafety.com: Donation Guide
A regularly-updated guide on how to donate most effectively to the AI safety field, structured by donation amount and time available.
AISafety.com: Advisors
A directory within AISafety.com that consolidates free guidance calls from AI safety advisors, helping newcomers identify how to contribute most effectively to the field.
Neuronpedia
Neuronpedia is an open-source interpretability platform for exploring, analyzing, and steering the internal features of AI language models. It serves as the primary public infrastructure for mechanistic interpretability research, particularly around sparse autoencoders (SAEs).
Effective Altruism Sweden Rationality Research Project
A rationality research project operating under Effective Altruism Sweden (Effektiv Altruism Sverige), funded through the Survival and Flourishing Fund's Lightspeed Grants program.
CHAI Internship
A research internship program within CHAI at UC Berkeley where participants work on AI safety research projects under the mentorship of PhD students and postdocs, typically resulting in first-author workshop papers.
Language Model Safety Fund
A fiscally sponsored AI safety project led by Ethan Perez that funded research engineers to work on language model misalignment, which later evolved into part of FAR.AI (Frontier Alignment Research).
FAR.AI YouTube channel
The FAR.AI YouTube channel (@FARAIResearch) publishes recordings of AI safety talks, seminars, and workshop sessions organized by FAR.AI, the AI safety research nonprofit based in Berkeley.
Technology Strategy Roleplay
A UK charity that develops and deploys Intelligence Rising, a strategic role-playing game designed to help decision-makers understand the risks, tensions, and governance challenges of AI development through facilitated wargaming exercises.
Buddhism & AI Initiative
A collaborative initiative bringing together Buddhist communities, technologists, and contemplative researchers worldwide to help shape the future of artificial intelligence through Buddhist wisdom and practice.
Orion AI Governance Initiative
A UK-based talent development initiative supporting outstanding students to build careers shaping the future of AI governance, run as a project of Arcadia Impact.
Safe AI London (SAIL)
Safe AI London (SAIL) is a London-based community program that supports individuals interested in pursuing careers in AI safety, operated as a project of Arcadia Impact.
Impact Research Groups (IRG)
An 8-week London-based programme run by Arcadia Impact that supports talented students in launching high-impact research careers across AI safety, AI governance, biosecurity, global health, and animal welfare.
The Events team at the Centre for Effective Altruism
The events program within CEA that organizes EA Global conferences, supports EAGx events run by local organizers, runs specialized retreats, and coordinates EA Summits to connect people working on the world's most pressing problems.
Effective Altruism Forum
The EA Forum (forum.effectivealtruism.org) is the primary online discussion and publishing platform for the effective altruism community, run by the Centre for Effective Altruism.
Transformer
Transformer is a journalism publication covering the power and politics of transformative AI, aimed at policymakers and decision-makers. It operates as a project of the Tarbell Center for AI Journalism.
Tarbell Fellowship
A one-year journalism fellowship that trains and places reporters at major newsrooms to cover artificial intelligence, providing stipends, a 10-week AI fundamentals course, and nine-month embedded placements at outlets like Bloomberg, TIME, The Guardian, and NBC News.
AI Safety, Ethics and Society (AISES)
AI Safety, Ethics and Society (AISES) is an open educational project of the Center for AI Safety that provides a free textbook and virtual course introducing AI safety, ethics, and societal risks to a broad, non-technical audience.
AI Safety Newsletter
A free newsletter by the Center for AI Safety covering the latest developments in AI safety research, policy, and industry news. No technical background required.
AI Frontiers
AI Frontiers is a publication run by the Center for AI Safety that hosts expert commentary and debate on the societal impacts of artificial intelligence, covering topics from AI safety to policy, economics, and national security.
Cambridge ERA:AI Fellowship
An 8-10 week fully funded research fellowship in Cambridge, UK, supporting early-career researchers and entrepreneurs working to mitigate catastrophic risks from frontier AI. Operated by the Existential Risk Alliance (ERA), a spin-off from the Cambridge Existential Risks Initiative (CERI).
Narrow Path
A Narrow Path is a comprehensive policy framework published by ControlAI in October 2024, laying out a three-phase plan for humanity to safely navigate the development of artificial superintelligence.
Foresight Fellowship
The Foresight Fellowship is a year-long program by the Foresight Institute that supports early-career scientists, engineers, and innovators working on transformative technologies including AI safety, longevity, nanotechnology, and neurotechnology.
Mentorship for Alignment Research Students (MARS)
Part-time research program run by Cambridge AI Safety Hub (CAISH), connecting aspiring researchers with experienced mentors to conduct AI safety research over 2-3 months.
UChicago Existential Risk Laboratory (XLab) Fellowship
The Existential Risk Laboratory (XLab) at the University of Chicago trains early-career researchers through a 10-week summer fellowship focused on AI safety, nuclear security, and global catastrophic risk. It serves as an interdisciplinary hub for existential risk research across the university.
Cooperative AI Summer School
An annual intensive summer school run by the Cooperative AI Foundation that trains students and early-career researchers in the emerging field of cooperative AI.
GovAI Fellowship
A three-month fellowship program run by the Centre for the Governance of AI (GovAI) to help early-career researchers and practitioners transition into full-time work on AI governance.
Impact Academy: Global AI Safety Fellowship
Impact Academy runs fully-funded research fellowships that place exceptional global STEM talent with leading AI safety organizations. Its flagship program, the Global AI Safety Fellowship, supports 10-20 fellows per cohort for 3-6 months at labs such as CHAI, FAR.AI, and the UK AI Safety Institute.
Pathfinder Fellowship
The Pathfinder Fellowship is a selective fellowship run by Kairos that provides funding, mentorship, and resources to students organizing AI safety and policy university groups worldwide.
IAPS AI Policy Fellowship
A fully-funded, three-month fellowship program run by the Institute for AI Policy and Strategy (IAPS) that trains professionals from diverse backgrounds to work on AI policy projects with the goal of securing a positive future with powerful AI.
PIBBSS Fellowship
A 3-month summer research fellowship that pairs PhD and postdoctoral researchers from complex systems and related sciences with AI safety mentors to produce interdisciplinary alignment research.
ML & AI Safety Updates
A weekly newsletter, podcast, and YouTube series by Apart Research covering the latest research in machine learning and AI safety. The series was later discontinued as Apart Research reprioritized its activities.
AI Safety Ideas
AI Safety Ideas is a collaborative, open-access platform for sharing and discovering AI safety research project ideas, built and maintained by Apart Research.
Apart Sprints
Apart Sprints is the research hackathon program of Apart Research, running weekend-long AI safety research events with thousands of global participants. Teams work on frontier AI safety questions and produce open-source research, evaluations, and prototypes.
FLI Fellowships
FLI's fellowship programs fund PhD students and postdoctoral researchers in technical AI existential safety and US-China AI governance, building the next generation of safety researchers.
FLI Podcast
The FLI Podcast is the Future of Life Institute's long-running audio program featuring in-depth conversations with researchers, philosophers, and policy experts on AI safety, existential risk, and the governance of powerful technologies.
BERI-SRL Collaboration
A collaboration between the Berkeley Existential Risk Initiative (BERI) and the Safe Robotics Laboratory (SRL) at Princeton University, through which BERI provides operational support to SRL's research on rigorous safety frameworks for autonomous systems and AI.
BERI-SERI Collaboration
An operational partnership between the Berkeley Existential Risk Initiative (BERI) and the Stanford Existential Risks Initiative (SERI), through which BERI provides administrative and financial services to support SERI's existential risk research and fellowship programs.
BERI-FHI Collaboration
A formal collaboration through which the Berkeley Existential Risk Initiative (BERI) provided operational and research support to the Future of Humanity Institute (FHI) at the University of Oxford, active from approximately 2017 until FHI's closure in April 2024.
BERI-DMIP Collaboration
A collaboration between the Berkeley Existential Risk Initiative (BERI) and the Data, Machines, Intelligence & People (DMIP) research group at the Universitat Politecnica de Valencia, focused on AI evaluation, intelligence measurement, and AI safety research.
BERI-CSER Collaboration
A long-running partnership between the Berkeley Existential Risk Initiative (BERI) and the Centre for the Study of Existential Risk (CSER) at the University of Cambridge, through which BERI provides operational, administrative, and financial support to CSER's existential risk research.
BERI-CLTC Collaboration
A collaboration between the Berkeley Existential Risk Initiative (BERI) and UC Berkeley's Center for Long-Term Cybersecurity (CLTC) focused on developing AI risk management standards and guidance for developers of increasingly general-purpose AI systems.
BERI-CHAI Collaboration
An operational collaboration in which the Berkeley Existential Risk Initiative (BERI) provides administrative, technical, and staffing support to the Center for Human-Compatible AI (CHAI) at UC Berkeley, enabling CHAI researchers to focus on AI safety work.
BERI-ALL Collaboration
A collaboration between the Berkeley Existential Risk Initiative (BERI) and the Autonomous Learning Laboratory (ALL) at the University of Massachusetts Amherst, focused on safe and fair machine learning research.
CLR Fund
The CLR Fund is the grantmaking program of the Center on Long-Term Risk, supporting researchers and projects working to reduce worst-case suffering risks from advanced AI systems.
CLR Fellowship
The CLR Summer Research Fellowship is an annual 8-week paid program that places researchers on s-risk and empirical AI safety projects, serving as CLR's primary talent pipeline into the field.
AI Safety Support Newsletter
The newsletter arm of AI Safety Support (AISS), an Australian charity that provided ecosystem updates and curated resources to the AI safety community. The newsletter is no longer active following AISS's shutdown in July 2023.
AE Studio Research
The research arm and grants portal of AE Studio, focused on advancing AI alignment and neurotechnology through neglected, high-impact approaches funded by AE Studio's profitable consulting business.
Superlinear Prizes
Superlinear is an x-risk prize competition platform created by Nonlinear that offers monetary prizes to incentivize effective altruists to work on existential risk reduction problems.
Nonlinear Network
A program of Nonlinear that connects AI safety projects with a network of 60+ funders through a single common application.
Advocacy Grants
A grant program run by Nonlinear that funds individuals and groups raising awareness about AI existential risks or advocating for a pause in AI development until it can be made safe.
AI In Context
A video content program produced by 80,000 Hours that uses cinematic, long-form storytelling to communicate the risks of transformative AI to general audiences. Hosted by Aric Floyd, the program publishes on YouTube, TikTok, and Instagram.
Problem Profile
80,000 Hours produces in-depth research profiles on the world's most pressing problems, helping people identify where their careers can have the greatest impact. Their problem profiles evaluate global issues using a rigorous framework of scale, neglectedness, and tractability.
Podcast
A flagship long-form interview podcast from 80,000 Hours exploring the world's most pressing problems and how careers can be used to address them, with a strong focus on AI safety.
Job Board
The 80,000 Hours Job Board is a curated listing of high-impact job opportunities at organizations working on the world's most pressing problems, particularly AI safety and biosecurity. It is a flagship product of 80,000 Hours, the effective altruism career advice nonprofit.
Tegmark Group
Max Tegmark's AI safety research group at MIT, focused on mechanistic interpretability, physics-informed machine learning, and frameworks for guaranteed safe AI.
AI Governance Project
A program within CSIS, a major Washington DC think tank, that conducts policy research on AI governance, geopolitics, and national security implications of artificial intelligence, advising U.S. and allied governments on responsible AI policy.
Stephen Grugett, James Grugett, Austin Chen
Shin-Shin Hua and Haydn Belfield
Rauno Arike, Elizabeth Donoway
Philip Tetlock, Ezra Karger, Pavel Atanasov
Michael Pearce, Alice Riggs, Thomas Dooms
Lucius Bushnaq, Callum McDougall, Avery Griffin
Javier Ferrando Monsonís and Oscar Balcells Obeso
David Hahnemann, Luan Ademi
Alex Cloud, Jacob Goldman-Wetzler, Evžen Wybitul, Joseph Miller
Standards Infrastructure Project
Earendil develops standards and infrastructure for hardware-enabled AI compute governance, building the protocols and specifications that enable privacy-preserving verification of compliance with AI development agreements.
Security Layers Project
Earendil develops hardware tamper response systems for AI compute infrastructure, enabling physical security layers that support governance and compliance verification of GPU clusters used in AI development.
AFFINE Technical Research
AFFINE (Agent Foundations FIeld NEtwork) runs intensive educational programs and supports technical research aimed at solving the superintelligence alignment problem, with a focus on agent foundations.
AFFINE Algorithm Design
AFFINE is an AI safety research training network that runs an intensive month-long superintelligence alignment seminar and year-long fellowship, with Algorithm Design as one of its core funded research focus areas.
AI Clarity
AI Clarity is the scenario planning research program of Convergence Analysis, exploring possible futures with transformative AI and evaluating strategies to mitigate existential risks through systematic scenario analysis.
Toronto and Michigan NLP Group for AI Safety
Angelina Wang's Responsible AI Lab
Angelina Wang's research group at Cornell Tech focuses on responsible AI, studying fairness, evaluation methodologies, and societal impacts of AI systems to make them more equitable and accountable.