Juniper Ventures is a pre-seed venture capital firm that invests in startups explicitly working to make AI secure and beneficial for humanity.

A small nonprofit research organization studying global catastrophic risks, best known for its insight-based AI timelines model and research on the feasibility of training AGI via deep reinforcement learning.

An expert-managed grantmaking fund that supports projects building the effective altruism community's capacity, including community building, prioritization research, epistemic infrastructure, events, and fundraising for effective charities.

An interdisciplinary research lab at Carnegie Mellon University, directed by Simon DeDeo, that studies complex social systems through mathematical modeling and empirical investigation to better understand humanity's past, present, and future.

No summary available yet.

A nonprofit organization working to steer transformative technologies -- particularly AI, biotechnology, and nuclear weapons -- away from extreme large-scale risks and towards benefiting life.

A community blog and forum devoted to refining the art of human rationality, with major focus areas including AI alignment, cognitive biases, decision-making, and effective altruism.

The Responsible AI Collaborative (TheCollab) is a nonprofit that maintains the AI Incident Database (AIID), the leading public repository of documented real-world AI harms and near-harms.

AI Explained is a London-based YouTube channel by a creator known as Philip that provides hype-free coverage of AI developments, capabilities, and safety topics for a general audience.

Founders Pledge empowers entrepreneurs to pledge a portion of their exit proceeds to high-impact charities, and conducts cause-area research and grantmaking to guide that giving.

ARLIS is the University of Maryland's Department of Defense University Affiliated Research Center (UARC) dedicated to intelligence and national security, combining AI, behavioral science, and systems engineering to address complex security challenges.

Oxford-based AI safety company developing concept extrapolation technology to enable AI systems to generalize human values and intent beyond their training data.

Anthropic is an AI safety company and public benefit corporation building reliable, interpretable, and steerable AI systems, best known for developing the Claude family of large language models.

Swiss nonprofit funder making grants and investments to reduce suffering risks from catastrophic AI misuse, AI conflict, and other large-scale harms. Formerly known as Center for Emerging Risk Research (CERR) and Polaris Ventures.

Imperial College London is a world-leading research university specialising in science, technology, engineering, medicine, and business, with significant programs in AI safety, trustworthy AI, and long-term AI risk research.

LISA is a London-based charity that serves as a hub and infrastructure provider for the AI safety ecosystem, hosting resident organizations, training programs, and independent researchers.

The Cooperative AI Foundation (CAIF) is a UK-registered charity that funds and supports research to improve the cooperative intelligence of advanced AI systems for the benefit of humanity.

AI Alignment Awards is a prize contest program that awards up to $100,000 for novel research progress on core AI alignment problems. It is a project of the Players Philanthropy Fund, funded by Open Philanthropy.

No summary available yet.

Hofvarpnir Studios is a nonprofit that builds and maintains GPU compute clusters to support academic AI safety research. It provides high-performance computing infrastructure to researchers who would otherwise lack access to the resources needed to study and advance AI safety.

CEA builds and supports the global effective altruism community through conferences, online platforms, local group support, grantmaking, and community health programs, helping people use evidence and reason to address the world's most pressing problems.

Stop AGI is a project and website launched by Andrea Miotti in April 2023 to communicate the extinction risks of artificial general intelligence to the public and propose policy solutions to prevent its development.

A Washington, D.C.-based nonprofit that educates the public and policymakers about the risks of advanced AI and advocates for bipartisan safeguards.

A high-level international dialogue series that brings together leading AI scientists and governance experts to build consensus on managing extreme risks from frontier AI systems.

Technical University of Munich (TUM) is one of Europe's leading research universities, with significant AI safety and reliable AI research programs including the Konrad Zuse School of Excellence in Reliable AI (relAI).

Czech national organization promoting effective altruism through community building, events, and project incubation, with a particular focus on AI safety and high-impact careers.

ML4Good runs intensive, fully-funded in-person bootcamps to train motivated people for careers in AI safety, covering both technical and governance tracks.

The Schwartz Reisman Institute for Technology and Society (SRI) is a University of Toronto research institute that convenes experts across disciplines to ensure that powerful technologies like AI are responsible, inclusive, and beneficial to everyone.

Joseph Gordon-Levitt's collaborative media platform, which established a dedicated AI safety arm (HitRecord AI Safety Project LLC and AI Safety Digital Media Fund) to use storytelling and public engagement to address AI risks.

A Washington, DC-based 501(c)(3) nonprofit that conducts AI policy research, develops actionable legislative proposals, and educates U.S. policymakers on responsible innovation. It is the research and education arm of the Americans for Responsible Innovation family of organizations.

Reporting and analysis on capitalism, great power competition, and the race to build machine superintelligence by freelance journalist Garrison Lovely.

Rational Animations is a YouTube channel producing high-quality animated videos about AI safety, rationality, and effective altruism to reach mainstream audiences.

An independent research institute producing policy analysis on the social implications of artificial intelligence, with a focus on corporate power, labor, and public accountability.

Ergo Impact finds, funds, and scales promising people and solutions to the world's most pressing problems by providing ambitious philanthropists a rigorous, high-leverage approach to deploying capital at scale.

Boston University is a large private research university in Boston, Massachusetts with over 37,000 students, 17 schools and colleges, and more than $554 million in annual research expenditures. It hosts AI safety and alignment student programs and has received Open Philanthropy funding for AI safety-relevant research.

A research group at MIT CSAIL developing algorithmic frameworks, techniques, and policies to make AI systems safe and socially beneficial. Led by Associate Professor Dylan Hadfield-Menell.

A 501(c)(3) nonprofit that funds, accelerates, and advocates for AI alignment research by providing engineering teams, compute, and infrastructure to researchers pursuing neglected approaches.

A monthly newsletter curating and summarizing the most important AI safety research papers focused on frontier models, written by Johannes Gasteiger of Anthropic's Alignment Science team.

A UK nonprofit applying formal methods and machine learning to build open infrastructure for mathematically guaranteed AI safety assurance.

Safe Superintelligence Inc. (SSI) is an AI research company founded by Ilya Sutskever focused solely on building safe superintelligence, with no other products or commercial distractions.

Astera's Neuro & AGI program is an in-house research effort that draws on neuroscience to develop safe and aligned artificial general intelligence, operating under the Astera Institute founded by Jed McCaleb.

Calling on policymakers to implement a global moratorium on large AI training runs until alignment is solved.

An AI security accelerator and research lab based in San Francisco that invests in and supports early-stage startups building infrastructure for safe AGI deployment.

A curated online hub for researchers to discuss technical AI alignment research, operated by Lightcone Infrastructure. It serves as the primary venue for sharing and coordinating cutting-edge alignment ideas across organizations including MIRI, OpenAI, DeepMind, CHAI, and others.

Georgetown University think tank providing decision-makers with data-driven analysis on the security implications of emerging technologies.

Purdue University is a major public research university in West Lafayette, Indiana, whose computer science department has received AI safety funding for research on language model robustness and adversarial deception detection.

A research project that investigates decision-relevant questions about the future of artificial intelligence, including AI timelines, expert forecasts, and the potential societal impacts of advanced AI systems.

A nonprofit creating crowdsourced datasets of prosocial behaviors to train ethical AI systems, and building the Creed.Space platform for personalized constitutional AI alignment.

A foresight and emergency response nonprofit that monitors global catastrophic risks using AI-augmented analysis and expert forecasters, publishing weekly risk briefings and maintaining a reserve team for rapid crisis response.