The ARIA Lab (Aligned, Robust, and Interactive Autonomy Lab) at the University of Utah, led by Professor Daniel S. Brown, conducts research on human-AI alignment, reward learning, and AI safety. The lab develops algorithms and theory to enable AI systems to safely learn from and interact with humans.

Personal blog of Victoria Krakovna, Senior Research Scientist at Google DeepMind and co-founder of the Future of Life Institute, covering AI alignment research and related topics.

Stop AI is a grassroots activist organization that uses non-violent civil disobedience and public advocacy to demand a permanent, enforceable global ban on the further development of frontier AI technology.

CFG is an independent think-and-do tank based in Brussels that helps policymakers anticipate and govern powerful emerging technologies including advanced AI, biotechnology, climate interventions, and neurotechnology.

INESIA is France's national institute for AI evaluation and security, a government coordination structure that federates ANSSI, Inria, LNE, and PEReN to evaluate AI systems, analyze systemic risks, and support AI regulation.

A research project that uses game theory and computational modeling to reduce catastrophic risks from competition in the development of transformative AI.

AI Prospects is a Substack publication by K. Eric Drexler exploring how advanced AI will transform society and what strategic options humanity has for navigating this transition safely.

Constellation is a nonprofit research center in Berkeley that supports AI safety work through fellowships, an incubator, and a collaborative coworking space hosting researchers and organizations across the field.

A nonprofit dedicated to ensuring that today's most consequential technologies, including AI and social media, actually serve humanity by exposing misaligned incentives and advocating for systemic change through policy, litigation, and public awareness.

Substack newsletter by Helen Toner (Interim Executive Director at Georgetown's Center for Security and Emerging Technology and former OpenAI board member) offering analysis on navigating the transition to a world with extremely advanced AI systems.

CASA is a research organization working to ensure the benefits of AI can be widely and equitably distributed globally without compromising essential security, with a focus on Global Majority countries.

The Compendium is a living document and website that presents a comprehensive, accessible argument for why artificial general intelligence poses an extinction risk to humanity and what can be done about it.

An annual 4-day academic summer school held in Prague focused on teaching AI alignment research frameworks to PhD students, ML researchers, and advanced students.

An independent research project focused on proving formal impossibility results in AI alignment using theoretical computer science methods, led by Alexander Bistagne as a Ronin Institute Fellow.

An online forecasting platform and aggregation engine that harnesses collective intelligence to produce calibrated predictions on questions of global importance, including AI timelines, biosecurity, nuclear risk, and climate change.

A major public research university whose AI safety-relevant work is centered on the AI+Human Objectives Initiative (AHOI) and Scott Aaronson's computational-complexity-meets-alignment research group, both supported by Open Philanthropy.

Friedrich Schiller University Jena is a major German research university that hosts the LAMALab, a research group led by Dr. Kevin Jablonka focused on AI-accelerated materials discovery and LLM benchmarking in chemistry.

A nonprofit research organization founded by Nick Bostrom to study how present-day actions influence humanity's long-term future, with a focus on existential risk, AI safety, and AGI governance.

A nonprofit R&D lab working to ensure that AI and future economic systems are built and deployed with genuine human objectives at their core, through research, open-source tools, and broad public input.

A major public research university hosting several prominent AI safety research groups, including work on formal neural network verification, adversarial robustness, and AI agent security benchmarks.

A US-based public charity that collaborates with university research groups working to reduce existential risk by providing them with free operational services and support.

An organizational incubator that launches new nonprofits and projects working to steer transformative technology away from extreme large-scale risks. FLF identifies gaps in the AI safety ecosystem, recruits founders, and provides seed funding and operational support to new ventures.

A European non-profit that investigates influential and opaque algorithms, holding major tech platforms accountable through independent technical audits and free software auditing tools.

A nonprofit applied research organization building universal reasoning engines grounded in probabilistic programming and causal inference to advance society's ability to solve intractable scientific and societal problems.

Lethal Intelligence is an AI risk awareness media project producing original explainer films, podcasts, and social media content about the existential dangers of advanced AI systems.

A French nonprofit research organization working alongside government institutions to address the security and international coordination challenges posed by general-purpose AI development.

Effective Altruism Sweden is a Stockholm-based nonprofit that builds the Swedish effective altruism community through career coaching, fellowship programs, and project incubation. Founded in 2016, it is one of the most established national EA organizations globally.

A university research lab at the University of Rhode Island directed by Dr. Sarah M Brown, studying how machine learning interacts with complex socio-technical systems, with a focus on fairness of automated decision-making and AI safety evaluation.

A research-focused think-and-do tank that conducts empirical research across animal welfare, global health and development, AI, and other cause areas to uncover high-impact, neglected opportunities for improving the lives of humans and animals.

WhiteBox Research is a Manila-based nonprofit that trains early-career researchers in mechanistic interpretability and AI safety, with a focus on building research capacity in Southeast Asia.

A free in-person bootcamp in Switzerland introducing students and early-career researchers to AI safety through technical and conceptual coursework. The camp covers alignment, mechanistic interpretability, and governance tracks.

No summary available yet.

The Vista Institute for AI Policy builds AI law and policy as an academic field and develops talent for careers in AI governance, with a focus on promoting risk-mitigating U.S. regulation.

A research center at UC Berkeley dedicated to developing the foundations of provably beneficial AI systems, ensuring that advanced AI remains aligned with human values and preferences.

A Dutch foundation (stichting) that uses legal expertise and advocacy within Europe to promote safer development and deployment of frontier technologies including AI, biotechnology, and nuclear technology.

OpenMined is a 501(c)(3) nonprofit building open-source privacy-preserving AI infrastructure that enables secure computation across siloed data. Their tools allow AI auditors and researchers to evaluate proprietary AI systems without requiring direct access to sensitive models or data.

Stanford University is a leading research university hosting several AI safety-relevant programs, including the Human-Centered AI Institute (HAI), the Existential Risks Initiative (SERI), the Center for International Security and Cooperation (CISAC), and the Center for AI Safety.

A database and website maintained by Issa Rice that tracks people, organizations, and products in the AI safety and alignment field.

Upgradable is an applied research lab and life optimization service that helps effective altruists, AI safety researchers, and existential risk advocates lead more impactful lives.

RAISEimpact is a consulting program that helps AI safety organizations strengthen their management, leadership, and organizational culture to amplify their effectiveness.

A nonprofit, nonpartisan think tank founded in 2011 that conducts research and policy work on risks that could significantly harm or destroy human civilization, including AI, nuclear war, climate change, and asteroid impacts.

Probably Good is a nonprofit that helps individuals build high-impact careers through free, evidence-based guides, 1-on-1 advising, and a curated job board.

A European multi-donor foundation that seeds and scales high-impact initiatives for the secure and beneficial development of AI. Astralis unites funders, experts, and entrepreneurs to steer AI toward beneficial outcomes through grantmaking, strategic guidance, and network-building.

Theorem Labs is an AI and programming languages research lab that builds tools to formally verify the correctness of AI-generated code before it ships.

No summary available yet.

AIGS Canada is a nonpartisan Canadian not-for-profit working to ensure that advanced AI is safe and beneficial for all, by catalysing Canadian leadership in AI governance and safety.

FIG runs a part-time, remote-first 12-week research fellowship connecting early-to-mid-career researchers with experienced project leads working on AI safety, AI governance, and AI sentience.

AIES is a peer-reviewed academic conference series jointly organized by AAAI and ACM that brings together a multidisciplinary community to examine the ethical, social, and policy dimensions of artificial intelligence.

The National Academies of Sciences, Engineering, and Medicine is the United States' preeminent independent scientific advisory body, providing expert consensus reports to inform government policy on science, engineering, and medicine, including AI safety and governance.