A major Washington, DC-based think tank founded in 1910 that produces independent policy research on international security, democracy, and governance, with a growing program on AI safety and technology governance.

Successif helps mid-career and senior professionals transition into high-impact careers in AI safety and governance through free personalized advising, workshops, and job market research.

Softmax is an AI alignment research startup developing the science of organic alignment through multi-agent reinforcement learning. Founded by Emmett Shear, Adam Goldstein, and David Bloomin, the company studies how agents learn to cooperate, share goals, and form collectively intelligent systems.

Singapore's national AI safety institute, operated by the Digital Trust Centre at NTU in partnership with IMDA, focusing on AI evaluation, testing, and governance to address gaps in global AI safety science.

A major public research university in Ann Arbor, Michigan, hosting faculty conducting AI safety and alignment research funded by organizations including Open Philanthropy.

Epoch AI is a nonprofit research institute that tracks and forecasts the trajectory of artificial intelligence by analyzing trends in compute, data, algorithmic efficiency, and capabilities. It produces leading databases and quantitative models to help policymakers, researchers, and funders understand the pace and impact of AI progress.

Guide Labs builds interpretable AI systems and foundation models that humans can reliably understand, audit, and steer. Their flagship model, Steerling-8B, is the first inherently interpretable large language model at scale.

AXRP is a podcast hosted by Daniel Filan featuring in-depth interviews with AI safety researchers about their published work and how it might reduce the risk of AI causing an existential catastrophe.

Oregon State University is a public research university in Corvallis, Oregon, whose hardware security research group contributed to AI compute governance through the Survival and Flourishing Fund's FlexHEG (Flexible Hardware-Enabled Guarantees) program.

Americans for Responsible Innovation (ARI) is a bipartisan 501(c)(4) nonprofit that advocates for thoughtful AI governance frameworks in the United States. It works to help policymakers develop policies that protect the public from AI-related harms while maintaining American technological leadership.

Explainable backs content creators shaping how the world understands AI, running fellowships and campaigns to communicate AI safety research to broader audiences.

Mythos Ventures is an early-stage venture capital firm investing in prosocial technologies and safe AI systems. They back pre-seed and seed-stage founders building AGI-resilient, positive-impact companies.

The ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT) is a premier peer-reviewed academic conference that brings together researchers and practitioners to investigate fairness, accountability, and transparency in socio-technical systems.

An AI safety research lab studying how software and industrial systems recursively improve themselves, building benchmarks and evaluation frameworks to understand the behavior and limits of self-improving AI systems.

UC Berkeley's multidisciplinary research center advancing AI safety, agentic AI, and decentralization technology to empower a responsible digital economy.

A nonprofit dedicated to ensuring the internet and AI remain open, trustworthy, and accountable to people rather than corporations. Distinct from the Mozilla Corporation, it funds research, fellowships, and values-aligned technology through grantmaking and advocacy.

One of the world's oldest and most prestigious universities, founded in 1209, and a major hub for AI safety and existential risk research through centers such as CSER and the Leverhulme Centre for the Future of Intelligence.

ICML is the premier annual academic conference for machine learning research, bringing together researchers from academia and industry worldwide. It is organized by the International Machine Learning Society (IMLS), a 501(c)(3) nonprofit.

EquiStamp is a Public Benefit Corporation that provides evaluation implementation, data annotation, red/blue teaming, and operational support so AI safety researchers can focus on research rather than logistics.

Lionheart Ventures is a seed-stage venture capital firm investing in transformative artificial intelligence and frontier mental health technologies to mitigate civilizational risk.

A personal Substack newsletter by AI safety researcher Daniel Paleka covering recent AI safety research papers and technical developments.

Penn State University hosts AI safety research led by Prof. Rui Zhang, whose group received Open Philanthropy funding to develop methods for detecting and mitigating sandbagging in AI systems.

No summary available yet.

A research initiative at the University of Virginia, led by Professor Anton Korinek, that produces and disseminates cutting-edge economic research to help society navigate the transition to transformative AI and guide it toward shared prosperity.

MIT FutureTech is an interdisciplinary research group at MIT CSAIL studying the economic and technical foundations of progress in computing and AI. The group produces rigorous insights on AI trends, risks, and impacts to inform policy, industry, and scientific funding decisions.

ENAIS connects AI safety researchers, field-builders, and policymakers across Europe to improve coordination and reduce the fragmentation of the continent's AI safety ecosystem.

A nonprofit research organization that builds open-source tools and conducts research on forecasting, epistemics, and uncertainty quantification to improve decision-making for the long-term future of humanity.

Leaf runs online fellowships for exceptional teenagers (ages 15-19) to explore how they can have the most positive impact, including through a flagship course on AI safety called Dilemmas and Dangers in AI.

A Washington, D.C.-based 501(c)(3) nonprofit that educates the public, policymakers, and media about the risks of advanced AI and advocates for bipartisan safeguards before AGI arrives.

The National Science Foundation (NSF) is an independent US federal agency that funds basic research and education across all non-medical fields of science and engineering, including substantial investment in AI safety-relevant research.

A leading AI podcast hosted by Nathan Labenz that interviews AI builders, researchers, and investors to help leaders make sense of transformative developments in artificial intelligence.

The AI & Democracy Foundation accelerates innovation, evaluation, and adoption of deliberative, democratic, human-centered governance and alignment systems for and with AI, serving as both a nonprofit funder and advisor to philanthropic organizations, AI companies, civil society, and governments.

Japan's national AI Safety Institute (J-AISI), established within IPA under METI, develops AI safety evaluation methodologies and standards and serves as a hub for domestic and international AI safety coordination.

CIGI is an independent, non-partisan Canadian think tank that produces research and policy recommendations on international governance challenges, with a dedicated program focused on managing global-scale risks from advanced AI systems.

An umbrella organization for applied mathematics research in AI alignment, now operating under the name Iliad. Organizes the ILIAD conference series, runs fellowship and intensive programs, incubates research organizations, and manages scientific publishing.

Good Impressions is a grant-funded digital marketing agency that applies for-profit growth techniques to help effective nonprofits, think tanks, and foundations maximize engagement with their work.

ARENA is a 4-5 week intensive ML engineering bootcamp in London that trains technically skilled individuals to contribute to AI safety research. It covers deep learning fundamentals, mechanistic interpretability, reinforcement learning, and model evaluations.

UC Santa Barbara is a major public research university whose Center for Responsible Machine Learning conducts AI safety-adjacent research on fairness, bias, transparency, and the societal impacts of AI systems.

A 501(c)(3) nonprofit that accelerates neglected approaches to AI alignment by providing researchers with engineering teams, compute resources, and operational infrastructure.

IAIGA is a Geneva-based non-profit initiative working to establish a supranational AI governance body and legally-binding global treaty to ensure AI safety and equitable distribution of AI-derived benefits.

Surge AI is a data labeling and AI training data company that provides high-quality human annotation, RLHF datasets, and adversarial red-teaming services to frontier AI labs including Anthropic, OpenAI, Google, Microsoft, and Meta.

Samotsvety is an elite team of superforecasters applying rigorous probability analysis to high-stakes questions in AI risk, nuclear risk, and existential risk. They are widely regarded as one of the best forecasting teams in the world.

Siliconversations is a YouTube channel that creates animated videos explaining AI safety risks and existential risk from advanced AI to general audiences. It is run by a former quantum scientist who became a full-time content creator.

A solo-researcher company founded by Koen Holtman that conducts AI safety research and participates in the creation of European AI safety standards in support of the EU AI Act.

A major public research university in Madison, Wisconsin, home to AI safety relevant research including interpretability work in the Statistics department and student-led AI safety initiatives.

UMass Amherst is a public research university whose AI safety-relevant work is centered in the SCALAR Lab, led by Associate Professor Scott Niekum, which focuses on safe and aligned machine learning and robotics.

Effective Altruism Geneva is a Swiss nonprofit community group based in Geneva that builds a local network of effective altruists and fosters high-impact careers in AI safety, policy, and global health.

A Sheffield-based hardware engineering consultancy focused on differential technology development across AI safety, biosecurity, humane tech, and accelerating science.

No summary available yet.