An AI safety research lab studying how software and industrial systems recursively improve themselves, building benchmarks and evaluation frameworks to understand the behavior and limits of self-improving AI systems.

UC Berkeley's multidisciplinary research center advancing AI safety, agentic AI, and decentralization technology to empower a responsible digital economy.

A nonprofit dedicated to ensuring the internet and AI remain open, trustworthy, and accountable to people rather than corporations. Distinct from the Mozilla Corporation, it funds research, fellowships, and values-aligned technology through grantmaking and advocacy.

One of the world's oldest and most prestigious universities, founded in 1209, and a major hub for AI safety and existential risk research through centers such as CSER and the Leverhulme Centre for the Future of Intelligence.

ICML is the premier annual academic conference for machine learning research, bringing together researchers from academia and industry worldwide. It is organized by the International Machine Learning Society (IMLS), a 501(c)(3) nonprofit.

EquiStamp is a Public Benefit Corporation that provides evaluation implementation, data annotation, red/blue teaming, and operational support so AI safety researchers can focus on research rather than logistics.

Lionheart Ventures is a seed-stage venture capital firm investing in transformative artificial intelligence and frontier mental health technologies to mitigate civilizational risk.

A personal Substack newsletter by AI safety researcher Daniel Paleka covering recent AI safety research papers and technical developments.

Penn State University hosts AI safety research led by Prof. Rui Zhang, whose group received Open Philanthropy funding to develop methods for detecting and mitigating sandbagging in AI systems.

A research initiative at the University of Virginia, led by Professor Anton Korinek, that produces and disseminates cutting-edge economic research to help society navigate the transition to transformative AI and guide it toward shared prosperity.

MIT FutureTech is an interdisciplinary research group at MIT CSAIL studying the economic and technical foundations of progress in computing and AI. The group produces rigorous insights on AI trends, risks, and impacts to inform policy, industry, and scientific funding decisions.

ENAIS connects AI safety researchers, field-builders, and policymakers across Europe to improve coordination and reduce the fragmentation of the continent's AI safety ecosystem.

A nonprofit research organization that builds open-source tools and conducts research on forecasting, epistemics, and uncertainty quantification to improve decision-making for the long-term future of humanity.

Leaf runs online fellowships for exceptional teenagers (ages 15-19) to explore how they can have the most positive impact, including through a flagship course on AI safety called Dilemmas and Dangers in AI.

A Washington, D.C.-based 501(c)(3) nonprofit that educates the public, policymakers, and media about the risks of advanced AI and advocates for bipartisan safeguards before AGI arrives.

The National Science Foundation (NSF) is an independent US federal agency that funds basic research and education across all non-medical fields of science and engineering, including substantial investment in AI safety-relevant research.

A leading AI podcast hosted by Nathan Labenz that interviews AI builders, researchers, and investors to help leaders make sense of transformative developments in artificial intelligence.

The AI & Democracy Foundation accelerates innovation, evaluation, and adoption of deliberative, democratic, human-centered governance and alignment systems for and with AI, serving as both a nonprofit funder and advisor to philanthropic organizations, AI companies, civil society, and governments.

Japan's national AI Safety Institute (J-AISI), established within IPA under METI, develops AI safety evaluation methodologies and standards and serves as a hub for domestic and international AI safety coordination.

CIGI is an independent, non-partisan Canadian think tank that produces research and policy recommendations on international governance challenges, with a dedicated program focused on managing global-scale risks from advanced AI systems.

An umbrella organization for applied mathematics research in AI alignment, now operating under the name Iliad. Organizes the ILIAD conference series, runs fellowship and intensive programs, incubates research organizations, and manages scientific publishing.

Good Impressions is a grant-funded digital marketing agency that applies for-profit growth techniques to help effective nonprofits, think tanks, and foundations maximize engagement with their work.

ARENA is a 4-5 week intensive ML engineering bootcamp in London that trains technically skilled individuals to contribute to AI safety research. It covers deep learning fundamentals, mechanistic interpretability, reinforcement learning, and model evaluations.

UC Santa Barbara is a major public research university whose Center for Responsible Machine Learning conducts AI safety-adjacent research on fairness, bias, transparency, and the societal impacts of AI systems.

A 501(c)(3) nonprofit that accelerates neglected approaches to AI alignment by providing researchers with engineering teams, compute resources, and operational infrastructure.

IAIGA is a Geneva-based non-profit initiative working to establish a supranational AI governance body and legally-binding global treaty to ensure AI safety and equitable distribution of AI-derived benefits.

Surge AI is a data labeling and AI training data company that provides high-quality human annotation, RLHF datasets, and adversarial red-teaming services to frontier AI labs including Anthropic, OpenAI, Google, Microsoft, and Meta.

Samotsvety is an elite team of superforecasters applying rigorous probability analysis to high-stakes questions in AI risk, nuclear risk, and existential risk. They are widely regarded as one of the best forecasting teams in the world.

Siliconversations is a YouTube channel that creates animated videos explaining AI safety risks and existential risk from advanced AI to general audiences. It is run by a former quantum scientist who became a full-time content creator.

A solo-researcher company founded by Koen Holtman that conducts AI safety research and participates in the creation of European AI safety standards in support of the EU AI Act.

A major public research university in Madison, Wisconsin, home to AI safety relevant research including interpretability work in the Statistics department and student-led AI safety initiatives.

UMass Amherst is a public research university whose AI safety-relevant work is centered in the SCALAR Lab, led by Associate Professor Scott Niekum, which focuses on safe and aligned machine learning and robotics.

Effective Altruism Geneva is a Swiss nonprofit community group based in Geneva that builds a local network of effective altruists and fosters high-impact careers in AI safety, policy, and global health.

A Sheffield-based hardware engineering consultancy focused on differential technology development across AI safety, biosecurity, humane tech, and accelerating science.

A nonprofit think tank researching the expansion of humanity's moral circle, with a primary focus on digital minds and the moral status of AI systems.

Juniper Ventures is a pre-seed venture capital firm that invests in startups explicitly working to make AI secure and beneficial for humanity.

A small nonprofit research organization studying global catastrophic risks, best known for its insight-based AI timelines model and research on the feasibility of training AGI via deep reinforcement learning.

An expert-managed grantmaking fund that supports projects building the effective altruism community's capacity, including community building, prioritization research, epistemic infrastructure, events, and fundraising for effective charities.

An interdisciplinary research lab at Carnegie Mellon University, directed by Simon DeDeo, that studies complex social systems through mathematical modeling and empirical investigation to better understand humanity's past, present, and future.

A nonprofit organization working to steer transformative technologies -- particularly AI, biotechnology, and nuclear weapons -- away from extreme large-scale risks and towards benefiting life.

A community blog and forum devoted to refining the art of human rationality, with major focus areas including AI alignment, cognitive biases, decision-making, and effective altruism.

The Responsible AI Collaborative (TheCollab) is a nonprofit that maintains the AI Incident Database (AIID), the leading public repository of documented real-world AI harms and near-harms.

AI Explained is a London-based YouTube channel by a creator known as Philip that provides hype-free coverage of AI developments, capabilities, and safety topics for a general audience.

Founders Pledge empowers entrepreneurs to pledge a portion of their exit proceeds to high-impact charities, and conducts cause-area research and grantmaking to guide that giving.

ARLIS is the University of Maryland's Department of Defense University Affiliated Research Center (UARC) dedicated to intelligence and national security, combining AI, behavioral science, and systems engineering to address complex security challenges.

Oxford-based AI safety company developing concept extrapolation technology to enable AI systems to generalize human values and intent beyond their training data.

Anthropic is an AI safety company and public benefit corporation building reliable, interpretable, and steerable AI systems, best known for developing the Claude family of large language models.

Swiss nonprofit funder making grants and investments to reduce suffering risks from catastrophic AI misuse, AI conflict, and other large-scale harms. Formerly known as Center for Emerging Risk Research (CERR) and Polaris Ventures.

Imperial College London is a world-leading research university specialising in science, technology, engineering, medicine, and business, with significant programs in AI safety, trustworthy AI, and long-term AI risk research.