Database

6-month stipend to do an unpaid internship focused on using theory/interpretability to increase the safety of AI systems

No summary available yet.

A research project that uses game theory and computational modeling to reduce catastrophic risks from competition in the development of transformative AI.

Artyom (Artem) Karpov is an independent AI safety researcher and ML engineer based in Istanbul, Turkey. He holds a degree in Applied Mathematics and has over 15 years of software engineering experience, having previously built real-time emergency response systems and contributed to .NET Core. He transitioned to AI safety research in 2022 after becoming interested in the field through the 80,000 Hours career guide, and has since completed the MATS, ARENA, MLSS, and Apart Fellowship programs. He participated in AI Safety Camp (2023), where he worked on the project "Inducing Human-Like Biases in Moral Reasoning Language Models," which resulted in a paper accepted at a NeurIPS workshop. His subsequent research has focused on LLM steganography and encoded reasoning in chain-of-thought, with papers accepted at AAAI, ICLR, and NeurIPS workshops. He has received early-career funding from Open Philanthropy (via Good Ventures Foundation) and has contributed evaluations to the UK AI Security Institute.

No summary available yet.

AI Prospects is a Substack publication by K. Eric Drexler exploring how advanced AI will transform society and what strategic options humanity has for navigating this transition safely.

Constellation is a nonprofit research center in Berkeley that supports AI safety work through fellowships, an incubator, and a collaborative coworking space hosting researchers and organizations across the field.

A nonprofit dedicated to ensuring that today's most consequential technologies, including AI and social media, actually serve humanity by exposing misaligned incentives and advocating for systemic change through policy, litigation, and public awareness.

Projection, sycophancy, and institutional artifact fabrication in AI-mediated supervision.

Substack newsletter by Helen Toner (Interim Executive Director at Georgetown's Center for Security and Emerging Technology and former OpenAI board member) offering analysis on navigating the transition to a world with extremely advanced AI systems.

No summary available yet.

No summary available yet.

No summary available yet.

No summary available yet.

Alexander (Sasha) Bystritsky, M.D., Ph.D., is a psychiatrist and neuroscientist who serves as President of the Institute for Advanced Consciousness Studies. He is Professor Emeritus of Psychiatry and Biobehavioral Sciences at the David Geffen School of Medicine at UCLA and is widely known for his work on anxiety disorders, focused ultrasound, and neuromodulation-based treatments.

Paul Saffo is a Silicon Valley-based forecaster who studies the dynamics of large-scale, long-term technological change. He teaches forecasting as an Adjunct Professor in Stanford University’s School of Engineering, chairs the Future Studies track at Singularity University, and is a non-resident Senior Fellow at the Atlantic Council and a Fellow of the Royal Swedish Academy of Engineering Sciences.

No summary available yet.

CASA is a research organization working to ensure the benefits of AI can be widely and equitably distributed globally without compromising essential security, with a focus on Global Majority countries.

Tristan Harris is a technology ethicist and co‑founder of the Center for Humane Technology, a nonprofit whose mission is to align technology with humanity’s best interests. A former Google design ethicist, he now focuses on how major platforms and AI systems shape society, co‑hosts the podcast Your Undivided Attention, and was a prominent voice in the Netflix documentary The Social Dilemma.

The Compendium is a living document and website that presents a comprehensive, accessible argument for why artificial general intelligence poses an extinction risk to humanity and what can be done about it.

No summary available yet.

An annual 4-day academic summer school held in Prague focused on teaching AI alignment research frameworks to PhD students, ML researchers, and advanced students.

Yoav Tzfati is an AI safety researcher and software engineer based in Berkeley, California. He is a MATS 5.0 alumnus who worked on scalable oversight research, specifically on experimental methodology for evaluating AI alignment techniques including Consultancy and Critiques in synthetic settings, mentored by Julian Michael of the NYU Alignment Research Group. He subsequently joined the Security Level 5 (SL5) Task Force at the Institute for Security and Technology as a Member of Technical Staff, focusing on supply chain and machine security, and contributes to developing the SL5 standard for securing AI data centers. He is also a mentor for SPAR Spring 2026 projects related to AI security and safety infrastructure. Prior to his AI safety work, he drove engineering for attack surface discovery automation at CyCognito and served as Tech Lead at Arbor Trading Bootcamp. He has spoken at the Berlin AI Safety Meetup on his scalable oversight research and has also developed educational programs teaching non-programmers to build full-stack applications using AI tools.

Lukas Berglund is an AI safety researcher currently serving as Technical Staff at the U.S. Center for AI Standards and Innovation (CAISI) at NIST. He is best known as the lead author of "The Reversal Curse: LLMs trained on 'A is B' fail to learn 'B is A'," published at ICLR 2024, which demonstrated a fundamental generalization failure in autoregressive large language models. He also co-authored "Taken out of context: On measuring situational awareness in LLMs," an influential paper exploring how models recognize whether they are in training or deployment. His research was conducted in part as a MATS Fellow through the SERI MATS program, with support from Open Philanthropy. He has an undergraduate background from Vanderbilt University and his work spans AI evaluation, AI security, and empirical research on the capabilities and failure modes of frontier AI systems.

Measuring whether AI can autonomously execute multi-stage cyberattacks to inform deployment decisions at frontier labs

Meghna Mann is President and Chief Operating Officer at Constellation Institute, overseeing programs and operations that strengthen AI safety talent pipelines and support the launch and growth of mission-aligned organizations. Previously, she held senior leadership roles at MetaMap—including serving as COO and later CEO of the identity-verification company—after earlier positions at BlackRock and the Brookings Institution, and she advises high-growth technology ventures through the Endeavor Global network.

No summary available yet.

Research Scientist at the UK AI Security Institute whose work focuses on bridging immediate AI harms and longer-term catastrophic risks in AI safety.

An independent research project focused on proving formal impossibility results in AI alignment using theoretical computer science methods, led by Alexander Bistagne as a Ronin Institute Fellow.

No summary available yet.

No summary available yet.

No summary available yet.

Alignment/digital minds researcher at AE Studio

An online forecasting platform and aggregation engine that harnesses collective intelligence to produce calibrated predictions on questions of global importance, including AI timelines, biosecurity, nuclear risk, and climate change.

Manifund's account for Mox, a coworking & events space in SF

Historian of Ideas focused on the history of AI.

No summary available yet.

Gergő Gáspár is a community builder with an academic background in psychology. Since 2019 he has grown EA organising work from a university group into the national organisation EA Hungary, founded AI Safety Hungary, and moved into full-time community building in 2021. He has served as a part-time Director at the European Network for AI Safety, co-founded Amplify, an EA-aligned digital marketing agency supporting fieldbuilding organisations, previously volunteered as a charity analyst and analysis coordinator at SoGive, and now directs Effective Altruism UK while writing the Building Capacity Substack on fieldbuilding strategy, careers and marketing.

Research Scholar at ILINA and Research Fellow at the Centre for AI Risk Management and Alignment (CARMA), where she works on AI liability regimes and maps whistleblowing channels and legal protections in the US, UK, and EU; she has co‑authored work on why Global South countries should care about highly capable AI and holds an undergraduate law degree from Strathmore University.

Funds for a 6-month project contributing to the clarification of goal-directedness

A major public research university whose AI safety-relevant work is centered on the AI+Human Objectives Initiative (AHOI) and Scott Aaronson's computational-complexity-meets-alignment research group, both supported by Open Philanthropy.

Friedrich Schiller University Jena is a major German research university that hosts the LAMALab, a research group led by Dr. Kevin Jablonka focused on AI-accelerated materials discovery and LLM benchmarking in chemistry.

Samuel Marks is a board member of the Cambridge Boston Alignment Initiative and leads the cognitive oversight subteam on Anthropic’s alignment science team, working on methods to oversee AI systems by analyzing their internal cognitive processes.

No summary available yet.

Iván and Jett are seeking funding to research unfaithful chain-of-thought, under Arthur Conmy's mentorship, for a month before the start of MATS.

No summary available yet.

No summary available yet.

A nonprofit research organization founded by Nick Bostrom to study how present-day actions influence humanity's long-term future, with a focus on existential risk, AI safety, and AGI governance.

Katie McMahon is a global technology executive and entrepreneur with more than two decades of experience at the forefront of sound recognition and natural language understanding, including senior roles at Shazam and SoundHound. She now advises and consults for early-stage AI and voice-technology companies and serves as a researcher and member of the Berryville Institute of Machine Learning, contributing to work on safe, secure, and ethical AI systems.