Database

Upskilling investigation of AI Safety via debate and ML training

A concrete safety experiment to detect when an LLM's local reasoning stops behaving like a single stable executive stream, using scalar hazard signals.

Yatharth Agarwal is a member of the Softmax team with a background as cofounder of Stanford AI Group and as a screentime addiction researcher. He created the first high-school curriculum and a for-credit Stanford class on AI safety, co-taught AI ethics for faculty, led an AI regulation seminar at Stanford Law, and previously led AI at Axle Health while writing extensively on self-alignment in humans.

Founder of Ergo Impact and board member at AVERI, Max Henderson is a philanthropist and investor who has helped incubate, operate, and fund close to $1B of social good efforts across areas such as global health, science, nuclear security, and AI, after earlier product and go-to-market roles at organizations including CovidActNow, Firebase, Google, Oracle, and Compass.

Transitioning into AI governance/safety/risk from national security

Mathematician focused on bringing category-theoretic ideas into science, technology, and society; after earning his PhD from UC Berkeley in 2007 he has pursued novel mathematical research with collaborators in fields such as materials science, robotics, aeronautics, and computer science and co-founded the Topos Institute and Weve.

No summary available yet.

PhD in machine learning with a focus on AI alignment

No summary available yet.

I work on AI safety at the Alignment Research Center

Lennart Heim is an independent researcher focused on semiconductors and AI policy, based in Washington, DC. He previously served as an Associate Information Scientist at the RAND Corporation, where he led the Compute Team at RAND's Center on AI, Security, and Technology and held a faculty position as Professor of Policy Analysis at the Pardee RAND Graduate School. He is an Adjunct Fellow at the Centre for the Governance of AI (GovAI), where he previously held a Senior Research Fellow role leading the Compute Governance initiative. He is also a co-founder of Epoch AI, a research organization tracking trends in machine learning compute, data, and hardware. His research centers on the role of computing power in advanced AI systems and how compute can be leveraged as an instrument for AI governance, including work on training compute thresholds, export controls, data center policy, and AI diffusion frameworks. He has a background in computer engineering from ETH Zurich and RWTH Aachen, is a member of the OECD.AI Expert Group on AI Compute and Climate, and is an active contributor to EA Forum discussions on compute governance and AI safety.

General Support of Milan Griffes to pursue an MHS in Mental Health

An experimental AI-generated sci-fi film dramatising AI safety choices. Using YT interactivity to get ≈880 conscious AI safety decisions per 1k viewers.

No summary available yet.

Siméon Campos is the founder of SaferAI and now serves as an advisor and chair of the board, contributing to work on AI standards, risk management research, fundraising, and external partnerships. He is a technical expert on AI risk management at ISO/IEC and CEN-CENELEC and led the development of SaferAI’s ratings of AI developers’ risk-management maturity. Before founding SaferAI, he co-founded EffiSciences, an organisation providing responsible-AI training that has been used to train hundreds of people across France and Europe, and he is a frequent commentator on AI governance and risk management in forums such as TIME, Euractiv, OECD and Le Monde.

No summary available yet.

No summary available yet.

Tiffany M. Moore is senior vice president of political and industry affairs at the Consumer Technology Association (CTA), where she leads federal advocacy on technology policy and oversees initiatives to expand diversity, inclusion, and innovation within the consumer technology sector.

No summary available yet.

FIG runs a part-time, remote-first 12-week research fellowship connecting early-to-mid-career researchers with experienced project leads working on AI safety, AI governance, and AI sentience.

No summary available yet.

No summary available yet.

9 months of funding for an early-career alignment researcher, to work with Owain Evans and others.

3-month stipend during post-SERI MATS alignment job search and wrapping up paper w/ mentor

No summary available yet.

No summary available yet.

Joe Kwon is an AI safety researcher and policy analyst based in Washington, DC. He holds a BS in Computer Science and Psychology from Yale University and has conducted research at MIT's Computational Cognitive Science Lab, where he studied moral and social cognition with Josh Tenenbaum and Sydney Levine. His technical background includes early RLHF work at OpenAI, empirical ML research at UC Berkeley with Jacob Steinhardt and Dan Hendrycks focused on evals and out-of-distribution detection, and a stint as a Research Engineer at LG AI Research working on multilingual large language models. He subsequently transitioned to AI governance work, completing a GovAI DC Fellowship focused on risks from internal AI deployment and automated R&D, and serving as a Technical Policy Analyst at the Center for AI Policy (CAIP). Most recently he has been an Astra Fellow working with Tom Davidson and Fabien Roger on threat modeling and ML experiments related to secretly loyal AI. He received a Long-Term Future Fund grant as a stipend to work on an ML safety project with the goal of joining an ML safety team full-time.

No summary available yet.

Michel Justen is the AI Grants Lead at the Tarbell Center for AI Journalism, an organization funded by Coefficient Giving (formerly Open Philanthropy), where he manages the center's reporting grants program. Previously, he was a 2024 Research Fellow at Pivotal Research, where he studied international AI governance and benefit-sharing — specifically how the US government might share financial or access benefits from advanced AI to unlock international safety agreements, mentored by Matthew van der Merwe and Max Dalton. Before that, he worked as an Events Associate at the Centre for Effective Altruism, running events including the Summit on Existential Security and the New Orleans Alignment Workshop, and co-founded and scaled the EA Opportunity Board. He also founded the EA student group at the University of Wisconsin–Madison, where he studied Neurobiology and Psychology. He received a Long-Term Future Fund grant to produce an animated video essay explaining how AI could accelerate AI R&D and the implications for AI governance. He writes on AI governance topics through his Substack newsletters 'What is this' and 'With (some) respect to AI.'

Kai Fronsdal is an AI safety researcher based in the San Francisco Bay Area, currently affiliated with Meridian Research Labs. He completed a masters degree at Stanford University studying mathematics and computer science, where he was also involved with Stanford AI Alignment (SAIA). He participated in MATS 6.0 (Summer 2024) under the mentorship of David Lindner, focusing on measuring instrumental self-reasoning in frontier models as a precursor to deceptive alignment. His primary research output from this period is the paper "MISR: Measuring Instrumental Self-Reasoning in Frontier Models" (NeurIPS 2024), which proposes evaluation tasks for assessing how well LLM agents can engage in instrumental self-reasoning across scenarios including self-modification and knowledge-seeking. He also contributed to Anthropic's alignment auditing tools as an Anthropic Fellow, co-authoring "Petri: An open-source auditing tool to accelerate AI safety research" and the Bloom behavioral evaluation framework, as well as AuditBench, a benchmark for evaluating alignment auditing techniques on models with hidden behaviors. He received a grant from the Long-Term Future Fund to conduct deceptive alignment evaluation research and explore control and mitigation strategies.

Systems Architect | Building The Verified Loop: A Cyber-Physical Protocol for Deterministic AI Safety.

Deger Turan is the CEO of Metaculus, specializing in forecasting technological uncertainty and its impact on society and the environment. Before joining Metaculus as CEO in 2024, he served as President of the AI Objectives Institute, where he led development of Talk to the City, a platform that strengthens communication between under-resourced communities and the government officials serving them. Prior to AOI, he founded Cerebra Technologies, which forecasted shifts of public opinion and demand trends for hundreds of millions of citizens and was used by governments, hedge funds, and international retailers.

No summary available yet.

Research Associate on the Frontier Security team at the Institute for AI Policy and Strategy. Previously served as Co-director of the Cambridge AI Safety Hub, where he ran the MARS (Mentorship for Alignment Researchers) programme, interned at the World Bank, and completed dual BAs in Philosophy, Politics and Economics and in Economics at Claremont McKenna College.

4-month part-time salary to work on interpretability projects with David Bau and Logan Riggs

Condoleezza Rice is the Tad and Dianne Taube Director of the Hoover Institution and Denning Professor in Global Business and the Economy at Stanford Graduate School of Business. She is also a founding partner of Rice, Hadley, Gates & Manuel LLC, where she advises companies on international strategy, drawing on her service as the 66th U.S. Secretary of State and previously as National Security Advisor to President George W. Bush.

No summary available yet.

No summary available yet.

David Manheim is the founder and head of research and policy at ALTER (Association for Long Term Existence and Resilience), an academic research and advocacy organization based in Israel, and a visiting lecturer at the Technion - Israel Institute of Technology. He holds a PhD in risk analysis and decision theory from the Frederick S. Pardee RAND Graduate School and an undergraduate degree in mathematics from Lander College. Before pivoting to policy research, he worked in equity derivatives risk at an investment bank and built reinsurance terrorism risk models. His research focuses on mitigating large-scale risks to humanity, with particular emphasis on AI governance and safety, biosecurity and pandemic prevention, systemic risks, and geopolitical and technological forecasting. He is widely known in the AI safety community for co-authoring the influential paper "Categorizing Variants of Goodhart's Law" with Scott Garrabrant, and for his work on multiparty dynamics and failure modes for machine learning systems. He is a Superforecaster with Good Judgement Inc. and was one of the original participants in the Good Judgement Project IARPA forecasting competition. He has received grants from the Long-Term Future Fund for research on AI risk structure and multi-model approaches to x-risk mitigation, and advises organizations including Bluedot Impact, the Foresight Institute grant program, and the Unjournal.

A benchmark for studying how failures spread across multi-agent AI systems and whether they can be detected and interrupted in time.

Michelle Yi is a technology leader specializing in artificial intelligence and machine learning, with about 15 years of experience in technology consulting and applied AI. She has served as Senior Director for Applied Artificial Intelligence at RelationalAI and later as Head of Applied AI at Able, and co-founded Generationship. Michelle serves on multiple advisory boards, sits on the board of Women in Data, volunteers with programs such as Girls Who Code, and is affiliated with Basis Research Institute in the broader AI ecosystem.

AIES is a peer-reviewed academic conference series jointly organized by AAAI and ACM that brings together a multidisciplinary community to examine the ethical, social, and policy dimensions of artificial intelligence.

AI governance and safety researcher and policy analyst who leads the AI Risk Index at MIT FutureTech and The University of Queensland, using systematic evidence synthesis and policy engagement to inform decision‑makers about high‑priority AI risks and mitigations.

I create films about AI.

4-month salary and office for MATS 5.0 extension on adversarial circuit evaluation + 2-month runway for career switch

Doctoral student in Computer Science at MIT CSAIL advised by Dylan Hadfield-Menell in the Algorithmic Alignment Group, broadly interested in AI alignment and reinforcement learning, including neurosymbolic and multi-agent RL.

No summary available yet.

Daniel Kestenholz is Chief Operating Officer at Macroscopic Ventures, overseeing the organization’s operational infrastructure after previously serving in similar roles at other nonprofits and working in civil engineering and as an executive coach.

Lola Carbonell is co-founder of the General-Purpose AI Policy Lab, where she serves as Secretary General and Head of International Affairs.