A project that tracks and evaluates frontier AI companies on their safety practices through a weighted scorecard, focusing on actions labs should take to avert extreme risks from advanced AI.
A project that tracks and evaluates frontier AI companies on their safety practices through a weighted scorecard, focusing on actions labs should take to avert extreme risks from advanced AI.
People
Updated 05/18/26Founder and sole operator
Funding Details
Updated 05/18/26- Annual Budget
- -
- Current Runway
- -
- Funding Goal
- -
- Funding Raised to Date
- $371,000
Org Details
Updated 05/18/26AI Lab Watch is a project created and run by Zach Stein-Perlman that tracks what frontier AI companies are doing in terms of safety and evaluates them against a comprehensive set of criteria. The project launched in beta in April 2024 and came out of beta in May 2024 after a scorecard redesign. The core of the project is a weighted scorecard that evaluates major AI companies across seven safety categories: risk assessment, scheming prevention, safety research boosting, misuse prevention, security preparedness, risk information sharing, and planning. Companies assessed include Anthropic, OpenAI, Google DeepMind, Meta, xAI, Microsoft, and DeepSeek. Each category is weighted by its current importance for safety and how much signal the criteria provide. In addition to the scorecard, the project publishes blog posts on its Substack analyzing what AI companies should do and what they are actually doing, and maintains resource pages documenting company commitments and integrity incidents. A related project, AI Safety Claims Analysis (aisafetyclaims.org), collects and assesses public information on AI companies' model evaluations for dangerous capabilities and their safety and security plans and practices. AI Lab Watch is unincorporated and unaffiliated with any organizations. It is funded by the Survival and Flourishing Fund (SFF) and by Stein-Perlman personally. The website was designed by Lightcone, with webdev work done by Michael Keenan. The project's data and methodology have been referenced by the Future of Life Institute's AI Safety Index. As of September 2025, Stein-Perlman indicated he is no longer maintaining the main website, though he noted he may return to it or hand it off to someone else in the future. Prior to AI Lab Watch, Stein-Perlman worked as a Research Analyst at AI Impacts, where he focused on AI strategy and governance research.
Theory of Change
Updated 05/18/26AI Lab Watch operates on the theory that public accountability and transparency can pressure frontier AI companies to adopt better safety practices. By collecting what AI labs should do to prevent extreme risks (such as AI takeover and human extinction), publicly scoring them on their actual practices, and documenting their commitments and integrity, the project creates reputational incentives for companies to improve. The scorecard highlights specific actions companies can take in areas like dangerous capability evaluations, scheming prevention, and safety research, making it easier for external stakeholders to assess and compare company behavior. By also analyzing companies' model eval reports and safety claims, the project helps ensure companies cannot make misleading claims about their safety practices without scrutiny.
Grants Received
Updated 05/18/26Projects– no linked projects
Updated 05/18/26Discussion
Key risk: The project is winding down with Zach not maintaining the site and not currently accepting funding, making impact fragile to single-operator continuity and raising counterfactual concerns that similar tracking could be reproduced by larger governance groups.
Case for funding: Fund AI Lab Watch because its rigorously weighted seven-category scorecard and companion claims analysis provide the most credible, independent synthesis of frontier labs’ safety practices, create public comparators that pressure labs to improve, and have already influenced external indices like FLI’s AI Safety Index.