A non-profit AI alignment research organization focused on agent foundations, pursuing formal goal alignment approaches that would scale to superintelligence.
A non-profit AI alignment research organization focused on agent foundations, pursuing formal goal alignment approaches that would scale to superintelligence.
People
Updated 05/18/26founder
researcher
Funding Details
Updated 05/18/26- Annual Budget
- -
- Current Runway
- -
- Funding Goal
- -
- Funding Raised to Date
- -
Org Details
Updated 05/18/26Orthogonal is a non-profit alignment research organization focused on agent foundations, announced in April 2023 by founder Tamsin Leake, an LTFF-funded AI alignment researcher who previously participated in the Refine research incubator in 2022. The organization is based in Europe and operates under the fiscal sponsorship of Ashgro Inc, a 501(c)(3) public charity. Orthogonal pursues formal goal alignment, a technical approach within agent foundations research. Their core methodology involves two components: building a formal goal that is expressed entirely in mathematical terms rather than natural language, and designing an AI system that takes this formal goal as input and returns actions pursuing it across the distribution of worlds we likely inhabit. Their guiding principle is that you do not align existing AI -- you build aligned AI from the ground up. The organization's primary research agenda is QACI (Question-Answer Counterfactual Interval), which uses counterfactual answers from a human decision-maker as a signal for aligned behavior. Rather than trying to formalize complex philosophical concepts like human values directly, QACI aims to formalize the relationship between an AI system and a human whose values it should serve. Research on formalizing QACI has been conducted by Tamsin Leake and Julia Persson. Orthogonal's theory of change employs a backchaining approach: starting with a plausible story of how existential risk from AI is averted, then working backward to identify the research needed. They describe their work as the kind of object-level research that cyborgism (human-AI collaboration approaches) would want to accelerate. The organization shares MIRI's model that hard takeoff is likely and that many current alignment approaches may be insufficient. Orthogonal exercises significant caution regarding AI capability infohazards, using encrypted locked posts for research that might pose exfohazard risks. They have published research on topics including epistemic states as benign priors, logical decision theory, and formalization of the QACI alignment plan on LessWrong and the AI Alignment Forum. In 2023, Orthogonal received $493,000 in funding through the Survival and Flourishing Fund's H2 grant round, funded by Jaan Tallinn via Lightspeed Grants. The organization has been evaluated by Zvi Mowshowitz in his 2025 nonprofits review as doing work that, if it succeeded, might actually amount to something, though described as a long shot worth trying.
Theory of Change
Updated 05/18/26Orthogonal believes that most current AI alignment approaches are insufficient to handle superintelligence, and that alignment must be built into AI systems from the ground up using rigorous mathematical formalization. Their theory of change uses backchaining: starting with a plausible scenario in which existential risk from AI is averted, then working backward to identify what research is needed. The causal chain runs from developing QACI (a fully formal mathematical goal specification that leads to good outcomes when pursued) to designing AI architectures that provably pursue such formal goals, to deploying systems that are aligned by construction rather than by post-hoc correction. They position their work as the object-level research that other strategies (like cyborgism or buying-time approaches) would want to accelerate.
Grants Received
Updated 05/18/26Projects
Updated 05/18/26Orthogonal’s QACI project aims to specify a fully formal mathematical goal for AI systems by using counterfactual answers from a human decision-maker, yielding an objective intended to remain robust when optimized by very powerful agents.
Discussion
Key risk: The program is highly speculative with minimal empirical validation and a small team, and may never connect to contemporary ML or produce testable milestones, making the counterfactual impact of additional funding dubious.
Case for funding: As one of the few groups seriously pursuing an aligned-by-construction path via a fully formal goal specification (QACI) within agent foundations, Orthogonal offers a neglected, high-upside bet aimed at hard-takeoff scenarios where only mathematically grounded guarantees are likely to matter.