Aligned AI

Aligned AI is an Oxford-area AI safety company incorporated in the United Kingdom in December 2021 (registered as ALIGNED AI LIMITED, company number 13798390, at 30 Upper High Street, Thame, Oxfordshire). The company was co-founded by Rebecca Gorman, a serial entrepreneur and AI researcher who has advised the EU, UN, OECD, and UK Parliament on responsible AI regulation, and Dr. Stuart Armstrong, who spent a decade as the Alexander Tamas Fellow in Artificial Intelligence and Machine Learning at the Future of Humanity Institute at the University of Oxford before co-founding the company in 2022. The company's central technical focus is concept extrapolation: solving the problem of how AI systems can understand and apply human-like concepts even when encountering scenarios outside their training distribution. This addresses what the team calls misgeneralization, where AI systems learn spurious correlations from training data and fail when presented with new situations. Their flagship algorithm, ACE (Algorithm for Concept Extrapolation), was demonstrated to surpass the CoinRun benchmark in September 2023, achieving a 72% success rate compared to the previous best of 59%. This work was widely covered as a significant milestone in applied alignment research. Additional products developed by the team include EquitAI (a gender bias mitigation tool for large language models), ClassifAI, and faAIr (an algorithm for measuring and ranking gender bias in foundation models). The faAIr and EquitAI tools won the CogX prize for Best Innovation in Mitigating Algorithm Bias in September 2023. In 2022, Aligned AI also released what was described as the first open-source ChatGPT jailbreak prevention prompt evaluator. Aligned AI is structured as a for-profit benefit corporation rather than a nonprofit, operating on a model that combines foundational research with commercial product development. The company has been funded through bootstrapping, angel investments, and UK government R&D tax credits. In September 2023, the company closed a $730,000 angel round at a $24 million valuation. The team is small (2-10 employees per LinkedIn), with known members including Gorman, Armstrong, Oliver Daniels-Koch, and Sophia Metz.