Modulo Research

Modulo Research Ltd is a private limited company incorporated in Cambridge, UK in November 2022, directed by Dr. Gabriel Recchia, a cognitive scientist who previously conducted research at the University of Cambridge's Winton Centre for Risk and Evidence Communication. The organization's mission is to increase the probability that advanced artificial intelligence leads to net positive long-run outcomes for society. Modulo's work is organized around three pillars. First, Evaluations: the organization systematically assesses language model capabilities, limitations, and potential risks to help companies and policymakers make informed decisions about AI development, deployment, and regulation. Second, Research: the team tests methods for scalable oversight and alignment, including LLM sandwiching experiments that examine how AI models can be used to supervise other AI models. Third, Data: Modulo develops and freely releases specialized datasets intended to support alignment research across the field. A notable output is the FindTheFlaws benchmark (published March 2025, arXiv 2503.22989), a multi-task, multi-domain collection of five expert-annotated datasets spanning medicine, mathematics, physics, coding, and Lojban. Each dataset contains long-form solutions with annotations marking specific reasoning errors, designed to facilitate scalable oversight research including debate, critique, and prover-verifier approaches. Modulo also evaluated frontier models' critiquing capabilities using these datasets, finding a performance range that can be leveraged for scalable oversight experiments. Modulo Research operates with a lean structure: Recchia is the sole director and principal researcher, supplemented by independent contractors for dataset development, project management, and research assistance. The organization received a grant of $408,255 over two years from Open Philanthropy (announced November 2023) to support this work.