UMass Amherst is a public research university whose AI safety-relevant work is centered in the SCALAR Lab, led by Associate Professor Scott Niekum, which focuses on safe and aligned machine learning and robotics.
UMass Amherst is a public research university whose AI safety-relevant work is centered in the SCALAR Lab, led by Associate Professor Scott Niekum, which focuses on safe and aligned machine learning and robotics.
People
Updated 05/18/26Assistant Professor of Computer Science
Associate Professor, College of Information and Computer Sciences
Professor, Manning College of Information and Computer Sciences
Funding Details
Updated 05/18/26- Annual Budget
- -
- Current Runway
- -
- Funding Goal
- -
- Funding Raised to Date
- -
Org Details
Updated 05/18/26The University of Massachusetts Amherst (UMass Amherst) is a major public land-grant research university located in Amherst, Massachusetts, founded in 1863. Within the AI safety community, UMass Amherst is notable primarily for the work of Associate Professor Scott Niekum and his SCALAR (Safe, Confident, and Aligned Learning + Robotics) Lab in the Manning College of Information and Computer Sciences. Scott Niekum joined UMass Amherst in 2022 after serving as an assistant and associate professor at the University of Texas at Austin (2015-2022). He earned his PhD from UMass Amherst in 2013 under Andrew Barto and completed a postdoctoral fellowship at Carnegie Mellon University's Robotics Institute. His research goal is to ensure that AI systems are well-aligned with human objectives and can be deployed safely in the real world. To this end, he develops efficient learning algorithms that enforce safety constraints, provide performance guarantees, and infer and align human and agent objectives. His work draws from imitation learning, reinforcement learning, AI safety, and human factors across settings from large language models to physical robots. Niekum is a recipient of the NSF CAREER Award and the AFOSR Young Investigator Award, and is listed as faculty on the Future of Life Institute's AI Safety Community. He was among 22 technology leaders who signed the 2023 statement that mitigating extinction risk from AI should be a global priority. His AI safety research has been funded by LTFF (Long-Term Future Fund) and Coefficient Giving, with grants covering projects on AI alignment based on downstream outcomes and human flourishing, LLM rare event estimation, and emergent misalignment. UMass CICS also hosts a separate AI Safety Initiative co-led by Professors Shlomo Zilberstein and Eugene Bagdasarian. This initiative conducts research across AI alignment, robustness, interpretability, and governance, and received a $500,000 grant from Schmidt Sciences in 2025 for research on multi-agent systems safety. UMass Amherst is a large institution with over 24,000 undergraduates and a systemwide endowment exceeding $1.3 billion.
Theory of Change
Updated 05/18/26Niekum and the SCALAR Lab believe that many AI alignment and safety issues are best addressed at the design stage rather than through post-hoc fixes. By developing technical methods for reward inference, safe reinforcement learning, probabilistic performance guarantees, and agent alignment verification, they aim to build AI systems that are structurally safer before deployment. Their work on emergent misalignment and rare event estimation in LLMs addresses the risk that AI systems may behave in unexpectedly harmful ways even when trained to appear aligned. The underlying theory is that rigorous technical foundations for alignment — grounded in human preferences and downstream outcomes rather than proxies — are necessary to prevent catastrophic outcomes as AI systems become more capable.
Grants Received
Updated 05/18/26Projects
Updated 05/18/26
The Safe, Confident, and Aligned Learning + Robotics Lab (SCALAR) in UMass Amherst’s Manning College of Information and Computer Sciences develops efficient learning algorithms that enforce safety constraints, provide performance guarantees, and align human and agent objectives so robots and other learning agents can be deployed in the real world with minimal expert intervention.
The UMass Amherst AI Safety Initiative conducts research, education, and collaboration to ensure that advanced AI systems benefit humanity, with focus areas in AI alignment, robustness, interpretability, and governance.
Discussion
No comments yet. Be the first to share your thoughts.