Question-Answer Counterfactual Interval (QACI)
About
Updated 05/18/26QACI (Question-Answer Counterfactual Interval) is Orthogonal’s primary formal-goal alignment proposal. It models a human decision-maker answering questions across a range of counterfactual scenarios and defines a mathematical scoring rule over answers; an AI system is then designed to choose actions that score well under this formalized goal. The agenda is intended to produce a clean mathematical target that can be safely optimized even by superintelligent systems.
Theory of Change
Orthogonal’s theory of change for QACI is to first construct a precise formal goal that would lead to good worlds when pursued—currently instantiated by the QACI objective—then design AI systems that take such a formal goal as input and return actions that optimize it across the distribution of likely worlds. This “backchaining” approach starts from scenarios where AI risk is averted and works backward to determine what formal-goal research, including QACI, is needed to make such scenarios possible.
Discussion
No comments yet. Be the first to share your thoughts.
Details
- Start Date
- -
- End Date
- -
- Expected Duration
- -
- Funding Raised to Date
- -