Project summary

Our Team: The main focus of the Rethink Priorities Existential Security Team (XST, formerly the General Longtermism team) is helping launch entrepreneurial projects that aim to reduce existential risk from AI.

• We think of our work in this area as being in three distinct buckets: project research, founder search, and founder support.
• Our current major goal is to help launch one new highly promising project by the end of November 2023.
• See here for more information on our current work, here for a summary of our work in 2022, and here for our strategy for 2023.

Planned Use of Funds: We’re seeking funding to hire a Research Fellow. This is a temporary junior researcher position lasting 3-5 months. The person filling this role would contribute to our work helping launch projects that aim to tackle existential risk from AI, by conducting research into new project ideas and supporting our search for founders for existing ideas.

To get a sense for what the Research Fellow might work on, this recent EA Forum post describes 20 project ideas we found particularly promising (the post lists projects across existential risk, but we'd plan for the Research Fellow to work on AI-existential-risk-related projects specifically). From that list, the projects we're currently most interested in investigating further include "field building for AI policy", "developing better technology for monitoring and evaluation of AI systems", and "crisis planning and response unit" (though we'd expect this list to be different by the time the Research Fellow hire starts as our investigations continue).

Feel free to email ben [at] rethinkpriorities dot org with any requests for further information.

What are this project's goals and how will you achieve them?

We have are two key theories of change for the Rethink Priorities Research Fellow positions:

1. Direct impact through the Research Fellow’s work. In XST we believe strongly in our mission to boost existential security by helping launch projects that aim to tackle existential risk. A Research Fellow hire would help us with this mission by accelerating our research into new project ideas and our search for founders.

We see our work helping to launch projects tackling existential risk as providing value by:

• Directly reducing existential risk by helping launch projects aiming to tackle existential risk that otherwise wouldn’t have existed, and by counterfactually accelerating and aiding the ones that would have otherwise existed.
• Gaining information value by testing the effectiveness of doing incubation in the existential risk space, since the space seems relatively unexplored and is very promising in our opinion.

2. Exploration and career capital for the Research Fellow. Perhaps as importantly as the direct impact from their work, the Research Fellow position allows junior researchers to (further) test their fit for research on existential risk related projects and provides them with valuable learning, networking, and other career capital.

How will this funding be used?

The funding would go towards the Research Fellow’s salary, operational costs, and a budget for expenses such as travel (detailed breakdown available on request).

Who is on your team and what's your track record on similar projects?

XST consists of 5 staff, with the team lead reporting into Peter Wildeford. The team includes experienced generalist researchers within longtermism and existential risk, as well as EA-related entrepreneurial experience, such as Research Manager Renan Araujo founding Condor Camp, a longtermist talent search organization focused on Brazil. See the bottom of this page for our bios.

See this EA Forum post for a summary of our work in 2022, and see this EA Forum post for a list of 20 promising project ideas tackling existential risk we identified earlier this year.

Career trajectories from past XST Research Fellows include:

• Max Rauker: contracted by the Rethink Priorities AI Governance & Strategy team (AIGS) to conduct the AI safety intermediate goals survey, now working on improving research sharing in AI governance.
• Jam Kraprayoon: hired as an Research Analyst at XST.
• Joe O’Brien: hired as a Research Assistant at AIGS.
• Past RP Longtermism Fellows from 2021 (XST didn’t exist as a separate team at the time) include Tom Barnes who now works at Founders Pledge, Marie Buhl who now works as a full-time Research Analyst at XST, Lizka Vaintrob who now runs the EA Forum for CEA, and Juan Gil who now works with SERI MATS as a Scholar Support Specialist.

What are the most likely causes and outcomes if this project fails? (premortem)

• We fail to find good founders or otherwise fail to help launch the promising projects we’ve identified. We are relatively new to the process of identifying, vetting, matching, and supporting potential founders. We have sought significant outside advice and have an initial track record in this area, but it’s possible that executing this work successfully will be a challenge.
  • Outcome: we don’t help launch any impactful projects.
• We fail to find further highly promising project ideas. We think there are highly promising ideas for projects aiming to reduce existential risk, and so far we have found at least two ideas we’re excited to help launch. But it’s possible that the new ideas we investigate turn out to be unpromising.
  • Outcome: we (perhaps) launch one or two impactful projects, but fail to launch any more.
• We make a Research Fellow hire who is not a good fit for the role. We have a track record of making good hires for our Research Fellow positions, but it’s possible that our vetting process leads to a hire who isn’t a good fit for the role. Part of the theory of change for the Research Fellow position is to test researcher fit, so it’s unclear whether this should be counted as a failure, but it wouldn’t be the ideal result.
  • Outcome: the Research Fellow does less impactful work during their contract with Rethink Priorities than they would otherwise and moves into an area other than existential risk research after their contract expires.

What other funding are you or your project getting?

The team’s existing funding is drawn from multiple small and large donors, and we are currently fundraising for the team more broadly and have submitted proposals to multiple donors for this. Please get in touch via ben [at] rethinkpriorites dot org if you’d like to discuss this further.