I have confidence in Sean and Jack's previous work and think it's worth boosting efforts to keep things like this operating in Australia. Bridging the ACNC registration gap seems worthwhile.

Here's my updated sense of where Sean and co. are at, extremely heavily influenced by colleagues, especially Nate Rush:

1. My original comment on Sean continues to stand up well today. He appears to be among the very most competent people building on top of METR work.
2. The work he links above was competently executed. Their baselines/time estimates are reasonable (less common than you might think), they have a high attention-to-detail vibe, the write-up is fairly impressive and not overclaiming, etc. The work compares favorably to UK AISI's analogous work.
3. Unfortunately, best guess is that all of their tasks would saturate for most recent models.

Coming out of this:

1. I think it's a shame that Sean needed to dip into personal funds to pay for this project.
2. I consider it ~totally obvious that Lyptus' work is worthy of further funding.
3. I really, really, really, really, really want Lyptus to (1) throw more AI labor at their problems and (2) create dramatically harder cyber tasks. I think the end work will be much more useful to the degree that these things are true, less helpful to the degree they're not.
4. My best guess is that unfortunately Sean is continuing his streak of under-asking for funding, wanting to bring about higher-depth proof of concepts, clarity of workstreams etc. than I feel I need to be confident for $Xk bar. From my perspective, we're in a state of extreme triage; I do not want Lyptus to be wasting time fundraising. (And to some extent wasting my time doing larger number of funding rounds!) I would like Sean to expand the maximum funding here by a minimum of $20k (so that he can absorb $20k backpay + $100k prospectively), and very plausibly by a lot more than that. I am not at all saying that I would be comfortable at $500k right now, but I think that's where we should be starting the conversation. (As in, that does not obviously seem like the maximum to me, although it doesn't feel like minimum either.)

I'm delighted to recommend funding Lyptus' maximum here; I hope they can shift work to more ambitious ends that I think would be more helpful, and I hope they ask for more funding now.

(Noting that after conversation with Nate it's now less obvious to me that Lyptus should focus on dramatically harder cyber tasks in particular. Although I continue to think that there is a very, very, very high premium on dramatically harder tasks in general. Want to leave the focus more flexible depending on Lyptus' taste.)

Approving this project! I'm glad that @joel_bkr and others at METR are so excited for Sean's work, and from my perspective, this kind of active, expert-led, fast grantmaking is where the regranting mechanism shines.

I also appreciate Joel taking the time to lay out his considerations in making this grant, which are hopefully helpful for Sean and others in similar positions. (And @seanpetersau, insofar as you would be open to accepting more funding through this Manifund project, let me know; we're very happy to increase the funding goal on your behalf!)

To start! Thanks Joel! I very sincerely appreciate the candid and clearly worded feedback. I spent a bunch of time reflecting on this today.

With this in mind, I've drafted two budget tiers today with the information I have. This would be spent over two workstreams.

Our initial bridging fund was allocated to 2 months, with lower API credit and contractor asks, while we scoped future work. These tiers would have us operational for 6 months, and greatly increase the API credit and contractor budgets.

Tier 1: $300K

Workstream 1 — Cyber & Control:

- Salaries: $100K

- Model API credits: $50K

- Red team contractors / human experts: $50K

Workstream 2 — Pragmatic Interpretability:

- Salary + compute: $60K

Backpay + Overheads + Buffer: $40K

Tier 2: $600K

Workstream 1 — Cyber & Control:

- Salaries: $160K (supports an additional senior cyber researcher hire)

- Model API credits: $150K

- Red team contractors / human experts: $150K

Workstream 2 — Pragmatic Interpretability:

- Salary + compute: $60K

Backpay + Overheads + Buffer: $40K

What we're thinking

Workstream 2 is nascent and exploratory. I am particularly excited about upcoming work exploring self-steering via activation probes with potential applications in model welfare, model preferences and speculatively alignment stability. This workstream is a first step to support a core mission of Lyptus. Creating and growing institutional capacity, support and direction for untapped talent in Australia.

Workstream 1 is our larger investment. Through the course of our own work (and further so with the recent Mythos model card), it's plainly clear to us that cyber evaluations are materially not keeping up with capability growth. Honestly, the accelerating progress did take me by surprise and our dataset would certainly be saturated at higher token budgets. That said, while these steepening trendlines show no signs of stopping, we do believe current absolute capabilities can be overstated. Full red teaming engagements against even moderate blue teams are a very different proposition to isolated benchmark tasks.

To ground this with an illustrative example of where our thinking is. We're currently scoping the tractability of scaling up the ideas presented in the Artemis paper. Evaluating AI agents via real red teaming engagements and comparing their performance to professional human red teamers. This captures the full end-to-end scope of an attacker against real production organisations.

This is clearly operationally difficult. Though early conversations with senior offensive security leadership suggest it is tractable. We believe with the right people involved we can design incentive structures that work for all parties. A typical engagement with two professionals runs around $30K USD per week. We would ramp up starting with smaller targets.

This is speculative. We are having early conversations, and we are genuinely in a phase of figuring things out. It is our current leading candidate project but not our only option. What it represents is our broader interest in human grounded studies and science communication. Results that are quickly interpretable beyond the AI security ecosystem. We think this suits our strengths better than building cyber benchmark tasks, which requires deeper offensive security expertise than we have in house.

What the tiers mean

At tier 1 we would more likely prioritise budget friendly work like human-grounded covert capability on cyber tasks orhuman-grounded studies on cyber stealth benchmarks. If we had strong multi-organisation traction on the red teaming idea we would still pursue it, but budget constraints have a way of subtly pressuring priorities against more ambitious work.

At tier 2 the more ambitious work becomes realistic. A senior cyber researcher hire, 2-3 red teaming engagements at small to medium scale with client co-financing, and enough API credits to properly test models at adequate token budgets.

Other ideas in this space would require internal budget rescoping but the broad structure holds.

I have confidence in Sean based on his previous work, and I think locations with great talent (like Australia, India, Germany, etc) must be funded appropriately in order to create institutional capacity to nurture that talent. I really really hope that this project gets funded. All the best, Sean and #TeamLyptus :)

Recommended an additional $50k to Lyptus on same thesis as the below. I'm still curious about Lyptus absorbing more than this, but I only had another $50k in my Manifund pot! I've reached out to some funders about considering funding Lyptus further. I hope my comment below can serve as some evidence. For now, without more regrantor recommendation budget, I'm bowing out. Wishing Sean et al. the best of luck!!