No description available.
People
Updated 06/10/26By grantmaking.aicreator
Funding Details
- Start Date
- -
- End Date
- -
- Expected Duration
- -
- Funding Raised to Date
- -
- Annual Budget
- -
- Monthly Burn Rate
- -
- Current Runway
- -
- Funding Stage
- -
- Fiscal Sponsor
- -
Project Details
Updated 06/10/26By grantmaking.aiProblem:
There is a currently a lot of work that goes into adding new models/architectures to TransformerLens. It requires a lot of additional reimplementations and verifications.
Solution:
The project ("TransformerBridge") will allow loading any nn.Module, including the current transformers/HuggingFace models into TransformerLens in a simple way. People will be able to easily use any architectures with TransformerLens, regardless of whether these models exist on HuggingFace by configuring one file. This project is not only designed to support and enhance all current TransformerLens usages, but it also opens the door for interpretability research in closed environments where HuggingFace hosted models may not be the target.
Timeline:
The proof of concept of this is done already, but it will take more time to complete, polish, and test, so that it can be rolled out for real world interpretability research. The next two months will allow us to enter beta, and begin helping people transition to using the new module as opposed to the existing HookedRootModules. Once all of the reasonable use cases of TransformerLens have been tested, we will release this new module into TransformerLens 3.0.
Funding:
-
$10,000 USD for Bryce to work on this for the next 2 months
-
$3,000 USD for Fabian, Bryce's mentee, who has already been making great contributions to TransformerLens over the past year
-
Any additional funds will be used to continue support for TransformerLens. If the full funding goal of $30,000 is met, that will be enough for Bryce to manage TransformerLens through the rest of 2025 with no issue.
Grants Received
Updated 06/10/26By grantmaking.aiDiscussion
TransformerLens has been invaluable to the creation of ARENA, and to many people I know in the community (including myself). I think it has the potential to expand and support really great future work.
Making libraries like this one run smoothly is a neglected and difficult task, and Bryce has been doing an awesome job - I'm very excited to see it continue!
Many new projects still use https://github.com/TransformerLensOrg/TransformerLens as a core dependency. Over 500 public code repositories on Github rely upon the transformer\-lens package, including ones created by leading organizations such as Meta Research, Redwood Research, Model Evaluation & Threat Research, Apollo Research, and Decode Research.
Bryce has a proven track record of consistent contributions to the library and is the best possible owner for ensuring the stability and growth of TransformerLens moving forward (as well as compatibility across other packages for doing mechinterp). He brings a deep expert understanding of the existing feature set and has demonstrated rapid iteration speed and skill to adapt the framework for recently launched models.
He also actively volunteers to answer community questions on Slack and integrates user feedback into the development roadmap with detailed progress updates to relevant stakeholders. I would also be excited to see Fabian's contributions going into TransformerLens 3.0 and beyond. I'd endorse and highly recommend donating to this manifund project to help it reach its full funding goal through the rest of 2025.
@sheikheddy Thank you for this comment, I've just made a small donation in support as well.
[Progress update]
What progress have you made since your last update?
We have released TransformerLens 3 beta! Second beta release is coming up tomorrow. We decided to keep the project in beta until all features of TransformerLens 2 are ready. Right now 90% of use cases are covered, and we have hypothetical support for almost 5k models, compared to TransformerLens 2 200.
What are your next steps?
We are continuing iterating on TransformerLens 3 until we cover absolutely all use cases from the old version of TransformerLens, at which point we will have an official TransformerLens 3.0.
Is there anything others could help you with?
Full time development has been funded through the end of the year!
I suggested Bryce apply, and have funded this for two months. Open source research tooling is really valuable for accelerating the work of people outside big orgs, TransformerLens is pretty popular, and I've often heard complaints about the problem this is solving.
Conflict of interest: I created transformerlens (though haven't been involved for a while), and several of my projects would benefit from this tooling, though only as a side effect of this benefitting the interp community as a whole. I don't financially benefit in any way from this