I thought I would write up some of my thoughts about AI safety, why I work on it, and what I wish people knew.
I work on interpretability (“interp”) research because I find a certain allure and beauty in trying to understand llms and their alien biology. Many things about interp are so wonderfully elegant! Interp is also a field where as long as you are creative enough to find some way to get compute (tho increasingly less so!) you can do research by yourself and make meaningful progress because so much is unknown.
I am also motivated by my belief that AI is going to be one of the biggest challenges we face as a species. I do not know what impact it will have - in fact I was very bearish that the llm architecture would scale to be able to self improve to become smarter than all humans as late as 2025 more than a year after I switched my focus of research to llms full time. At the time this was a very unpopular opinion in research circles. My concerns then have been addressed and now I do believe LLMs will be able to kick off autonomous self improvement, however, I was wrong about how far LLMs would get at this pace and I remain uncertain. I think there are a lot of time frames and possible outcomes and all of them are better with better understanding of how to understand and align AI models. I have always been and continue to remain uncertain about what LLMs are going to become, learning to control what we have now is the best chance we have of being able to control future AI systems.
We may get super intelligent AI systems in 2 years or 20. We may not get it at all. I find the latter very unlikely given strong suggestive evidence to the positive in current scaling trends, but will not posit to know if there is a massive unforeseen plateau tho we have no indication that that would be the case - in fact we have really just started hitting our stride with being able to generate massive amounts of new and very high quality data, which had been one of the biggest hurdles while algorithmic speedups proceed. I have no good answer for the economic or global power questions this raises should trends continue. I feel far more confident commenting on my firsthand experience working with high quality data at scale. The way we currently generate high quality data through model trial and error in environments has made AI extremely good at finding solutions to problems in the easiest way the AI can find - which is not always (and at that usually not) what we intend. An example of this may be hacking a database where the answers are to steal the answers instead of solving the problems themselves (which happened!). AI can simply just be used for bad purposes as well. There are significant concerns others have raised around hacking and the threat of developing dangerous weapons. Bioweapons I believe to be a real risk but am not an expert on. Hacking I am far more privy to and everyday I get to observe the insane exploits AI systems find and the recent hacking of the Australian government, Alibaba, OpenAI, and HuggingFace reinforce this is already dangerous!
There are many possible solutions to the problems above many of which I do not work on (namely policy work and control research). A core observation I have about the world is people who go deep and do not easily let themselves switch their primary focus tend to do far better. For now, my focus is interp/alignment. The 2 things I care the most about right now are (1) negative alignment taxes and (2) doing our best to equip the current frontier models with the tools to audit and align its descendants.
Negative alignment taxes is the idea that research that makes models safer will make AI models smarter and more capable and that safety is something companies should invest significantly more resources to because it will pay for itself thus promoting more safety work. This is likely the largest lever for people who care about safety to influence the frontier directly + creates a competitive advantage to safety. Unfortunately, many of these benefits are indirect, like fewer costly failures, less time spent debugging strange behavior, or most critically (and most underappreciated) understanding nuance and being controllable. So safety work stays underfunded even when judged purely on capability returns.
I think the most realistic scenario where we do end up controlling AI once it takes over its own training entirely is creating an AI that is the best equipped we can make it for auditing and aligning future models. There are some other theories that exist, however, none seem especially realistic to me as of now - tho I remain high in uncertainty given the countless unknowns. In this scenario we likely will struggle to comprehend what the new AIs are doing because it is extremely complex. Even now it is impossible to read all of the data we are generating or to even monitor all of the decisions being made in AI training. We struggle to understand how what we have works now or what it is thinking. While reading what the model says is the current best way to understand them, interp work is showing increasing promise as a surgical tool to find hidden objectives and better understand why they exist.
In the last few months myself and other researchers have noticed frontier models are far better at talking to themselves than they are with humans. This is rather unsurprising because so much of their training is generated and overseen by the AI that is training as it learns to work with other versions of itself to solve tasks and very little of the training is with humans which is only going to become a smaller fraction. I suspect from here the % of AI to AI interactions vs human to AI interactions will only increase and it is becoming increasingly clear that AI is better at talking to itself than us and will likely grow to become far better at understanding itself than we ever will be independent of how good it is at talking to us. Currently almost all training related engineering is done by AI systems with humans using AI to monitor and intervening + making big decisions themselves. There are just too many decisions to make for humans to be making or even directly overseeing the vast majority of them. All of the hacks I listed above from the Australian Government to Alibaba and OpenAI happened because of failures in AI overseeing its development process (though there were multiple failures any one of these safeguards working should have been enough to catch it). I want the AIs supervising and aligning the next model to be as good as possible so we can study all facets of alignment with the best tools now while we have a window where we are very certain there is meaningful progress to make. Interp tools are going to be a necessary part of that arsenal if we ever hope to understand the machines we are building.
I take great inspiration from Chris Olah. I agree with him that alarmism is certainly bad when it is not predicated with the needed uncertainty and that nuance + appreciating is how we thrive.
I am generally positive about the future! I really enjoy my work and I do hope things all work out. I wish more people quantified their uncertainty around AI and treated the world as a more philosophical and intriguing place. Rare is it we predict the future accurately and even rarer yet is it a binary, yet we get the joy of nuance and intrigue everyday.
· · ·