Leading AI Progress With Safety

I recently saw a checklist from Daniel Kokotajlo. One of the questions was, “Have I ever said anything in public like ‘The companies should stop unilaterally’?” So this is my position.

I’m a staunch advocate and user of AI for uplifting human activity and scientific discovery. I’m also excited about the possibility of superintelligence, AI vastly smarter than humans in almost all areas. I think it could also cause massive disempowerment or kill everyone… which I hope doesn’t happen and is why I’m writing this article. There are other safety concerns leading up to that, at different capability levels and levels of implementation in society.

My position is that we should lead AI progress with safety. Safety and alignment should take priority over capabilities, and progress in those areas should lead and govern the pace of capabilities development. That could mean slowing from the current pace, which is already rapid, or even pausing if needed. Pacing now could still mean things keep going super fast, rather than super, super, super fast.

How Fast Are Capabilities Improving?

METR just released an intriguing, terrifying, and frustrating report. It’s intriguing because it estimates that Anthropic’s capabilities on well-specified tasks are improving at around twice their earlier rate, with AI assistance appearing to contribute substantially. It’s terrifying because of how quickly we’re getting to those capabilities, without safety seeming to move as rapidly. And it’s frustrating because Anthropic redacted much of the evidence METR found most informative. I think the rate of AI-assisted AI progress is one of the most important numbers for the public to watch.

But that number alone doesn’t tell us whether safety is keeping up. I think we should consider safety in a few ways. Are different aspects of safety adequate for a certain level of capability? As capabilities change, is safety improving quickly enough? Do safety methods scale along with different capability improvements, up through the alignment of a superintelligent system?

We have graphs looking at capabilities, but I want to see how the different safety domains are following those capabilities, and where they aren’t. Safety research and evaluations exist, but public reporting on responsible AI benchmarks is sparse compared with reporting on capabilities.

Four Goals of Psychology and AI Safety

I’d like to propose a parallel between four goals of psychology and AI safety.

In psychology, we ask, what is happening? We observe and describe behavior. In AI safety, behavioral evaluations, benchmarks, and monitoring help us observe what systems do. This is relatively well developed within tested conditions, although observations can miss rare behaviors or situations the tests don’t cover.

We also ask, why is it happening? We try to explain behavior. Mechanistic interpretability attempts to open the black box of machine learning by examining internal representations and computations. Researchers have identified and intervened on particular mechanisms, but their explanations remain partial.

Then, what will happen under certain conditions? This is prediction. We have some ability to predict model performance in defined settings. Predicting behavior in unfamiliar environments, under different incentives, or when a system recognizes it is being evaluated is more difficult. AISI’s alignment investigations distinguish testing what a model can do from determining whether it will misbehave when given the opportunity.

Finally, what can we reliably influence, change, or prevent? In psychology, we work to influence or change behavior. With AI, this also brings us to the science of control. Training, safeguards, and targeted interventions can change behavior. But changing behavior in particular conditions is different from reliably preventing consequential failures across new conditions. AI-control research shows progress within experimental settings, with broader assurance still unresolved.

My concern is whether the evidence across these four goals is sufficient as capabilities increase.

Pacing AI Progress

Coordinated pacing is obviously preferable, but unilateral pacing can still be meaningful. A single lab stopping could be a costly signal with implications beyond its own development. That’s a possibility worth considering, even while taking the risk of competitors continuing seriously.

AI can assist with research, experiments, and hypothesis generation, from helping people with tasks up to autonomous work. Recursive self-improvement represents a particularly concerning feedback loop. I support banning RSI until safety is adequate. But there doesn’t need to be a rapid takeoff for this to be dangerous. Even if capabilities kept improving at today’s rate, we could reach levels capable of significant or irreparable harm if safety continued much more slowly. We don’t necessarily know when those thresholds will be crossed.

I think AI is going to have to be part of safety and alignment research. It’s necessary, especially as we move toward superalignment, aligning systems vastly more capable than we are.

OpenAI’s iterative deployment approach offers opportunities to learn from use alongside predeployment testing. There are pros and cons, but detecting problems and correcting them after deployment won’t necessarily always be viable as capabilities increase.

AI Safety Within Psychology

We can apply these same questions, at a different scale, to AI within psychology. AI uses and products within psychology should follow the same principle of leading with safety. Safety evidence should lead deployment and use, with monitoring continuing after deployment. AI therapists are one example. We want to observe outcomes, explain what contributes to them, predict benefits and harms under different conditions, and reliably intervene when needed. We also need to understand the human–AI relationship and its effects on dependence and empowerment.

We can look at what current clinical safety evidence tells us. Research on AI in mental health is emerging and developing rapidly, from clinical trials of conversational interventions to evaluations of screening, treatment support, and other clinical uses. Some findings are promising, but the evidence varies across products and uses. Research on potential harms is also developing, and questions remain about safety in different populations, longer-term outcomes, and real-world use. The evidence needs to match the intended use and population.

Psychologists can contribute to that science through research and clinical expertise. With demonstrated safety and increased capability, AI could empower individuals and help address unmet needs. Some people will choose to reserve therapy relationships as completely human, and I think that choice and informed consent should be protected. I’m excited about the possibilities. I want our evidence of safety to help determine how quickly we pursue them.

Process note: AI was used as a research and editorial collaborator for research, organizing, drafting, editing, fact-checking, and publication preparation. The position and main ideas are my own.