14 Sep 2026
I was reflecting this weekend on the events surrounding the release of Anthropic’s Dario Amodei’s essay, calling for a coordinated slowdown in AI development, and on its subsequent endorsement by rivals Sam Altman and Elon Musk.
It made me think of an amazing book I read some years ago, Human Compatible by Stuart Russell, in which he explains how AI could lead to a golden age for humanity, but only if we ensure we never lose control of machines more intelligent than us. He illustrates his point with a simple example. If we make a superintelligent coffee machine, but don’t give it clear instructions as to what it can and can’t do when fulfilling its mission to make coffee, it may well decide that turning the sky pink is an acceptable side effect, or more chillingly, that any and all threats to its ability to make coffee must be neutralised.
It also made me recall a somewhat chilling moment from the FT’s last Future of AI conference, where Yoshua Bengio, one of the “godparents” of today’s AI, talked candidly about the concerning behaviours that researchers had witnessed when lab testing frontier models, and his fears for the future (he specifically mentioned his grandchildren) unless meaningful changes are made to risk management. You could feel the atmosphere in the room change instantly. Akin to when the saloon bar piano suddenly stops playing in a Spaghetti Western. Unfortunately, as in such movies, the metaphorical piano started back up quite quickly once he finished speaking, and everyone got back to talking about how amazing AI is.
Stuart Russell's book was first published in 2019. Around the same time as a terrifying video about a make-believe superintelligent AI called Earworm that was accidentally released into the wild by a well-meaning but woefully inexperienced and under-resourced (from a due diligence perspective) technology team trying to tackle copyright infringement. The aforementioned moment at the FT’s last conference was almost a year ago.
Now, in late 2026, with an increasingly polarised world in which profits and prestige are so often prioritised above all else, the timing could not be worse: the window for humanity to put in place the necessary safeguards to prevent a potentially existential catastrophe remains open, but clearly not for long.
Thankfully, there are encouraging examples of where we have been able to come together to avert disaster. The global ban on commercial whaling and nuclear testing are but two. In both instances, public outcry was a major factor in motivating governments to act, so the recent slew of viral posts by Anthropic researchers, decisive in thrusting the issue into the mainstream, may well prove to be the moment that compelled the global community to coordinate before it was too late.
Author: Benjamin Colchester
