Josh Marshall at Talking Points Memo commends a cautious approach to AI:
Make sure new things don’t endanger us before we build them or make them widely available.
This sounds sensible. It is impossible.
First, we have the work of Gödel, Turing, and Church from the 1930s that demonstrates very real limits to what we can know about software. It is sometimes true that we can examine a computational system, think carefully about how it operates, and prove that it does what we expect and does not do anything else. But for every system where this is possible, there exist countless systems where this cannot possibly be accomplished.
Second, ensuring that new things don’t endanger us requires that we be able to enumerate all the ways new things might endanger us. That, too, is impossible, even for simple and well-understood engineering systems. We may be confident that a proposed bridge will not fall down. But might the traffic it attracts choke our business district? Might it provide an invasion route for our enemies? Might lead in the paint that protects the bridge poison our children? Might certain critical members of the community use the bridge to build distant homes, ultimately severing their ties with our residents?
A bridge is just steel and stone on which we can stand; AI is more complex. Could an AI be our Iago? Might it be our Friar Lawrence?
The confluence of these two problems is singularly difficult for computing. We cannot know exactly what a system can do, and we cannot know what risks to avoid. The counsel of prudence in such situations is “don’t build any of these things.”
We cannot do that! We could shut down OpenAI and Anthropic and Meta, perhaps. But you don’t need to be a billionaire to train a model. You could do it in your basement. There are countless thousands of people who know how it’s done. Could we stop them all?
This is not some arcane mad scientist formula known only to a single genius. Now that we know this works, it appears that the details of the way these models work are not terribly important, anyway. The success of the current approach was a real surprise to most of us, but now that we know it can be done, we can do it other ways.
So you’re going to have to outlaw computer programming, in a country that cannot manage to regulate guns. Good luck with that!
But even if we could do that, China exists. They’ll continue to use whatever technologies make them safer and more powerful. Japan and Korea and Ukraine will want AI. So will Europe. Israel, of course. Not everyone will stop because we stand athwart the highway—especially since they all understand the limits that Gödel places on what we can know, even if we try to elect ourselves superior to reason.