AI and the risk of catastrophe

I am reminded of an episode from history involving Bertrand Russell and the U.S. writer and editor Leo Rosten. In the early 1950s, Rosten had approached Russell in London to write an article on agnosticism for Look magazine. The resulting piece was published in November 1953. A couple decades later, in February 1974, Rosten recalled his conversations with Russell in an essay titled ‘Bertrand Russell and God: A Memoir’ in The Saturday Review. They contained one amusing exchange: Rosten asked Russell to suppose that Russell had died only to find himself in heaven, standing before god. What would he say? “I probably would ask,” Russell said, “‘Sir, why did you not give me better evidence?’”

I do not think the statement by @gro_tsen is true per se. The only way it could make sense is with ‘better evidence’ in the form of two additional ‘hidden’ possibilities.

First: Those who say the technology has a 10% chance of “killing us all” (setting aside questions about how one is measuring that possibility) are playing into the hands of the Big AI cos. that hope the figure will bolster their claims to continue self-policing, since framing AI with the potential to have such unintended and unprecedented consequences can nudge regulation to focus more on technical safety rather than on questions of labour, copyright, competition, surveillance, etc. while allowing incumbents to remain involved in designing the rules to govern themselves — all while the exacting safety requirements disproportionately burden smaller competitors.

Or, second, that, aided by the sheer rubbishability of the figure, governments allow AI companies to continue working as they are and nothing changes — the sort of consequence already familiar to people following climate action and vaccination drives, where extreme but poorly substantiated claims, such as “the Arctic will be free of glaciers by 2050” or “mRNA vaccines for COVID-19 will prevent transmission”, allow doubters to altogether dismiss the underlying issue — the need to pay heed to the effects of climate change or vaccination — just by discrediting the flimsy claims. Put another way, a spectacular but nearly-impossible-to-falsify claim can render regulation a choice between accepting an extraordinary hypothetical danger and dismissing AI regulation altogether, and both of which sideline the more mundane reasons to regulate the industry today.

Finally, I disagree with @gro_tsen’s binary because, at the outset, you can imagine the space of possible positions vis-à-vis AI to already be constrained by at least three axes: the desirability of AI, the risk of catastrophe posed by AI, and the confidence in risk estimates, each of which ranges from very low to very high. Further, “pro-AI” or “anti-AI” is itself a volatile classification because it does not capture whether the support or opposition refers to enthusiasm for the technology, the companies developing it, slow v. rapid deployment or automation or to the technology’s net social value. It also overlooks the possibility where two people can agree on the technical risk but disagree on whether emphasising an extinction-level risk confers any political leverage.

Or, of course, the statement is just a joke.

Coda: After Look magazine published Russell’s article, the New York Daily News attacked him in an editorial and said the continued existence of someone with Russell’s views only demonstrated god’s mercy. When Rosten sent Russell the editorial, Russell replied:

Dear Mr. Rosten:

Thank you for sending me the extract from the N.Y. “Daily News.” I think the evidence for the existence of God supplied by my continued existence is strengthened by the continued existence of the N.Y. “Daily News.” It and I can agree in wishing that His mercy were less infinite.