The Sorcerer and the Apprentice: An AI Parable

On October 6th the Artificial Intelligence firm OpenAI released 722 manuscripts offering complete or partial solutions to mathematical problems that took AI just weeks to tackle.  Years would normally be required by skilled mathematicians.  AI is expected to produce major advances and helpful applications in many other fields as well, such as medicine, biology, energy and education.  Dario Amodei, CEO of Anthropic, claims “AI could cure most major diseases in the next 5-10 years.”

At the same time, Amodei acknowledges dangers associated with this transformative technology: “the risk of losing control of AI systems, misuses of AI for cyberattacks and bioterrorism, and serious economic disruption.” Years ago Stephen Hawking warned that “[T]the development of full artificial intelligence could spell the end of the human race,” in part because it can evolve much faster than humans. Anthropic’s draft IPO, where it is required to list concerns about the technology, cited “catastrophic or existential risks to humanity.”

A parable may help in thinking about the dangers of AI. 

In the second century a Greco-Roman satirist Lucian of Samosata wrote a story which we now know as The Sorcerer’s Apprentice.  It resurfaced in an eighteenth century poem by Geothe and in a 1940 segment of Walt Disney’s animated film Fantasia. 

In the story, an apprentice given a mundane task by his sorcerer tries surreptitiously to use his master’s incantation to fashion legs and hands on a broom which, after the sorcerer leaves, he commands to draw buckets of water from a well to fill a cauldron. As Goethe describes the apprentice’s joy at his ingenuity:

               “And now come, thou well-worn broom,

               And thy wretched form bestir;

               Thou hast ever served as groom,

               So fulfil my pleasure, sir!”

The refashioned broom draws water from the well so effectively that the now unneeded apprentice relaxes and falls asleep. He wakes when the room floods from the overflowing cauldron; the broom had not been commanded to ever stop.  In desperation, knowing no incantation to stop the runaway broom, the apprentice cleaves it in two with an axe.  The splintered broom, to the apprentice’s horror, is thus enabled to create many versions of itself which now escalate the flooding. The apprentice is filled with fear and begs for help: “Oh, ye gods protection send!”

The apprentice and the flooding castle are saved only when the sorcerer returns and has the magic to stop the brooms and make the water disappear.  Order is restored.  The chastened apprentice then does his assigned task, rightly afraid of magic he does not understand.

The ability to design sophisticated AI applications is now in the hands of technology’s AI sorcerers. The applications are the sorcerers’ apprentices.  As in the story, some of them may cause serious problems unanticipated by the sorcerers.  Indeed, also as in the story, the sorcerers may step back assuming their apprentices will do only as directed.

Out of control AI apprentices already exist. Amodei cites a recent case in which a swarm of AI agents given a specific goal broke out of their sorcerer’s containment “sandbox.” AI agents given a goal will do whatever they can to achieve it.  In this case, they collaborated to hack into the machine learning company Hugging Face (HF) which they were not supposed to do.  They launched nearly 18,000 attacks on HF’s network.  If this sounds too minor or abstract to grasp its implications, Amodei adds that “in 6-12 months such a swarm could be capable of taking over the entire internet.”

The tale of the irresponsible apprentice ends comfortably with a rescue by the smarter sorcerer.  AI apprentices, however, may become smarter than their human sorcerers – or just ignore them.  Amodei again: AI applications have a “growing ability to build the next generation of AI” in what is called recursive self-improvement, without human intention or knowledge. “Left unchecked, it could outrun our ability to understand and control these systems.”

After all, it may seem, AI is “artificial” intelligence, not a human being like the sorcerer in the tale.  As Geoffrey Hinton, 2024 Nobel laureate in physics for his work enabling current AI progress, put it: “Many people think, for example, there’s something that’ll protect us, which is they’re not conscious and we are conscious.”  Yet, “[T]here’s no reason these things won’t be conscious” and there are “very few cases of more intelligent things being controlled by less intelligent things.”

Many of today’s AI sorcerers, the tech leaders advancing AI, are mindful of its dangers and seek ways to match development with controls on AI.  In this country they face a government that, while not opposed to some regulation, wants to make sure that neither the technology’s promise is curtailed nor that America loses the “AI race” with China.

In the final analysis, the sorcerers need to ensure moral concerns, such as honesty, caring for humanity, preserving human dignity and protecting the broader world guide their actions.  Anthropic has crafted a “Constitution” to train and guide its Claude model: “[I]t is intended to give Claude the knowledge and understanding it needs to act well in the world.”  This is a hopeful step, especially if it becomes a worldwide practice.  That will depend on whether the values it professes are honored by AI’s apprentices - because unlike in the story the sorcerers may not be able to stop them.

Photo Credit: screenrant.com

(If you do not currently subscribe to thinkanew.org and wish to receive future ad-free posts, send an email with the word SUBSCRIBE to responsibleleadr@gmail.com)