The science fiction writer Arthur C. Clarke once said: "Any sufficiently advanced technology is indistinguishable from magic."
Typically, Clarke’s Law is used to describe an encounter with an unimaginable technology. To someone from the Middle Ages, my robot vacuum cleaner would seem eerie. We, on the other hand, know exactly what it can and cannot do. We do not expect it to make us coffee or explain how the world works; we can easily make sense of the device and how it works.
When a technology seems magical, it is precisely this classification that is missing. It becomes difficult to gain clarity about what it can and cannot do. False expectations lead to wrong decisions, both in business and in private life.
Why is it that Clarke’s Third Law applies to AI of all things?
On 30 November 2022, OpenAI released a tool called ChatGPT as a free research preview. Within five days, a million people had used it. It was the fastest growth in user numbers the digital world had seen up to that point.
To the general public, this tool appeared out of nowhere. There was no gradual transition, as there had been from the rotary dial telephone to the push-button telephone and then to the smartphone. There wasn’t even a preview in the traditional sense. Suddenly, anyone could talk to a machine that responded fluently and had an appropriate answer for almost anything. The astonishing thing is that even OpenAI was taken by surprise by the impact. Internally, ChatGPT was planned as a ‘research preview’, not a product launch. Virtually no one outside the research laboratories was aware of the years of development that had led up to this moment. This is precisely the gap defined by Clarke’s Law. To anyone who hasn’t witnessed the stages of development and suddenly finds themselves at the top of the ladder, it must seem like magic.
This impression is reinforced by what the machine does. A robot vacuum cleaner moves around and vacuums. Until now, speaking and understanding were the preserve of humans. With ChatGPT, we encountered something for the first time that struck us as human: the chatbot spoke fluently, engaged with us and gave meaningful replies. And because we couldn’t see behind the process, our brain filled in the gap with the only image it had ever known: a thinking counterpart.
What’s really behind it?
The idea underlying today’s language models is over a thousand years old. In the 9th century, the Arab scholar Al-Kindi realised that in every language, characters and character sequences occur with a characteristic frequency. Anyone who knows these frequencies can crack an encrypted message, because the most frequent character in the ciphertext is likely to correspond to the most frequent character in the language.
In 1843, Edgar Allan Poe based his short story "The Gold-Bug" on precisely this method. His hero deciphers a secret code by counting which character appears most frequently, and concludes that it stands for the most common letter in the language. Sixty years later, Arthur Conan Doyle had his Sherlock Holmes proceed in exactly the same way in “The Dancing Men”; he recognises the stick figures as a secret code and cracks it by analysing the frequency of the symbols.
There are two crucial steps between counting individual characters and predicting entire sentences. The first was taken by the Russian mathematician Andrei Markov in 1913. He no longer asked merely how common a letter was, but how likely it was to appear given the previous one. Using Pushkin’s verse novel "Eugene Onegin", he counted how often a consonant follows a vowel and vice versa. That is the real breakthrough. He calculated the probability of one letter following another.
The second step was taken by the engineer Claude Shannon in 1948. He applied Markov’s idea from letters to words and demonstrated something astonishing. If you select words solely on the basis of their overall frequency, without taking into account what comes before, the result is gibberish. If, on the other hand, you take into account the most recently seen words, the result becomes all the more meaningful the more context you consider. This essentially described the core principle of the modern language model.
What has changed since then is not the principle, but the scale. Today, a machine counts patterns in vast amounts of text, basically half the internet. And it takes into account the broader context rather than just a few words. The result is impressive, but the basic idea has remained the same, from Al-Kindi through Markov to Shannon. It is a matter of counting and predicting, on an unimaginable scale.
What was OpenAI actually trying to achieve?
It’s just like Columbus. He wanted to go to India and ended up in the Caribbean. OpenAI wanted to build a general artificial intelligence and ended up with ChatGPT.
OpenAI was founded in December 2015 as a non-profit research laboratory. Its official aim was explicitly non-commercial and went far beyond the development of a chatbot: to develop general artificial intelligence (AGI) that would benefit all of humanity. The OpenAI Charter defines AGI as “highly autonomous systems that outperform humans in most economically valuable tasks”.
In this early phase, language was just one of several parallel lines of research. The most high-profile project between 2017 and 2019 was an artificial intelligence designed to beat world champions at the computer game Dota 2. Alongside this, there was research into robotic hands, reinforcement learning and tools such as OpenAI Gym. GPT-1, published as a research paper in June 2018, was a single paper from the language team. Not a product, not a business model, not even the company’s main focus.
Four years of development lay between GPT-1 and ChatGPT. From a technical perspective, the models were trained on ever-increasing amounts of text. By 2020, GPT-3 had already been trained on hundreds of billions of words – practically everything that could be found on the internet. Nevertheless, GPT-3 was only accessible to developers via a programming interface. From a financial perspective, OpenAI was pushing closer to its limits with every new scale-up. The computing power required for ever-larger models was becoming unaffordable. In March 2019, OpenAI therefore established a subsidiary with capped profits, OpenAI LP. A few months later, in July 2019, Microsoft invested a billion dollars and became the exclusive cloud partner.
It was only from 2019 onwards that OpenAI was able to start thinking commercially at all. And even then, the plan was not for a consumer-facing application, but for a programming interface for developers. The decisive step towards ChatGPT was a research finding from January 2022 called InstructGPT. In this study, humans taught GPT-3, through targeted feedback, to follow instructions rather than simply completing text. The method is called Reinforcement Learning from Human Feedback, or RLHF for short. The remarkable result: an InstructGPT variant that was more than a hundred times smaller than the original GPT-3 was nevertheless preferred by humans. Size alone was therefore not enough. Alignment with human preferences was the deciding factor.
ChatGPT, which was used by a million people in November 2022, was essentially a combination of the RLHF method, GPT-3.5 and a chat interface. It is a language model that has learnt what helpful answers sound like because developers have taught it exactly that through reinforcement learning.
But does this mean ChatGPT is on the way to AGI?
OpenAI has declared AGI to be its goal. The machine feels like a thinking counterpart. The obvious conclusion is that this is the much-vaunted general AI. It is precisely this confusion that is the real problem.
This assumption is incorrect. What we have today, even with the most advanced models, is specialised AI for language. A system that mimics language astonishingly well, but does not understand it, has no intentions, makes no plans and is not self-aware. Outside the realm of language, it fails miserably at tasks that a child can solve. Ask a language model how many Rs there are in the word ‘strawberries’. It will give you a plausible-sounding number, often the wrong one. The reason lies in its design: the model sees the word as one or two characters, known as tokens, which it has never learnt letter by letter. You don’t have to take my word for it; try it out for yourself on OpenAI’s Tokenizer: https://platform.openai.com/tokenizer
General artificial intelligence would be something fundamentally different: a system that, like a human being, reacts flexibly to new problems, learns, plans and takes responsibility. According to experts, we are still a long way from achieving this, and it is by no means certain that we will ever get there.
Yann LeCun, Chief AI Scientist at Meta and one of the pioneers in the field, has been warning about this very confusion for years. He argues that, due to their architecture, large language models will not lead to AGI. His key point: a model that merely predicts the next word has no model of the world, no ability to plan, and no capacity to reflect causally on what has happened. These capabilities cannot be achieved through more data or more parameters; they are structurally absent.
Back to the vacuum cleaner
This breaks the spell, revealing the very clarity with which we view the robot vacuum cleaner. We know what it can do: vacuum the floor, navigate around obstacles, and map out the living room. We know what it cannot do: make coffee or chat about the weather. Nobody would expect it to take on the role of chief construction manager.
With AI, this clarity is often lacking because its capabilities so precisely match our idea of what it means to ‘understand’. A language model can generate, translate, summarise and answer questions with astonishing accuracy. It cannot make sense of the world, cannot verify whether something is true, and cannot vouch for its own statements. It estimates the most likely next word, and in doing so has not the faintest idea whether what it says is correct.
Once you’ve taken that on board, you’ll see today’s AI for what it is: a tool with clear strengths and limitations. That doesn’t make it any less useful. Nobody stops using a robot vacuum cleaner just because they understand how it works. They use it more effectively. That is precisely the aim. AI should be deployed where its capabilities are suited to the task. There are more such areas than sceptics suspect, and fewer than enthusiasts promise. The trick lies in finding the right ones.