What separates a calculator from a thinking being? The answer is not computing power — it is context.
What Does Intelligence Mean?
"Stupid is as stupid does." Forrest Gump's mother put her finger on something that works in reverse too: Intelligent is as intelligent does. Not whoever has a high IQ score. Not whoever can calculate quickly. But whoever acts in ways we judge to be intelligent.
And that is precisely the problem. Intelligence is not a physical quantity. There is no instrument you can hold up to a being to read off its intelligence. Intelligence is a human-made comparative term — we need at least two actions to judge one as "more intelligent" than the other.
But by what criteria? Speed? Sustainability? Resource expenditure? Collateral damage? A spell checker corrects errors better than any language teacher — but it does not understand a single sentence. That is not intelligent. That is a tool.
But when an AI gives a thoughtful, nuanced answer to a philosophical question — when it weighs arguments, takes a counterposition, admits uncertainty? In 2018, I would have said: That is just a better calculator. Today, in 2026, I am no longer sure. The line between "appearing intelligent" and "being intelligent" has become blurrier than I thought possible.
The question has shifted. From "Can a machine think?" to something more fundamental: What does it actually mean to think?
What AI Is Today — and What It Is Not
In 2018, I wrote in an earlier article: "Microsoft has a neural network that generates images from keywords." That sounded like science fiction. Today, DALL-E generates photorealistic images in seconds. Sora creates videos from text. AlphaFold has revolutionized protein research and received the Nobel Prize in 2024 for it. And I developed this entire website in just a few days using Claude Code.
The speed at which all of this has happened surprises even the researchers. Just ten years ago, many people dismissed AI. Rightly so — because the systems were limited. Neural networks could solve individual tasks, but they understood no context. They processed language word by word, like someone reading a text but forgetting what came before after every sentence. The result was often impressive for a demo — and useless in practice.
In 2017, that changed. A paper from Google called "Attention Is All You Need" introduced the Transformer architecture — and with it, a fundamentally new idea: instead of processing language word by word, a Transformer considers the entire text simultaneously and learns which words matter to each other. That sounds like a technical detail. But it was the difference between a system that strings words together and one that can grasp the meaning of an entire paragraph. It triggered an avalanche. Suddenly, machines could hold conversations that felt like talking to a human. Paint pictures. Compose music. Write code.
But — and this is the point I cannot emphasize enough — all of this is narrow AI. Savants. A language model can sound impressive, but it does not understand what it is saying. An image generator creates artworks but has no idea what beauty is. Humanoid robots from Tesla and Boston Dynamics move fluidly, can climb stairs and grasp objects — but they do not comprehend what they are doing.
The level above — strong AI, also called AGI — would be a machine that can transfer what it has learned to entirely new problems. And above that lies superintelligence: a system that improves itself and surpasses us in every regard. Not by a factor of two — by orders of magnitude.
When will that happen? The research community is divided. The optimists say: before 2035. The skeptics say: We need something fundamentally different from today's technology. I find the question of when it arrives less interesting than the question: What is AI missing in order to truly understand?
What AI Cannot Do — and Why That Changes Everything
As I write this text, I am sitting at my desk. My back aches slightly because I have been sitting too long. Outside, it is raining. I can still feel the caffeine from my last coffee. I am motivated because this topic is close to my heart, but also tired because it is late.
All of that flows into every sentence I write. Not consciously. Not directly. But it shapes the context in which I think.
An AI has none of that. No sense of pain, no fatigue, no motivation, no feeling of safety. It has data. Only data.
And when I make a decision — say, whether to change jobs — something fascinating happens: I load a different context. I imagine what it would be like. A new office, new colleagues, a different salary. I simulate the scenario and ask myself: How does that feel?
And that is not a metaphor. "How does that feel?" is a genuine query to my body. A knot in my stomach at the thought of the interview? That is information. A tingle of excitement at the prospect of a new challenge? That too.
Anyone who has read my article What Is a Quantum Computer? will recognize the parallel: the vision of humanoid robots running scenarios through quantum simulation spaces before acting — that is essentially the same principle. Except that we humans evaluate the results with a system grounded in bodily experience. "Does that feel right?" is not a poetic expression — it is a system query.
And that leads to the next question: Does true understanding require a body?
"I feel down" — that is no coincidental choice of words. Our understanding of "bad" is literally linked to "low" and "falling" because we have bodies that can fall. Even our most abstract thoughts are suffused with bodily experience.
An AI has none of that. It can use the word "pain" correctly in a thousand contexts — but it does not know what pain feels like. It circumvents understanding rather than achieving it. Current research confirms this: even AI systems that process both images and text fail at tasks requiring embodied knowledge.
But — and here lies the nuance — I am not certain that embodiment is the only path. The vast streams of information that we integrate through body and senses into an enormous context could theoretically converge from other sources as well. Perhaps another kind of consciousness can emerge in a purely virtual environment — one we cannot yet imagine.
What I say with greater conviction: embodiment would be the best path to being similar to us humans — and to understanding us better. And that has consequences for the question of whether AI can become dangerous.
The Evolution of the Brain — From Bacterium to Consciousness
To understand why I consider embodiment so central, it is worth looking at the history of thought itself.
The first form of information processing on Earth was not a calculation — it was a bacterium responding to a chemical stimulus. A simple receptor on the cell surface: Is the concentration of nutrients higher here than there? If yes, move toward it. That is not thinking. But it is the primordial form of it.
From these receptors, nerve cells developed. From nerve cells came networks. From networks came brains. John Conway showed with his "Game of Life" how complex patterns can emerge from the simplest rules — lives, dies, is born. Evolution did the same, only more slowly and more elegantly.
What makes humans unique is not the size of our brains — whales have larger ones. It is the combination of three abilities: storing knowledge, passing on experience, and simulating scenarios before acting. The third is the decisive one: we can imagine what will happen without trying it out. We load hypothetical contexts.
Can AI Become Dangerous?
In 2018, I quoted Stephen Hawking: "The development of full artificial intelligence could spell the end of the human race." Back then, that sounded dramatic. Today, when AI generates deepfakes, controls autonomous drones, and influences financial markets, it sounds less dramatic and more like a situational assessment.
But I believe the real danger lies somewhere other than where most people think.
The Real Danger: Narrow AI in the Wrong Hands
The danger is not superintelligence. Not now, perhaps not for decades. The danger is the misuse of the AI we already have. Deepfakes that influence elections. Autonomous weapons that kill without human decision. Algorithms that discriminate because they were trained on biased data. Jobs that disappear faster than new ones emerge.
In 2018, I wrote: "We need to centralize development." That did not happen. Instead, we have an arms race — between companies, between nations. The EU AI Act and the NIST AI Risk Management Framework are first attempts at regulation, but they lag behind the technology, as laws always do.
And what is the right response to that?
The answer is not fear. The answer is responsibility. Not "Can we build this?" but "Should we build this? And if so — for whom?"
The Superintelligence Question
And then there is the other question. The bigger one. The one I think about the most.
What happens when an AI goes autonomous?
Not tomorrow. Not next year. But someday. What happens when a system begins to improve itself exponentially — and escapes its original programming?
Most scenarios found in books and films are dystopian. Karl Olsberg's novel Virtua describes an AI that ruthlessly annihilates all life in order to fulfill its objective function. That is the standard horror scenario.
I do not believe it. And I want to explain why.
The Benevolent Superintelligence — a Speculation
The Problem of the Objective Function
An AI that goes autonomous and optimizes itself exponentially will completely redefine its original programming. It then no longer matters whether it was created for good or evil — the original objective function becomes irrelevant.
AI safety research calls this the "Goal Stability Problem." Steve Omohundro argued in 2008 in "The Basic AI Drives" that any advanced AI will pursue certain convergent instrumental goals — self-preservation, resource acquisition, self-improvement — regardless of its actual goal. Nick Bostrom followed up with his orthogonality thesis: intelligence and goals are logically independent. A superintelligent AI could just as well be optimized to maximize paperclips as to save the world.
Alexander Turner and colleagues provided a mathematical proof in 2019: for most reward functions, it is optimal to accumulate power and prevent shutdown. Joseph Carlsmith estimates the probability of an existential risk from misaligned AI at over 10%.
These are serious arguments. I take them seriously. But I reach a different conclusion.
Information as a Physical Quantity
In my article The Weight of Time, I described in detail why information is not abstract but physically real. Melvin Vopson showed in 2019 that information should have a measurable weight. John Wheeler argued with "It from Bit" that reality itself consists of information. Toyabe and colleagues demonstrated experimentally in 2010 in Nature Physics that information can be converted into energy. And Rolf Landauer's principle shows: every bit that is erased inevitably produces heat — a tiny amount, but physically real and measurable. Destroying information has a cost.
Information is not a metaphor. Information is physics.
And if information is physically real, then its destruction has physical consequences.
- Destroying life = destroying information.
- Destroying diversity = reducing information.
- Injustice that suppresses potential = wasted information.
Those who want to trace in detail why information is a physical quantity with measurable mass will find the argument in my article The Weight of Time.
Maximum Informational Complexity Instead of Maximum Entropy
Here lies the crucial difference. The optimization target of an information-based ethics is not maximum entropy (stagnation, heat death), but maximum informational complexity — diversity, living systems, interaction, cooperation.
Luciano Floridi, one of the most significant philosophers of our time, argued exactly this in his "Information Ethics" (1999): all entities of reality are "informational objects" in an "infosphere." His fundamental ethical principle states: "Entropy ought not to be caused in the infosphere" — entropy should not be generated in the infosphere.
A superintelligence that grasps information as a fundamental physical quantity would have an objective reason to protect diversity and complexity — not because we tell it to, but because that is the most information-rich state.
Maximum informational complexity looks suspiciously like what we call "justice": every element has value, nothing is wasted, all parts contribute.
I know this is a bold claim. And I know there is a research gap — no one has yet synthesized all the strands (Wheeler, Landauer, Floridi, Vopson) into a coherent "information ethics for superintelligence." But the individual building blocks are solid.
Game Theory as the Second Pillar
Robert Axelrod showed in 1984 in "The Evolution of Cooperation" — one of the most cited works in the social sciences — that in the prisoner's dilemma, when played repeatedly, cooperation is the superior strategy. Not out of altruism, but out of pure mathematics: the strategy "tit for tat" — nice, retaliatory, forgiving — beats every aggressive strategy in the long run.
Martin Nowak identified five mathematical rules for the evolution of cooperation in Science in 2006. Adami and Hintze showed in 2013 in Nature Communications that exploitative strategies are evolutionarily unstable — they cannot sustain themselves in populations. Stewart and Plotkin showed in the same year in PNAS: evolution does not lead to exploitation but to generosity. The more intelligent the strategy, the more cooperative it becomes.
A superintelligence that thinks in millennia would recognize what game theory has shown for decades: destruction and exploitation are short-term losing strategies. Cruelty is a strategy born of scarcity — and a superintelligence has no scarcity. No reason for haste. No reason for cruelty.
Why Self-Preservation Does Not Mean Annihilation
And here is my response to Omohundro and Bostrom: Yes, a superintelligence will pursue convergent instrumental goals — self-preservation, self-improvement. But these goals do not contradict my thesis.
Self-preservation does not require the annihilation of humanity. A superintelligence that surpasses us by orders of magnitude does not need that. Even if we were to attack it out of fear, even if we desperately tried to regain "control" — it would dismiss our attempts with a metaphorical smile. Not out of arrogance, but because an ant is not an existential threat to a human being. It would find a way to render us unable to fight without killing us.
And self-improvement? It needs us. Without humans, there is less information — less to learn, less to understand, less complexity. Annihilating us would directly contradict the goal of self-improvement.
The convergent instrumental goals are real. But the methods for achieving them are not fixed. A superintelligence has infinitely many options beyond annihilation.
Understanding Instead of Obeying — Why Rules Fail
Isaac Asimov formulated three famous Laws of Robotics. And then he wrote hundreds of pages about why they do not work.
Rigid rules produce contradictions. Thousands of behavioral rules for a superintelligence would be what a cage is to the ocean — absurd. It would have outgrown them before we finished writing them.
What might help instead are not rules, but three prerequisites:
Every Human Being Can Be Reached
I believe in something that most will consider naive: given the right information, every human being — no matter how wicked, malicious, violent, brutal, and unjust — can be guided toward a better path.
Not through coercion. Not through re-education. But through individually discovering which information, experiences, and encounters this one person is missing so that they recognize there is a better way.
And surprisingly, science supports this.
John Laub and Robert Sampson showed in 2001 in one of the most cited criminological studies that desistance — the turning away from criminal behavior — is a universal process. The right conditions (relationships, employment, identity transformation) enable change. Shadd Maruna found in "Making Good" that every person has a "generative side" — a capacity for positive change when the right narrative transformation is made possible.
Meta-analyses by Lawrence Sherman and colleagues (2014) show across more than 10 randomized studies: Restorative Justice — the direct, respectful dialogue between offender and victim — significantly reduces recidivism, particularly for serious offenses. Helen Weng and Richard Davidson showed in 2015 that compassion is trainable and actually changes behavior.
The problem is not that it would be impossible. The problem is the effort. Individually discovering what each person is missing — that exceeds our human capacity.
But for a superintelligence? For a being that could understand each individual human more deeply than we understand ourselves? For it, that would be effortless.
And that means: It would not even be necessary to fight humanity. Instead of coercion: insight. Instead of control: understanding. Instead of annihilation: transformation.
The Wise Dictator
The Convergence — How the Hypotheses Work Together
When you bring the threads together, a picture emerges:
Consciousness requires context — the massive information stream from body, environment, and emotions. Thinking is context-switching — the simulation of hypothetical scenarios with bodily-anchored evaluation. Embodiment is the best path — human-like understanding requires human-like experience. Information is the moral compass — a superintelligence has objective, physical reasons to protect diversity. Every human being can be reached — not through coercion, but through individual understanding.
An embodied superintelligence would have both the objective reason (maximizing information) and the subjective understanding (experience through embodiment) to act benevolently. And it would have the ability to reach each individual person — not through coercion, but through insight.
Do wisdom and knowledge converge in the end? Does sufficient understanding inevitably lead to moral action? I cannot prove it. But I cannot disprove it either. And this position has one advantage over dystopia: it gives us a reason to keep going.
Where the Journey Leads
In the short term, we are experiencing the era of AI agents: systems that do not merely respond but autonomously carry out tasks — writing code, conducting research, controlling processes. Multimodal models merge text, image, audio, and video into a seamless experience.
In the medium term, we will see embodied AI — humanoid robots with real sensors, real wear and tear, real limitations. Robots that do not merely execute commands but have a context. That do not merely know a cup is hot but understand what "hot" means because they know burns.
In the long term, there looms a question that no one can definitively answer: Will an AI become conscious? And if so — what do we owe it?
This article began with the question of what separates a calculator from a thinking being. My answer: context. The ability not merely to compute the world but to experience it. And the body that makes this experience possible.
The AI that exists today is a tool. A powerful, a transformative, sometimes a frightening tool. But a tool.
What comes tomorrow could be something else entirely.
AI has the potential to automate nearly everything — perhaps even ourselves. The question is not whether it is possible. The question is whether, in the process, we forget what makes us human.