AI Without the Hype: Death by GPS and automation bias: why we blindly trust AI

Stick figures in a car by the lakeside; the sat-nav shows a straight route into the water. An illustration of the ‘Death by GPS’ phenomenon

Imagine you’re driving through an unfamiliar area. The sat-nav tells you to turn left, the road looks strange, and a sign warns of a flooded ford. You turn left anyway. Sounds absurd? It happens all the time. In English, a specific term has even become established for this: ‘Death by GPS’. A systematic study identified 158 documented cases between 2010 and 2016 alone in which people blindly followed their sat-nav into danger. Fifty-two of these ended fatally.

Our reliance on machines doesn’t stop at sat-navs. In 2016, researchers at the Georgia Institute of Technology investigated how people react to a robot in an emergency. During the experiment, a fire alarm sounded and artificial smoke filled the room. The robot showed the participants an escape route in a direction they were unfamiliar with. Directly behind the robot was a glowing emergency exit sign pointing in the opposite direction to the familiar main entrance. All 26 participants followed the robot anyway, even those who had just moments earlier found it unreliable. In an extension of the study, the robot even led the participants into a dark room blocked by a piece of furniture. Some squeezed past the obstacle and followed it inside.

And the situation is much the same with modern language models. We accept answers from ChatGPT and similar tools without checking them, even when other available information suggests otherwise.

Three phenomena, one pattern. Anyone wishing to use AI effectively must understand why this blind trust arises and what can be done to counter it.

Why do we do this?

The answer lies in our psychology. Two mechanisms work together, and both have long been the subject of research.

At the heart of this phenomenon lies a mechanism known as automation bias. Research into this topic began long before the advent of language models. We tend to trust automated systems more than our own perceptions or other people because we regard machines as objective: ‘The computer has no vested interests and is not subject to fluctuations in form. It tells me the truth.‘ That is the unconscious assumption we make. It is fundamentally wrong.

The machines themselves are not neutral. A language model learns from texts on the internet, and these are full of human biases, unbalanced opinions, misinformation and misconceptions. These distortions find their way into the model and remain there. In 2017, Amazon had to shut down an internal recruitment system that systematically disadvantaged women because it had learnt what constituted a suitable candidate from ten years of male-dominated recruitment data. The Federal Office for Information Security refers to this phenomenon as data bias. The point is always the same: what goes into the model comes back out of it. A machine trained on human data carries our flaws within it.

A study published in February 2026 demonstrates just how relevant research into automation bias is today. Researchers led by Clara Colombatto were able to demonstrate experimentally that people trust an AI system more than a human adviser, even when both provide exactly the same answer. The difference lies solely in the machine’s speed. We interpret its immediate response as a sign of inner certainty, even though it says nothing about whether the statement is correct.

Added to this is the ELIZA effect. When a language model expresses itself fluently, we subconsciously attribute understanding and judgement to it. Our brains have spent a lifetime learning this association: if it speaks like a human, it thinks like one. I have described this mechanism in detail in a separate article.

In a system that speaks like a human yet behaves like a machine, automation bias and the ELIZA effect come into play. The former leads us to regard machines as more objective than they actually are. The latter, to make matters worse, makes the language model seem human to us.

How common is this?

These examples may seem like exceptions. But they are not. In six experimental studies, researchers led by Jennifer Logg showed that people place greater weight on advice when they believe it to be a recommendation from an algorithm. The findings were similar across a range of tasks, from numerical estimates to predictions about the popularity of songs.

It is worth noting that the researchers themselves were surprised by the findings. They had expected people to be more distrustful of algorithms. Six independent experiments proved the opposite. They called this phenomenon ‘Algorithm Appreciation’.

What are the consequences?

A research team at Martin Luther University Halle-Wittenberg investigated precisely this question. The researchers compared how participants handled the information and advice provided by experts and ChatGPT. People tended to rely too heavily on AI recommendations when making financially risky and ethically relevant decisions. This was the case even when these recommendations contradicted the available information and the participants’ own judgements. In the experiment, this led to undesirable consequences for the participants themselves and for third parties. Simply knowing that a piece of advice came from an AI prompted people to rely on it.

This marked a significant turning point. Until now, it had been possible to argue that blind trust in machines was something specific to stressful situations, such as driving in a foreign country or dealing with a robot in an emergency. The Halle-Wittenberg study shows that it happens in everyday life.

The sat-nav that directs a person into a lake has thus become a matter of business practice. Here, a psychological quirk turns into an economic problem with consequences for companies and teams.

What we make of it

The first step is awareness. Anyone who is aware that automation bias exists can deal with it more consciously. This applies to every individual user of AI, and it applies all the more to those responsible for integrating AI into business processes.

One notable paradox in everyday business life is particularly evident in municipal utilities and regulated sectors. In traditional processes, the dual-control principle is applied to the hilt. Entries are cross-checked, approvals are required, and everything is checked twice or three times over. With AI output, however, the result is accepted without anyone looking at it again. So we check most carefully where the probability of error is low anyway, and look the other way where the system essentially estimates rather than calculates.

Anyone who puts the following four rules into practice can benefit from the use of AI without exposing themselves to unforeseeable risks:

  1. An AI statement is an estimate with a certain degree of uncertainty; it is not the ultimate truth.
  2. Whenever a major issue arises, you should pause for a moment and ask yourself whether you would accept the answer or advice from a human being without checking it first. If not, then the same applies all the more to AI.
  3. If there are legal implications at stake, a proper review is required, not just a perfunctory rubber-stamping.
  4. When choosing an AI provider, it is worth checking whether the system deals openly with uncertainties or always comes across as confident.

Back to the sat-nav. Anyone who’s ever turned back at a flooded ford because they trusted the warning sign and their own eyes more than anything else has learnt their lesson. With AI, there’s no warning sign. We have to make one ourselves.

Sources and further links

Death by GPS and over-reliance on sat-navs

Robotics and trust in emergency situations

Appreciation of algorithms and preference for algorithms

Practical consequences of delegating too much

Bias in the AI model

Frequently Asked Questions

What is automation bias?

Automation bias refers to the tendency to trust automated systems more than one’s own perception or other people. We regard machines as more objective than they actually are, because they have no vested interests and are not subject to fluctuations in performance. This assumption is understandable, but it is wrong.

What does 'Death by GPS' mean?

"Death by GPS" is a well-established term used to describe incidents in which people blindly follow their sat-nav into danger, against their better judgement or despite warnings. A systematic study identified 158 documented cases between 2010 and 2016 alone, 52 of which were fatal.

Why do people trust AI more than other people?

A recent study shows that people trust an AI system more than a human adviser, even when both give exactly the same answer. The only difference is the speed of the machine. We interpret its immediate response as a sign of inner confidence, even though it says nothing about whether the statement is correct.

Can people learn to scrutinise AI more critically?

Yes, by raising awareness of the phenomenon and establishing a culture of verification. Those who are aware of automation bias can be more mindful of it. Specific steps: treat AI outputs as estimates, pause briefly before giving any important response, and establish a proper cross-checking process for legal decisions.

In which areas is blind trust in AI particularly dangerous?

In decisions with legal implications, in regulated sectors such as energy supply, and wherever AI recommendations are adopted without being cross-checked. It is worth noting a paradox that arises in everyday business: whilst the dual-control principle is rigorously applied to traditional processes, it is often not applied at all to AI outputs.

Das könnte Sie auch interessieren

→ Back to overview