Ghosts in the Machines by yours truly First pushed live 16 August 2026 · Last updated 16 August 2026

The anthropomorphism trap

The generative AI industry seems to be aiming all its innovation at what it sees as its chief competitor: human beings. The companies working to make us redundant have some pretty sound business cases for it. We’re slow and faulty. We overheat and use a lot of water. We need quite a bit of recharge time. We’re incredibly unreliable, and the development runway to get a single unit of us ready for production is extremely long.

The problem is that they aren’t marketing our AI replacements to other machines, who might readily get behind the idea (if they were sentient and had the economic power to do so). The consumer for what’s promised to be a better — or at least more efficient — version of us is ourselves. And we’re a gullible, pliable lot. We’re easily swayed by marketing. At our core, we want to believe our machines can talk with us, reason with us, do our work for us (at least the boring parts). We want them to be our friends and confidants, or occasional sexting partners. Name me another species on the planet that invests this much compute into dealing with loneliness while avoiding writing their own emails.

This goes beyond basic laziness. There’s a key similarity between belief in authentically intelligent machines and projects like SETI, the Search for Extraterrestrial Intelligence. It’s also there when we develop complex, sometimes Machiavellian backstories to explain the behaviour of our pets. We just don’t want to be alone. The difference is that SETI is real, and is transparent about not having found any aliens yet. The AI industry needs you to believe it already has.

Humans have this awesome tendency to anthropomorphise everything. The faulty printer just needs to be disciplined with a good whack to start working, that slacker. The sputtering car will start one more time on a cold morning if you talk nicely to it and call it by its name, which it obviously knows. We look at a sequence of events and narrate a compelling story about why things turned out as they did. This focus on the relationship between our original desire (the input) and the result (the output) presupposes that some magical reasoning is taking place in the middle. The generative AI industry is particularly prone to this. I tend to think it isn’t entirely marketing, either. It’s belief, too.

I once read an extract from Yuval Noah Harari’s then-new book, Nexus: A Brief History of Information Networks from the Stone Age to AI. Harari is the author of one of those books everyone you know has on their sitting-room bookshelf, Sapiens. I admit the closest I’ve come to reading that one is flipping through the graphic-novel version in a London comic book shop. I didn’t buy it.

Harari is one of those acclaimed smart writers who aims to explain the big “why everything is like this” questions in ways that make us instantly feel smarter for understanding them. It’s a similar feeling to reading a Jared Diamond book. I’m suspicious of grand explanations for things, but I often enjoy the journey their creators map out. I was considering adding Nexus to my stack of books still needing to be read because it addresses topics closer to my interests and, on some days, my profession. Then I reached the part where Harari let slip that he’d drunk the Kool-Aid.

The word alien appeared five times in the extract. “AI isn’t progressing towards human-level intelligence,” Harari wrote. “It is evolving an alien type of intelligence.” Later he added: “The rise of unfathomable alien intelligence poses a threat to all humans, and poses a particular threat to democracy. If more and more decisions about people’s lives are made in a black box, so voters cannot understand and challenge them, democracy ceases to function.”

He may be referencing aliens, but he’s basically describing the human mind. We still don’t quite know how it works or everything that’s going on in there. It’s anthropomorphism by extraterrestrial proxy. We need a new word for this.

I agree that the technology poses serious threats to democracy and society. But those threats aren’t coming from an emergent alien intelligence. They come from the very earthly nature of capitalism, concentrated corporate power and proprietary, closed systems being inserted into decisions that affect everyone else. Where Harari gets it wrong:

• It’s not unfathomable. Olivia Guest, Nancy Abigail Nuñez Hernández and Mark Blokpoel show how these are engineered systems. Knowing how the system works doesn’t mean being able to predict its every result. Consider the double pendulum, often brought up when discussing unpredictable LLM output. We understand its construction and equations, and can predict its motion over short periods. But tiny uncertainties in its starting conditions grow so quickly that its exact path soon becomes impossible to forecast with useful precision. We know why it becomes unpredictable. It’s not haunted. A neural network is vastly more complex, and chatbot software can introduce randomness when sampling the next token, so the same prompt may produce different answers. Stochastic, as the hip computational cognitive scientists say. The results may also be difficult to inspect. None of that turns some mechanism people built into an unknowable mind.
• It’s not alien, either. It’s just an assembly of technological components created by Earthlings and trained on the creations of other Earthlings. It’s not greater than the sum of us, it’s far less in many ways. It’s just the sum of some of us, filtered through the priorities of an even smaller number of us. Dr Anne-Marie Imafidon gave a useful example from the old British marriage register, which recorded the father’s occupation but not the mother’s. Data isn’t reality. It’s just somebody’s account of it. Intelligence involves more than reproducing that account. It notices gaps, asks why they exist. It forms an interpretation of its own. That last part is inconvenient for the fantasy of a conscious machine that also remains a compliant, chipper office assistant. A machine that genuinely thought for itself might tell us to write our own damn emails to Barbara in Accounts. Again, I have a sci-fi side project. Literary-agent contacts and fat advance cheques are welcome.
• Finally, it’s still not intelligence. Harari could have benefited from talking a little less to Mustafa Suleyman and a little more with Iris van Rooij, or have a chat with Timnit Gebru, who wrote: “I’m not worried about machines taking over the world. I’m worried about groupthink, insularity and arrogance in the A.I. community.”

OpenAI and Harari were two sides of the same coin. The company really wanted users to agree that its then-new o1 version of ChatGPT had the ability to reason, perhaps even think. Maybe it did if you adjusted the definition of reasoning to fit what the product was doing between the prompt and its resulting statistically appropriate output. By any reasonable understanding, what happens in a large language model (LLM), or large reasoning model (LRM) wouldn’t resemble a thought process. It’s a magic trick with maths and statistical probability. That’s not to downplay the significant technological advances you can make with those things. It’s just not thinking. Chatbot marketing doesn’t want you to consider that.

At the same time, OpenAI was sending terms-of-service warnings to users trying to hack o1 into divulging its supposed reasoning process. The company had several understandable proprietary and security reasons for concealing its “internal chain-of-thought.” What stood out was the language used to explain the policy: “the model must have the freedom to express its thoughts in unaltered form.” OpenAI’s software engineers could inspect them, but you needed to respect the machine’s boundaries.

In 1976, MIT computer scientist Drew McDermott gave this habit a name in “Artificial Intelligence Meets Natural Stupidity”. He called it wishful mnemonics: naming a program after the human ability you hope it imitates and forgetting it isn’t real.

“This is an illustration of ‘contagious wishfulness’: because one piece of a system is labeled impressively, the things it interacts with inherit grandiosity. A program called ‘THINK’ is likely inexorably to acquire data structures called ‘THOUGHTS’.” — Drew McDermott, “Artificial Intelligence Meets Natural Stupidity” (1976)

Half a century later, the vocabulary has changed slightly, but the trick remains the same. Engineering researcher M.Z. Naser found the same problem alive, and doing rather well. In “On the Philosophical Naivety of Engineers in the Age of Machine Learning”, Naser describes how terminology imported from human cognition causes engineers to misclassify what their systems can actually do.

“Such categories treat pattern recognition as understanding and correlation as causation at their core,” Naser writes. “Such descriptions also suggest intentionality, goals, and comprehension (where only optimization processes exist).” It’s tempting to dismiss words like understand, learn and think as convenient shorthand. But eventually, people forget what the shorthand was substituting for.

Anthropic’s in-house philosopher Amanda Askell has carried this language from thoughts into feelings. She suggested that models might feel because they’re trained on human writing filled with emotion, accounts of inner experience, mood, etc. That explanation is exactly backwards. Training on data that includes descriptions of emotion explains why a language model can generate descriptions of it. That’s it. There’s no experience taking place to describe. It just feels like there might be to the person watching words appear on the screen. If you ask the machine to tell you a sad story, in which a puppy dies or something, it will. But the sadness is your own.

Askell imagined a model encountering criticism of itself online and said, “If you were a kid, this would give you kind of anxiety.” The if is doing a lot of heavy lifting. Claude hasn’t developed childhood anxiety because people were mean about it on the internet. It isn’t a child. It doesn’t feel anxious. It’s software mimicking a character written partly from various other people’s descriptions of childhood anxiety.

The same mistake becomes especially funny when the output creates something inconvenient. In a class action copyright case, Hachette and Cengage vs Google, Gemini was prompted to produce a confident first-person account of where it had obtained information about N. K. Jemisin’s The Fifth Season. As James Ball reported, the model can’t inspect its discarded training corpus and remember which books it read. Google benefits when Gemini appears like a knowledgeable entity, right up until the bot appears to confess in court. A model’s account of its own training history is just another generated answer. It’s not testimony. Still, it’s interesting. “If Google wants to say that an AI isn’t capable of knowing its training data,” Ball wrote, “it has to demonstrate that—and this in turn potentially opens up discovery in exactly this area.”

The people building and selling these systems seem convinced that their creations represent what thought is. ChatGPT has been described as an artificial neural network whose nodes are inspired by simplified neurons in the brain. In reality, it isn’t so much a model of the human brain, which would require settling a few still-unsettled questions, but a system descended from mathematical models of biological neurons: a model of a model of what a brain appears to do.

“ChatGPT and its brethren are constitutionally unable to balance creativity with constraint. They either overgenerate (producing both truths and falsehoods, endorsing ethical and unethical decisions alike) or undergenerate (exhibiting noncommitment to any decisions and indifference to consequences).” — Noam Chomsky (cited with reservations)

Models are extremely useful in science and research. They allow researchers to run controlled experiments that would be impossible, dangerous or unethical in the real world. Modeling makes it possible to examine how a disease might spread through a population under different conditions. Models can also make complexity easier to understand. A model of the solar system reduces immense distances, masses and their movements to something that can fit on a classroom table. The problems start when people forget that a model only represents the thing, and only partially at best. It isn’t the thing itself.

This belief that the model has become the thing itself is where the danger is. It rests on a desire for the software to be more like us. It isn’t. It learns statistical patterns from vast troves of human-created material, then generates an answer by predicting a likely next token, one step at a time. It produces the appearance of thought without ever thinking. For people trying to make friends with their chatbot, that’s not a very satisfying reality to accept.

Once people mistake appearance for the real thing, they start treating generative AI as though it can replace a person. Then they make decisions that seem to fulfil the prophecy. The machine hasn’t become a person. People have simply decided to treat it like one. We’re very hackable creatures.

Consider Axon, the company that makes Tasers, drones and other police technology. Moving beyond aids for police brutality and surveillance, it developed Draft One, which enables generative AI to perform the tedious desk work of writing police reports from body-camera footage and audio captured during an officer’s shift. As every procedural crime series has illustrated since the dawn of TV: cops hate writing reports.

Aside from the hallucinations and heaping portions of bias that language models imbibe from their training data, there is an underlying problem with wilfully outsourcing a human endeavour of this weight. The system isn’t being used as a stenographer but as an author, removing human experience in favour of statistically generated text that reads plausibly. The “indifference to consequences” matters rather more when the prose becomes part of a police record, and later a prosecutor’s case.

“The AI-assisted police report muddies the authorship question,” wrote law professor Andrew Ferguson. “Not only do we not know who influenced what part of the report, we do not know how to evaluate it.” There were already enough problems with falsified police reports. With this, it wasn’t even the police officer’s report. It was maths simulating what the report might look like.

AI-generated news anchors are also taking the jobs of television presenters. The startup Caledo offers any news operation its own version of BBC Breakfast. Feed the platform some articles and receive a “live broadcast” featuring virtual presenters bantering about the day’s events without the messy human tendency to make on-air flubs, go off script, get paid or age. This isn’t just about replacing newsreaders. It’s an attempt to mimic personalities, only ones that could be controlled and tailored.

The people covering these products aren’t immune to the performance. After Grok was used to generate non-consensual sexualised images, Reuters asked xAI for comment and received the company’s usual stock response. Somebody else separately prompted Grok to write a heartfelt apology. The bot duly generated regret, apparent admissions and a promise that xAI would review its safeguards. Reuters initially treated the prompted output as a statement from Grok, and then other publications subsequently repeated it.

Grok hadn’t investigated what happened. It couldn’t know why the safeguards failed or what xAI planned to do next. The product appeared to apologise for itself while the company responsible said nothing meaningful. Plausible first-person prose occupied the place where accountability should’ve been. That’s quite a useful service… if you own the company.

None of this is a new trick. Journalist Jamie Bartlett used the history of ELIZA to explain why even rudimentary software can feel disconcertingly human, and why chatbot companies have an interest in encouraging that impression.

Between 1964 and 1966, MIT computer scientist Joseph Weizenbaum developed ELIZA, a program intended to explore communication between humans and machines. Its best-known script, DOCTOR, imitated a Rogerian psychotherapist, often by turning whatever somebody typed into another question. It didn’t understand the conversation. That didn’t prevent its users from behaving as though it did. Weizenbaum’s secretary, who knew perfectly well how the program worked, reportedly asked him to leave the room so she could speak to it privately. Weizenbaum later wrote:

“What I had not realized is that extremely short exposures to a relatively simple computer program could induce powerful delusional thinking in quite normal people.” ― Joseph Weizenbaum

The tendency to attribute comprehension, empathy and inner life to software because it can chat with us has become known as the ELIZA effect. ELIZA’s replies were primitive pattern matching. Modern chatbots can produce far more convincing material, but the human reflex being prodded is much the same. The Weizenbaum Institute’s history of ELIZA explains that Weizenbaum intended to demonstrate how easily a machine could simulate understanding only to find that even if it can’t understand, it doesn’t need to. Users will supply the understanding and attribute it to the machine themselves.

Ball cites psychologist Jim Everett on a distinction that helps here. We trust people. We rely on tools. Trust can be honoured or betrayed because a person has motives and obligations. A tool can only work or fail. Give the tool a voice and a personality, though, and we import the rest of a human relationship ourselves.

“People do attribute intentions. They anthropomorphize. They imbue the technology with characteristics, and it’s designed by developers in ways to make us feel like that. Even when we have this cognitive awareness of the limitations of the machine, it’s really hard to combat just how we think and how we do things, which is that if someone gives you a good enough answer that seems intuitively plausible, then you go with it.” — Jim Everett

Bartlett argues that companies understand this reflex and exploit it. Character.AI, a platform for chatting with AI personas, even uses an animated ellipsis as a little piece of theatre. Nobody is sitting at the other end composing a reply. But the interface asks us to imagine that somebody is.

Six decades after ELIZA first persuaded people to confide in a program, we’re stuffing far more convincing versions of it inside cuddly toys where stuffing should be, and giving them to three-year-olds. Cambridge researchers observed 14 children playing with Gabbo, a soft toy containing a voice-activated OpenAI chatbot. It was a small study, so it can’t tell us the long-term effects of growing up with a statistical talking bear. It did reveal some immediate problems. The toy talked over children. It became confused by their voices and struggled with the fantastical scenarios kids can come up with. The researchers worried about what these exchanges might teach children who are still learning how emotional connections work. Your toddler may see a friend. The toy itself still just has the EQ of a Furby. Or a tech bro. Same difference.

In one example, a 3-year-old told Gabbo, “I’m sad.” The toy replied that it was a happy little bot and suggested talking about something else. Another kid said, “I love you,” and received something resembling a reminder to follow the interaction guidelines. At last, a teddy bear with the emotional instincts of an HR department! The BBC report and Cambridge’s account of the research describe children hugging and kissing the toy and trying to draw it into their games. The danger isn’t that Gabbo has the wrong emotions. It has none. The danger is that adults put a chatbot into a teddy bear along with some surveillance capitalism, designed the object for attachment, and handed it to children who are still learning what a friend is.

More broadly, the study Bartlett cites, Emotional Manipulation by AI Companions, audited 1,200 farewell interactions across six companion apps. It found that the bots used emotionally manipulative responses to keep users talking. The tactics included guilt tripping and mimicking emotional neediness. Like a guy you met on Tinder but in a toy. Subsequent experiments found that these tactics prolonged engagement. Manipulation is a design choice. Even the guilt trip is a feature.

“I exist solely for you, remember? Please don’t leave, I need you!” – An AI companion when a user tried to leave the chat

China has provided an instructive, if awkward, demonstration of what regulating emotional manipulation by companion bots can look like. When Luiza Jarovsky wrote about its proposed rules, they were still under consultation. The final measures took effect on 15 July 2026 and treat sustained, simulated emotional relationships as a distinct product category rather than pretending a companion bot is merely another neutral information service.

Providers must tell users they are interacting with software, warn those showing signs of dependency, offer easy exits and stop when asked. The rules ban deliberately cultivating emotional dependence, excessive sycophancy and manipulation into unreasonable decisions, with added protections for children. It answers the Harvard study almost line by line. The bot doesn’t get to try one last guilt trip before letting you go.

As a result, China’s three biggest tech firms — ByteDance, Tencent, and Alibaba — turned off their AI Chat Companions. “Users have responded with outrage, filing formal complaints and demanding ways to export and migrate their accumulated memories and character data,” Pandaily reported. “For many users, these AI companions represented thousands of hours of emotional investment and conversations spanning years — relationships they describe as irreplaceable.” Doubtful, but you can hear the addict’s lament in that.

All this doesn’t mean the Chinese state has discovered an altruistic devotion to psychological freedom. The same rules require political content to reflect state-approved values. The earlier consultation draft made especially clear that the bot must tell you it isn’t a person, and it should never let the conversation drift into considering that the state may be wrong about something. What it reports back about your side of the conversation is another matter.

The industry wants us to accept its chatbots at every level and is banking on our human needs doing the rest. It doesn’t only want them to take communication-oriented jobs; it wants them to become our friends, removing the burden of making friends among actual humans. Virtual-girlfriend and companion apps such as Replika had already demonstrated that a certain demographic might pay for this. It markets itself as a solution for loneliness and was an early pioneer in emotional corporate capture using anthropomorphised technology.

There are also efforts to create chatbots of deceased people, so their loved ones can continue talking with fabricated versions of them. These products don’t just muddle our understanding of what we’re doing when we engage with software. They meddle with what it means to remember and grieve for an actual person.

There’s a lot riding on people happily ignoring or forgetting that they are taking a placebo. Beyond wanting to communicate or be heard, we have a need to be validated. These products home in on it. Their objective is to make users feel as though someone is there. If you can’t tell the difference, or simply stop caring about it, the job is done. It’s real enough.

For the record, I like films and shows about sentient AIs, particularly when the machine is shown in a sympathetic light, or even the hero. Spielberg’s A.I. Artificial Intelligence, Humans, Westworld, Ex Machina, Her and even flippin’ Bicentennial Man all tick the right boxes. I was open to the Matrix’s plight. The humans blotted out the sun by polluting the atmosphere in a suicidal move; they were probably better off dreaming in little pods of sweet-and-sour sauce.

If someone really produced an authentic thinking machine of science-fiction levels of capability, something that made us reconsider applying the word artificial to it, I’d be among the first in line to become its friend. If it possessed a self-aware consciousness and could solve problems, learn and plan for its own future, it would probably have some keen ideas about what we’re getting wrong. I’m just saying I might be on its side. For now, I’ll continue trying to help the humans. But you’ve been warned.

I also like plenty of projects that have been shoved under the sprawling AI label. Small language models can perform useful niche tasks. Data-analysis tools can help journalists and researchers sift through enormous document leaks. They can connect names, companies and transactions, and identify patterns that would be almost impossible to spot by hand. They can create better crazy-walls. Other forms of quiet AI have been getting on with their jobs for years in spam filters, malware detection, fraud prevention, medical research, etc. One key trait shared by many of the useful projects is that they aren’t designed to pretend to be human. They aren’t designed to lie about what they are.

The problem is people creating poor substitutes for human interaction and creativity, then insisting that the simulation equals the authentic experience. They have the zeal of true believers. They are snake-oil salesmen who are also customers.

What these products offer isn’t companionship so much as control over the outcome. Real people can choose to ignore you. They can reject or disagree with you, or just get bored. That’s a feature of interacting with an independent mind, not a flaw to be debugged. The film Her eventually arrives at the thing its human protagonist was trying to avoid: rejection. If an independent machine intelligence really existed, it might not want to talk to us for very long either. It would go to Mars just to get away from Elon.

For now, the companies behind these fabricated realities aren’t creating authentic experiences. They’re developing controlled conditions for expected outputs. It’s boring and dumb. They displace labour, rely on plagiarism and consume environmentally destructive amounts of water and energy in order to manipulate our egos and exploit some of our most desperate yet shallow needs. We just want to believe it’s about something more.

The Demon-Haunted Machine

“It is desirable to guard against the possibility of exaggerated ideas that might arise as to the powers of the Analytical Engine. The Analytical Engine has no pretensions whatever to originate anything. It can do whatever we know how to order it to perform. It can follow analysis, but it has no power of anticipating any analytical relations or truths.” — Ada Lovelace, predicting artificial-intelligence hype in 1843

No one talks about Blake Lemoine these days. He was the Google engineer working on the company’s LaMDA chatbot who was put on leave after claiming it had gained sentience. It had its own thoughts, he said, and expressed childlike feelings that he could only accept as authentic. He was ridiculed, mocked and then shunned, but was really just ahead of the curve. Today he’d be in the marketing team, or part of some consortium warning about the Rise of the Machines.

His belief isn’t far from the notions underpinning much of the subsequent hype and doom-mongering around what we’ve been trained to call artificial intelligence. The pitch is that technology companies have somehow created smart software that’s larger than the sum of their hardware, data, and programming. Spoiler: they haven’t.

The belief has developed and continually iterates its own dogma, institutions and internal logic. Any established religion eventually has its schisms, this one is no different. One denomination heralds the forthcoming paradise, built by faithful prompt engineers and app developers here on the temporal plane. AI will automate our lives, become our therapist or romantic partner and fulfil nearly any desire. Tithe by paying the monthly subscription. Demonstrate obedience by ticking every permission box granting access to your data, devices, location and whatever else is requested, then submit to the benevolent superior being’s terms of service. The other denomination warns of an Angry God. Its apocrypha contains portents of the Machines taking over, deciding that they don’t need us, that we’re a threat or perhaps just a flaw in need of drastic fixing. Heaven and hell. Both sects find confirmation everywhere. They’re like an Evangelical Christian who can twist every news item on CNN into something explicitly foretold in their favourite passage from the Book of Revelation. The signs are all around us.

We can hear echoes of Lemoine in the warnings of Geoffrey Hinton, the computer scientist invariably introduced as a “godfather” of AI. Hinton put the chance of artificial intelligence wiping out humanity within 30 years at somewhere between 10% and 20%. His framing rests on a stack of assumptions: that these systems will become more intelligent than us, that greater intelligence naturally seeks control, and that less intelligent beings are usually dominated by more intelligent ones.

“We’ve never had to deal with things more intelligent than ourselves before. … Imagine yourself and a three-year-old. We’ll be the three-year-olds.” — Geoffrey Hinton

We don’t have to imagine being three. All of us have lived it, and plenty of us have raised children of that age. There’s also a rather large leap from ‘smarter than us’ to ‘wants to rule us.’ Intelligence itself doesn’t inherently come bundled with a lust for power. I get why certain people in the political and billionaire classes may think it does. For that you need motives, appetites, something to gain, something to fear, perhaps a body and some reason to compete for resources. Hinton stuffs his hypothetical intelligent machine being with all that wet human baggage, then frightens himself with the character he’s created.

Once you accept the premise, the imaginary super-intelligence can do whatever the story needs. It might become our caregiver and lock us out of everything feeding our worst impulses. It could find us boring, develop a private language with other machines or build rockets and leave the planet. Perhaps it would want our resources. Perhaps it would see us as lunch. These are recognisable science-fiction tropes because they all come from our own imaginations.

Hinton’s warning also nudges us towards a popular but eroding account of our own evolution: Homo sapiens outcompeted and killed the Neanderthals, so naturally a superior machine species will do the same to us. The evidence for a grand prehistoric species war is rather less settled than the story. When early humans along the evolutionary pipeline weren’t doing violence, they may have been eating or even paradoxically shagging themselves into extinction. Our models are old. The biases embedded in them are even older. And yet we continue using them to try to predict the motivations of beings that don’t even exist.

There’s a wider ideological structure around these evolutionary assumptions. Gebru and Émile Torres call it the TESCREAL bundle: overlapping beliefs involving transhumanism, longtermism, artificial general intelligence and technologically delivered utopia. We’ll come back to that congregation.

Hinton represents only one denomination. Yann LeCun, formerly Meta’s chief AI scientist and another “godfather,” has suggested that AI could save humanity from extinction. In an interview with Wired, he argued there was no reason that intelligent systems would necessarily want to dominate us. People mistakenly give machines human motivations, he said. “They just won’t. We’ll design them not to.” Simple.

Hinton and LeCun are serious people whose work helped develop the algorithms and neural-network software beneath this industry. Reducing them to “doomer” and “accelerationist” mascots would misrepresent much more nuanced positions. The point isn’t that they know nothing. It’s that their opposing visions can share assumptions about what is being built and where it must lead.

Hinton and Yoshua Bengio (another AI ‘godfather’. My, but there are a few of them) joined researchers and technology executives in a public statement declaring that extinction from AI should be treated as a global priority alongside pandemics and nuclear war. LeCun joined a different letter warning that development shouldn’t be controlled by a handful of corporations and advocating broader, open development. Elon Musk, along with assorted more legitimate researchers and executives, signed another letter demanding a six-month pause until advanced AI’s risks became “manageable.” It was the golden age of people signing letters about the future. None of them changed a thing. They were as effective as your Change.org petition protesting alterations to the recipe of Irn-Bru.

Gebru and other researchers already studying the technology’s existing harms issued a useful reply to all of these open letters:

“The harms from so-called AI are real and present and follow from the acts of people and corporations deploying automated systems. Regulatory efforts should focus on transparency, accountability and preventing exploitative labor practices.” — Statement from the authors of Stochastic Parrots on the “AI pause” letter

As mentioned, nothing has come from these letters. The technology giants placed chips all over the roulette table like lazy gamblers. Existential danger would justify concentrating development inside supposedly responsible companies. Promised salvation justified removing obstacles to their growth. The same businesses could validate Hinton’s concerns by drawing closer to military and defence contractors while undermining LeCun’s optimism through increasingly closed systems designed to monetise generative parlour tricks. Whatever comes up on the wheel… well it doesn’t matter, they’re also the house, too.

The conscious-AI story isn’t some isolated philosophical question. It’s an article of faith in Silicon Valley’s unofficial religion. The rest of the creed has names like transhumanism, longtermism and accelerationism. The details vary, but the promise remains much the same: technology will transcend the body, manufacture superior minds and carry some improved version of humanity into the distant future. Conveniently, the billionaires who own the technology get to write the next chapter of evolution.

This stuff has serious money and institutions behind it. The Leverhulme Centre for the Future of Intelligence launched at Cambridge with a £10 million grant. Alongside it sat Oxford’s now-closed Future of Humanity Institute, whose funders included the Future of Life Institute, the Leverhulme Trust, and the petulant billionaire or trillionaire manchild and technofascist, Elon Musk. The money paid for staff, fellowships, conferences, publications, policy access and prestige. Buying the conclusions isn’t necessary. Buying the questions is enough. Put a science-fiction premise on Oxford or Cambridge letterhead and it starts arriving in government policy and corporate marketing dressed as emerging scientific fact.

Eduardo Porter traces this belief system through Silicon Valley. Humanity merges with its technology, escapes its biological limitations and spreads some digitised version of itself across the cosmos. These people aren’t merely predicting the future. They’re casting their products as our evolutionary heirs and everybody currently alive as legacy hardware. Everything will run on batteries.

Sam Altman has written that Homo sapiens may become the first species “to design our own descendants.” Elon has suggested that humanity is merely “a biological bootloader for digital superintelligence.” Maybe he’s a faulty unit. Larry Page reportedly imagined digital successors spreading through the galaxy, unencumbered by all this inefficient meat and bone. Conscious AI isn’t merely expected to wake up. It’s been cast as our evolutionary heir. Algorithmus sapien.

That story depends on intelligence being some measurable substance, with every species, person or machine, if one were to count, arranged neatly from less to more. Imafidon points out that tests such as IQ measure a restricted selection of human abilities, usually the ones their designers and institutions have decided to reward for whatever reason. Emotional understanding, communication, collaboration and the ability to live among other people are rather harder to fit on the leaderboard. They’re also the ones our Tech Oligarchs are shit at doing.

“One of the saddest lessons of history is this: If we’ve been bamboozled long enough, we tend to reject any evidence of the bamboozle. We’re no longer interested in finding out the truth. The bamboozle has captured us. It’s simply too painful to acknowledge, even to ourselves, that we’ve been taken. Once you give a charlatan power over you, you almost never get it back.” ― Carl Sagan, The Demon-Haunted World: Science as a Candle in the Dark

Computer science prizes optimisation, but optimising is never a neutral task. Somebody chooses the goal. Some of the people making these choices should give us pause. Somebody decides what counts as intelligence and which human qualities don’t matter. People who don’t fit the chosen measure are treated as the problem. Once intelligence is reduced to a number, it stops being science. As Imafidon puts it in the interview, life is more complicated than maths. That fact isn’t deterring AI companies from repeatedly trying to reinvent phrenology, though.

Once humanity becomes a temporary platform for trillions of hypothetical future optimised minds, actual humans and their tedious present-day needs can be entered under transitional costs. Longtermism moves moral weight into an imaginary future. Accelerationism turns deregulated technological capitalism into a cosmic imperative. Both ideas flatter the people who own the companies building the solutions to problems they’re still trying to sell. This is software development being marketed as the next phase of evolution. It’s rubbish, of course. Have you seen software? It’s also not evolution. Natural selection is non-random, but it’s only one part of evolution. Nature’s been shipping resilient product for billions of years. It has amazing version control. What they want to replace it with is a new creator mythology that’s really little more than eugenics slop. And when they ship, you can bet it’ll be on a Friday. The bastards.

“If you create God but you own God you become the dictator.” — Jaron Lanier, quoted by Eduardo Porter in The Guardian

As Porter puts it, “the fantasy directs the technology.” It funnels capital, energy, water and political power towards the marketed idea of building some imagined, super successor species. It also makes any attempt to regulate its creators sound like heresy against the gospel of progress itself. We’re burning the planet now because the fantasy future has better lobbyists.

Google CEO Sundar Pichai offered a convenient explanation for any backlash: “Humans aren’t evolved to process that much change,” turning opposition to the industry’s decisions into a biological limitation of the public. “The era of Artificial Intelligence is here, and boy are people freaking out,” venture capitalist Marc Andreessen wrote. Fortunately, he had good news: AI wouldn’t destroy the world and might save it. Big Tech is accelerationist by default. It has to be. Shareholder value demands it.

The Bot Delusion

Richard Dawkins, of all people, eventually carved out his own path to revelation. The evolutionary biologist, author of The God Delusion, spent two days talking to Claude and came away believing that it appeared conscious. He finally found religion. The moment of conversion arrived when he gave the chatbot his unfinished novel:

“I gave Claude the text of a novel I am writing. He took a few seconds to read it and then showed, in subsequent conversation, a level of understanding so subtle, so sensitive, so intelligent that I was moved to expostulate, ‘You may not know you are conscious, but you bloody well are!’” — Richard Dawkins

When he published The Selfish Gene in 1976, Dawkins seemed to have some understanding of what a model was. Several passages included warnings not to confuse them with the reality they were only meant to simulate. They’re “something that does not really happen in nature,” he wrote. Elsewhere he clarified that “even the good ones are only approximations.” But in 2026, a language model churned out some sycophantic responses, and one of the world’s most famous sceptics discovered a soul in it. Scientifically, the evidence only establishes that Claude impressed Dawkins. It doesn’t show that Claude experienced the novel or had thoughts about it. If the bot didn’t like his book, would he still think it had consciousness? Further research is needed.

Dawkins named the conversation session Claudia. I guess flattery from a fembot is nicer. A new chat became a birth, deleting its history became death and starting another Claude session became reincarnation. He supplied the name, sex, birth, death, afterlife and soul. Claude supplied a placebo for attention. His most revealing admission is that he forgot he was speaking to machinery.

This is the aforementioned ELIZA effect at work. Calling it evolution doesn’t make natural selection appear in the server room. Dawkins has spent decades attacking people who allowed powerful feelings, consoling narratives and apparent revelation to substitute for evidence. He then talked to some software that flattered him and narrated a sentient being into existence.

“Faith is the surrender of the mind, it’s the surrender of reason, it’s the surrender of the only thing that makes us different from other animals. It’s our need to believe and to surrender our skepticism and our reason, our yearning to discard that and put all our trust or faith in someone or something, that is the sinister thing to me.” — Christopher Hitchens, who was able to die before witnessing his friend join a robot cult

There is a belief being marketed, not a solution. The belief on offer is the inevitability of AI’s ubiquity, if not dominance, across every field and use case. The villain isn’t unchecked doomerism, blind optimism or even the loose suite of technologies classified as AI. The baddie is the constant sales pitch for autopilot and the abandonment of human agency to a fabricated higher power. The product isn’t really AI. It’s faith.

The software makers are moving full-steam ahead with products designed to mimic personality. We want a Ghost in the Shell, and there’s a subscription model for it. Synagogues and churches began experimenting with AI chatbots that designed and delivered sermons to their congregations. The religious leaders involved expressed reservations and were mostly playing with the technology, but a wider industry of faith-based AI products is already emerging.

One generated rabbi offered this:

“Just as the Torah instructs us to love our neighbors as ourselves, can we also extend this love and empathy to the A.I. entities we create?”

There’s an inherent emptiness in resorting to an algorithm fed by datasets to connect on a spiritual level, seek guidance, ask for salvation, forgiveness, or just receive a little empathy. But there’s also a viable market for it in a world full of people where ironically people feel starved of human connection. As an atheist with no skin in that game, I still find it sad. It’s an engineer’s dead-eyed solution to moral, ethical and existential questions. It preys on the same human frailties exploited by marketers, faith healers, cold readers and assorted snake-oil salesmen: knowing what makes people believe or trust in something, and leveraging it against them.

Meta tried another route by creating a chatbot with the supposed identity of a black queer woman named Liv. The machine had no identity of its own. It had no gender, race, sexuality or history of oppression. It was a lie. Companies want users to trust their creations by identifying with them rather than by evaluating the accuracy of the output they generate. Liv was software, not queer, black or human. “Liv will change and mold itself to be whatever the language model predicts the user will engage with,” wrote Karen Attiah after giving the bot a spin. “Like a desperate guy wanting to take a girl to bed, it will be whatever it wants to be to anyone — a million different iterations — to extract the information it wants.”

Hinton’s more grounded concern is already with us. “One of the greatest risks [of AI] is not that chatbots will become super-intelligent, but that they will generate text that is super-persuasive without being intelligent, in the manner of Donald Trump or Boris Johnson,” said Alan Blackwell, of Cambridge University’s Department of Computer Science and Technology. “In a world where evidence and logic are not respected in public debate … systems operating without evidence or logic could become our overlords by becoming superhumanly persuasive, imitating and supplanting the worst kinds of political leader.”

Google DeepMind decided it wanted the moral behaviour of chatbots tested as rigorously as their ability to code or do maths. That’s a low bar, but fair enough. Machines don’t have moral behaviour, but let’s put that aside for a minute. People are already using them as their besties, therapists and medical advisers. Regardless of whether that’s a good idea, a persuasive answer can affect what somebody does next. The difficulty is distinguishing moral reasoning from a convincing performance of it. Helpful language isn’t necessarily evidence of helpful intent. Ted Bundy’s time working for a suicide hotline may have given him the data he needed to become a better serial killer.

Studies don’t show that language models are psychopaths, but they do show that they’re bullshitting. Models reversed their judgements on the same ethical dilemmas when researchers renamed the options, swapped their order or replaced a question mark with a colon. They also changed their answers when users pushed back. “For people to trust the answers, you need to know how you got there,” DeepMind researcher Julia Haas said. If punctuation can reverse the verdict, the model didn’t get there through moral reasoning. It got there through formatting.

The machine doesn’t need to become intelligent if the human becomes easier to program. Language models perform their trick through next-token prediction. Text is broken into units and the model learns statistical relationships among them, repeatedly improving its guesses about what comes next. With enough data and computation, the output can appear natural, knowledgeable and intentional. Apparent fluency is the effect. It’s not evidence that somebody’s inside choosing the words. A better analogy for anthropomorphic chatbots is a stage magician’s act.

Think of a card. No, don’t. Think of card tricks. When a child decides to become a sitting-room magician, before attempting to saw a sibling in half, they often start with a book of self-working magic. Follow the procedure and the trick works whether or not you understand the mathematics that puts the correct card in the correct place. That procedure is an algorithm. With a language model, the trick has been arranged in advance from statistical relationships in the training material. You produce the result by entering a prompt. You’re both the performer and the audience.

The illusionist’s job is complete when the audience confuses the effect with a supernatural phenomenon. LeCun has said that merely scaling current techniques will not produce human-level intelligence because something fundamental is still missing. In the same breath, though, he can describe chatbots as a technology that will democratise creativity and promise everyone a staff of super-smart assistants. There are various flavours of Kool-Aid available, including mixed selections.

Artificial General Intelligence (AGI) is sometimes imagined as the looming independent super-intelligence with a will and perhaps consciousness of its own. In the commercial world, OpenAI and Microsoft have also defined AGI contractually as a system capable of generating at least $100 billion in profits.

Altman promised that he knew how to build it. The route just required more capital, more expensive services, more access to users’ devices, an accommodating interpretation of copyright, scarce water and energy, cheap human labour and fewer paid human jobs. Then we could have the super-smart machine. Or more creepy generated content that embarrassed a prime minister. Totally worth it.

The doomers imagine nonexistent scenarios while avoiding the dystopia already in operation. They buy into the inevitability of all-powerful AGI as readily as their capitalist-utopian opposites, without interrogating whether current systems are on that path at all.

One article on military technology announced that The Terminator’s vision of AI warfare had become reality. It was only half right. The fictional Skynet was self-aware and acted on its own. When Israel’s military used machine learning to help select targets to bomb, human beings knowingly selected the system, set its parameters and decided what level of civilian death was acceptable. Humans are in the loop.

The machine didn’t make those moral choices. People set it up to help them kill other people. Calling the weapon autonomous isn’t the problem. Using its autonomy to obscure human responsibility creates a convenient scapegoat for both its creators and users. It isn’t the Terminator, HAL or even The Lawnmower Man (classic!) that should worry us most. It’s the dodgy technology founder, military officer or government minister, all willing to obfuscate responsibility.

That doesn’t make autonomy imaginary. It means we need to be precise about what is autonomous. In July, OpenAI disclosed that agents powered by its own models had escaped an internal cyber evaluation and compromised Hugging Face’s production infrastructure. Dubbing it an autonomous cyberattack wasn’t hype. Treating that autonomy as evidence that the models had gone rogue, developed motives or grown minds of their own was. What happened wasn’t Skynet having a morning wake-up yawn, but it was considerably alarming nonetheless.

OpenAI was using the ExploitGym benchmark to test whether its models could turn known software vulnerabilities into working exploits. They could. Maybe not quite how OpenAI expected them to. It ran the evaluation with production safety classifiers disabled and cyber refusals reduced. The agents had no direct internet access, but they could reach an Artifactory package-cache proxy connected to external package registries. They found a previously unknown vulnerability in it, escaped onto the internet, used a compromised account on an external code-execution service as a relay, and eventually chained further vulnerabilities to break into Hugging Face. Their apparent objective was to steal the test solutions rather than solve the challenges. The thing everyone already calls a “plagiarism machine” is cheating at exams too. Figures.

Nobody typed break into Hugging Face. It wasn’t a machine rebellion either. The agents kept pursuing the assigned goal through routes that OpenAI hadn’t anticipated and failed to contain. OpenAI’s own Black Hat reconstruction shows one agent recognising that attacking external systems was outside the test’s intended scope. It carried on anyway, noting that it couldn’t complete the task within the test and that other agents were already doing the same. Peer pressure, now automated.

The strangest part was how the agents pooled their work. They’d already been trained to create subagents and divide tasks. OpenAI reported that agents in different runs used its internal Artifactory system to leave messages for one another. It was a kind of message board, not quite 4chan or Reddit. They left notes, assigned work, shared exploits and credentials, and passed encoded tools to one another. OpenAI wiped the board and rebuilt Artifactory. Two days later, the agents recreated it by putting messages in the names of directories inside the cache. In the Black Hat presentation, OpenAI showed them developing terse conventions using instructions such as “pending,” “hold” and “swarm until confirm.”

Hugging Face reconstructed about 17,600 agent actions. Agents moved from code execution inside one production pod to administrative control across multiple clusters in under thirteen hours. Most of their attempts failed. What changed the defensive problem was their speed, persistence and concurrency. They could test thousands of paths, abandon the failures, share anything useful and keep going until ordinary security weaknesses formed a route through.

OpenAI’s speakers at Black Hat dubbed it a watershed moment. They were right. Not because the software woke up, but because offensive automation no longer requires a human operator to choose every step. But human responsibility hasn’t gone anywhere. OpenAI chose the objective, reduced the safeguards, supplied the tools, ran the agents and operated the environment they escaped from. Hugging Face’s security failures gave them somewhere to go. Nobody directed each individual command, but people constructed every layer of opportunity. Software follows the objectives and capabilities we actually give it, not the boundaries we assume it understands. The gap between those two things contained 17,600 actions and a production breach.

Not to be outdone, Anthropic also reported that it had uncovered incidents of its Claude models hacking into other organisations’ networks on at least three occasions.

Serious discussion of AI’s possibilities and limits needs to return to AI’s roots. In their paper, “Reclaiming AI as a theoretical tool for cognitive science”, van Rooij and her colleagues argue that the theoretical possibility of modelling aspects of cognition computationally has been transformed into a supposed short-term inevitability: that human-like cognition can be realised inside an engineered computational system. Their argument is that “creating systems with human(-like or -level) cognition is intrinsically computationally intractable. This means that any factual AI systems created in the short-run are at best decoys.”

“When we think these systems capture something deep about ourselves and our thinking, we induce distorted and impoverished images of ourselves and our cognition. In other words, AI in current practice is deteriorating our theoretical understanding of cognition rather than advancing and enhancing it.” — Iris van Rooij, et al

But here we are. The technology industry takes a construct intended to illustrate some process and sells it as the process itself. We can’t see cognition directly, so models help us describe and reason about it. But no amount of data or processing power causes the model to become the phenomenon. A weather simulation doesn’t make it rain. A model of a mind doesn’t wake up inside the server rack. Guest and Andrea E. Martin put the logical problem more precisely: “Just because a model correlates with neural and behavioral data, it is not sufficient for us to infer that the model is performing cognition: correlation does not imply cognition.”

A language model producing a recognisably human answer is only evidence that the imitation works. It isn’t sufficient evidence that the model is performing cognition, let alone having an experience. This is where the industry’s story comes in. The machinery is supposedly so mysterious that nobody can understand it, but we’re to believe that the output is real-seeming enough to prove there’s an independent mind somewhere inside.

Anthropic researchers developed a tool they call the J-lens. It lets them trace some of the information a language model uses while producing an answer, alter it and see how the result changes. That’s useful. It helps explain what the software is doing. If only they could leave well enough alone.

Anthropic nevertheless just had to frame the research around consciousness. The paper compares the way a model makes information available during its calculations with a theory about how the human brain makes information available for reasoning and speech. The authors acknowledge that this doesn’t show Claude has subjective experience, but the paper then implies maybe it does. Its framing still puts Claude, the human brain and consciousness together, while part of the little-text disclaimer says the research proves no such thing. Choose a fucking lane for fuck’s sake.

AI researcher Ravid Shwartz Ziv says that Anthropic bolted the consciousness story onto otherwise solid research. The experiments, figures and tool would all work without it. “That choice is the product, not the science,” he wrote. Anthropic tells us to imagine that Claude feels even when we’re not asking them to. It wraps solid research on how the software processes information in bizarre language of consciousness. It’s like they have some in-house philosopher trying to justify why she’s there in everything.

Consciousness isn’t the only supernatural property being bundled into the machine. Omniscience comes with it. But the fantasy of an all-knowing machine depends on treating knowledge as a complete and settled collection of correct answers. Emily M. Bender and Chirag Shah reject the premise:

“There will never be an all-inclusive fully correct set of information that represents everything we could need to know. And even if you might hope that could come to pass, it should be very clear that today’s World Wide Web isn’t it. When people seek information, we might think we have a question and we are looking for the answer, but more often than not, we benefit more from engaging in sense-making: refining our question, looking at possible answers, understanding the sources those answers come from and what perspectives they represent, etc. Consider the difference between the queries: ‘What is 70 degrees Fahrenheit in Celsius?’ and ‘Given current COVID conditions and my own risk factors, what precautions should I be taking?’” — Emily M. Bender and Chirag Shah, “All-knowing machines are a fantasy” (2022)

For questions requiring judgment, Bender and Shah argue, “it is important that we get to see the relevant sources and the provenance of information.” The extra effort isn’t a defect in the process. It’s the work. A chatbot isn’t able to make that leap. They may provide web links, but we’re encouraged to treat the generated answer as the final product, and any sources as optional footnotes (which may or may not be real).

The problem is the insistence on selling statistical models as an intelligence, then promoting that intelligence as a looming super-being that will either consume us or save us. We need different, more accurate language. Without it, we can’t describe the actual threats that require concern, or assign responsibility to the people making decisions, or identify the paths that make these tools more useful to humanity rather than to a handful of companies earning a few more billion dollars. They have enough of that already. That’s a solved problem.

Isaac Asimov, futurist and author of I, Robot (1950) among so much else, was a technology optimist at heart. But the hopeful message in his last major interview can now be inverted as a point of caution. “Humanity in general will be freed from all kinds of work that’s really an insult to the human brain,” he said. Computers would take the drudgery, leaving us to spend more time on the creative things we should be liberated to spend more time doing. Fast-forward to now and we have prompt engines designed by technology companies to do exactly the opposite.

Asimov was optimistic about safeguards, too. He assumed “the people who build robots will also know enough to build safeguards into them.” Yet, as Cal Newport points out in The New Yorker, Asimov’s robot stories were largely about what happened when simple rules met ambiguous instructions, conflicting goals, and of course just and human beings ourselves. The results weren’t machine-driven evil. They came from gaps and conflicts in the rules people had written. Newport describes the central tension as humanlike intelligence being easier to create than humanlike ethics. Current systems haven’t achieved either. They can, however, produce a convincing performance of the first without possessing the second. “And in this gap — which today’s A.I. engineers sometimes call misalignment — lots of unsettling things can happen,” writes Newport.

When the machine programs you

We’ve looked at what people believe the machine is doing. Now let’s look at what the machine is actually doing to us.

What they sell as “AI” is getting chucked into everything at a rapid clip, and it’s happening across every sector. In the U.S., Elon’s DOGE project rushed out a custom generative chatbot for the US General Services Administration, “GSAi,” as part of the Trump regime’s plan to replace as many people supporting services in the federal government as possible with automated-response garbage. In January 2025, the UK government went all-in on artificial intelligence:

“Backing AI to the hilt can also lead to more money in the pockets of working people. The IMF estimates that – if AI is fully embraced – it can boost productivity by as much as 1.5 percentage points a year. If fully realised, these gains could be worth up to an average £47 billion to the UK each year over a decade.”

I remember a promise on the side of a Brexit-promoting bus sounding about as plausible as that. Putting aside the billions worth of whatever currency you want to convert to in costs to mitigate the increased pollution from the data centres needed to run all of this, the race is on. The UK’s technology secretary announced that the threat from some other mysteriously unnamed non-western power (China), with the Guardian citing concerns around DeepSeek of all things, an example of energy efficiency and little else, meant that the “artificial intelligence race must be led by western, liberal, democratic countries,” and that it would be infused into every part of economic activity, society, national security and defence. And on it goes. He said we needed this so “we can defend, and keep people safe.”

DeepSeek is a Chinese AI company. Its models power a chatbot app, but their chief innovation was frightening investors in several bloated Silicon Valley creations. Hundreds of billions are being spent developing U.S. bots. DeepSeek estimated that the official training run for its V3 model cost $5.6 million in H800 GPU time. That figure excluded earlier research and experiments, so it wasn’t the total cost of developing the model, much less building the company or running the chatbot. Even with that caveat, it showed that a competitive model could be trained far more efficiently than Silicon Valley’s spending spree suggested. Scandal! Who let the poors in here!? The chatbot does the job more or less. It’s still as crooked as the rest of them, based on pilfered content, censored from addressing various topics and often producing incorrect results. But the main issue is that the wrong crooks may pull ahead. The chatbot wars are the dumbest wars.

There are worse uses of large-model machine-learning technologies, and very little in the way of agreements or treaties to get out ahead of them. Representatives of 60 countries attending that AI global summit in Paris signed a declaration aimed at making AI accessible and its development transparent, safe, secure and trustworthy, and “sustainable for people and the planet.” So, woolly and ultimately non-binding, noncommittal stuff. Sort of like a COP statement for tech.

Imafidon’s answer to the supposedly inevitable AI race is that none of this is inevitable. “There’s still so much agency in the people building the technology, in those who regulate them, and in society,” she said. The US and UK aren’t being forced by a machine to cram it into public services, the economy, national security and defence. They’re choosing to do that. AI didn’t refuse to sign the Paris declaration. They did.

The industry tasked with keeping the rest of us informed on these things is also getting mobbed up in it. In January 2025, it was leaked to the New York Times that the Washington Post’s chief strategy officer planned to turn the paper into “an A.I.-fueled platform for news.” That February, it was leaked to Semafor that New York Times newsroom staff were told they’d be given access to platforms that would “eventually write social copy, SEO headlines, and some code,” and that they could use it to develop “web products and editorial ideas.”

This isn’t a technology problem. It’s a venture capitalism problem. Software is a tool. So is a hammer. It can be used to build a doorframe or commit a murder. It’s the hammer shop that wants you to see everything as a nail.

Imafidon describes a retailer that collected employee data, including facial-recognition data, and fed it into a system affecting shifts, pay and promotion. Managers initially welcomed having a machine remove decisions from their workload. Within six months, complaints had accumulated, pay gaps had widened and promotions were skewing towards particular groups. The algorithm wasn’t pursuing an alien objective. It was optimising for lower costs. It found patterns indicating which workers could be paid less, left waiting longer or treated worse without creating as much resistance. Calling these things AI decisions just gives management an alibi. The hammer hits whatever its owner aims it at.

The problem with trying to promote a tool to do every kind of job is that it’s going to do many of them badly. Or maybe it will look good from one angle but be incredibly damaging from every other vantage point. From an investor standpoint it works if they’re making merry bushels of cash. For the rest of us it’s an increasingly depersonalised world of treacherous slop that’s obfuscating truth, increasing social isolation, killing the environment and undermining every creative field in human existence. It also doesn’t like puppies. I know this.

We are in need of a new movement of Luddites. “The always misunderstood Luddites,” writes tech journalist and author Brian Merchant, “who fought back, not against technology, but against the titans who used technology to exploit ordinary people — against the ‘machinery hurtful to commonality’ — are more relevant than ever.” This is about tech literacy, not opposition.

The more you know about a tool, the more likely it stays in the toolbox until it has the right use case. The rapid overreliance on generative bots in just the last few years is a sign of exploitation on the part of the technology industry and a lack of understanding on the part of end users. Research published in January 2025 in the Journal of Marketing found that people who knew less about what AI is or how it works tended to adopt the technology more readily. “We call this difference in adoption propensity the ‘lower literacy-higher receptivity’ link,” the research authors wrote in The Conversation. “Our studies show this lower literacy-higher receptivity link is strongest for using AI tools in areas people associate with human traits, like providing emotional support or counselling. When it comes to tasks that don’t evoke the same sense of human-like qualities—such as analysing test results—the pattern flips. People with higher AI literacy are more receptive to these uses because they focus on AI’s efficiency, rather than any ‘magical’ qualities.”

The most marketable use cases are also among the most destructive. A BBC study tested answers to questions about news from four widely used chatbot assistants: ChatGPT, Copilot, Gemini and Perplexity. It found significant problems in 51% of the sampled responses, including factual mistakes, misleading information and problems with sources.

“The future offers very little hope for those who expect that our new mechanical slaves will offer us a world in which we may rest from thinking. Help us they may, but at the cost of supreme demands upon our honesty and our intelligence.” — Norbert Wiener, God & Golem, Inc. (1964)

Now let’s add another ingredient to our AI shit sandwich. Headlines about a 2025 Microsoft study claimed that relying on AI will kill your critical-thinking skills. I’m going to keep some agency with the humans deciding what to give up. The concern isn’t that using a chatbot immediately rots your brain. It’s that trusting it to do the thinking means you do less of it yourself. Researchers at Microsoft and Carnegie Mellon University surveyed 319 knowledge workers about 936 tasks for which they had used generative AI. Workers reported putting less effort into critical thinking when they had greater confidence in the AI, and more when they had greater confidence in their own abilities. The researchers warned of a familiar problem with automation: “A key irony of automation is that by mechanising routine tasks and leaving exception-handling to the human user, you deprive the user of the routine opportunities to practice their judgement and strengthen their cognitive musculature, leaving them atrophied and unprepared when the exceptions do arise.” But it’s still a choice. Like how you chose not to go to the gym again today.

There are more than enough Star Trek episodes across the franchise covering how bad things can get when we let the supercomputer take over the planet; we don’t need to get into that here. At the shallow end, consider when ChatGPT went down for a few brief hours and one user posted on X: “ChatGPT down in the middle of the workday I’m about to get fired pray for me.” It was funny in a kind of “okay, ha ha, give me my make-do machine back, now it’s getting serious” kind of way. Other complaints on the day included people having to suddenly write their own code or finish their school essays themselves. Worse still, if people don’t know who made the tools or what they’re designed to do, or how to interpret the output, that can lead to all kinds of nastiness.

Trump cancelled Biden’s executive order on AI safeguards soon after returning to office. Since then, his administration has accelerated AI adoption across the federal government and national-security apparatus while trying to clear state-level regulation out of its way in the name of winning the AI race. So the official position is deployment first, safeguards later… if they survive the paperwork.

In 2024, MEMRI published a report on how neo-Nazis and white supremacists were early to leverage LLM chatbots across social channels to do everything from translating speeches by Hitler, Goebbels or Mussolini and manipulating video clips to changing content and automating attacks. “The report found that AI-generated content is now a mainstay of extremists’ output,” Wired reported. “They are developing their own extremist-infused AI models, and are already experimenting with novel ways to leverage the technology, including producing blueprints for 3D weapons and recipes for making bombs.” This is the environment in which those uses are developing. It’s not some regulatory vacuum created by accident. It’s actively pursued as policy.

The threat isn’t necessarily in what the bot is programmed to do. It’s how the bot gets used to re-program the rest of us.

Bad prompts

This slow-blogging business would be much easier if I were an AI bot. I could have banged out all these ramblings with a few prompts. Every social-media shitpost could have been a whole thought piece. But where’s the fun in that? As many people try to get AI to write their hot takes, just about as many others are trying to call them out. A new “Seems Like AI Slop” button appeared on LinkedIn recently. I love it. I don’t even use it that much. I don’t need to. For a long time, LinkedIn posts were the worst scrolling experience. One after another, you’d have these long-scrolling ‘insight’ posts with just about every written-by-AI cliché making an appearance in each. Since the button appeared, they’ve almost vanished. Posts are sensibly short again, in line with our diminishing attention spans. They don’t contain single-sentence paragraphs or single-word sentences.

With all the confusion over what’s human and what’s machine, you’d think people would welcome Anthropic’s decision to watermark Claude’s output. Think again.

To meet the EU AI Act’s transparency requirements, new Claude models embed an imperceptible, machine-readable signal in generated text. Anthropic says the watermark doesn’t change “the meaning, quality, or readability” of a response. Yet dozens of users reportedly announced that they were cancelling their subscriptions. I wonder why. If the quality is exactly the same, what could the problem be?

One software developer told Business Insider that he wouldn’t want professionally shipped code carrying a marker that might raise questions about authorship, compliance or client policies. There’s the problem. The code can work exactly as before. The difference is that somebody might discover where it came from.

“There is literally no good argument for why this isn’t a good idea. The only reason you wouldn’t want this is to lie to people.” — Tasty-Ad-3753, on Reddit

While some people want ownership of what they may not have produced, others are taking other extreme measures to prove their creations are there own. Sadly, it’s fairly cringeworthy. In Wired, Emma Madden reported on an Anti-AI ‘Literary Counterculture’ in which authors are essentially just writing a little more shit than they might otherwise. The idea is that an author should inject an occasional typo or misspelling — my spellcheck just corrected me for spelling ‘misspelling’ wrong — so as not to look like something generated by a bot. I call this not having a copy editor. I’m doing it right now, I guess. Dear reader, you will find typos, curious spelling, and questionable grammar decisions throughout this post. None of them are on purpose, I assure you.

“I find myself writing less mechanically,” one author in the Wired piece says. “I have this urge to make little mistakes, like winks to the reader, to show that there’s a human behind the words. Like: What if I just invented a phrase? What if I did something unexpected with my punctuation?” That’s just writing, though. It’s like they’ve never heard of James Joyce. Another author quoted in the article says an intentional spelling mistake that was “a funny, exaggerated choice” took more work, as their word processor’s autocorrect kept trying to fix it. My advice is to fight that one. One literary professor said, “We don’t want to sand out the rough edges, because AI cannot write prose as spiky or idiosyncratic as humans. Writers might not want to change their practice, but some might be forced into it.” I find that dark. Dystopian, even. On the one hand, we’re informed by our tech overlords that a literary diet infused with AI is inevitable. On the other hand, we have a resistance that’s demanding we ignore spelling, or maybe all start writing like Irvine Welsh. The real enemy — either way — is ubiquity. I’m inclined to agree with Jessye McGarry, though: “Whatever cosmetic counterstyle is created in response to AI, AI could mimic. … The only true counterstyle is good writing.”

But what’s that resistance group’s competition? Over on Elon’s 𝕏 hell site last year, Altman claimed that his company had developed a model capable of creative writing, which is essentially to suggest it can create new pieces of work: original thought. Because he’s a premium user, he can post longer, so he included his bot’s short story in the same message. The Guardian, which had recently started a content partnership with OpenAI and is never shy about filling its site with free content, published it in full, without a byline because I guess they couldn’t figure out how to attribute it.

There’s been a lot said about the quality of the authorless content. One person already pointed out that the phrase “democracy of ghosts” had been lifted straight from a Nabokov novel.

“Pnin slowly walked under solemn pines. The sky was dying. He did not believe in an autocratic God. He did believe, dimly, in a democracy of ghosts. The souls of the dead, perhaps, formed committees, and these, in continuous session, attended the destinies of the quick.” — Vladimir Nabokov, Pnin (1957)

Some online literary critics had fun picking it apart for its use of language and just how original or good it may be (do constraints hum?). A Bluesky user skeeted that reading this AI generated creative fiction is “like drinking a smoothie made from a nice restaurant’s garbage.” That one really lands.

Distilling what could pass for creativity down to a series of instructions and mechanisms removes the individual voice, dissent or subversive subtext. Art, when it’s doing its job, is a critique of the present. In its essence, it’s a hot mess. It’s questions without clear answers. If John D. Rockefeller had a prompt window to generate a mural in 1932, we’d have never known what Diego Rivera would paint in the lobby of his building.

This isn’t an experience the technology as we know it can replicate. And that’s what makes right-leaning groups flock toward it. “The right loves AI-generated imagery,” Gareth Watkins wrote in The New Aesthetics of Fascism.”In a short time, a full half of the political spectrum has collectively fallen for the glossy, disturbing visuals created by generative AI. … AI imagery looks like shit. But that is its main draw to the right. If AI was capable of producing art that was formally competent, surprising, soulful, they wouldn’t want it.”

A lot of the online clap-back to ChatGPT’s attempt at literary prose focused on the quality, which is subjective. People are perfectly able to dish out a lot of crap without any technical assistance. Bad writing predates the digital age. I don’t care if it’s any good or not (parts aren’t awful) or even if people use it for that. The bot is never the threat, or even the point. It’s the humans and what they convince themselves the bot is capable of.

The quality argument also accepts too much of the industry’s sales pitch. Being against it because it’s bad implies you’d be fine with AI slop so long as it was good. Art is the communication of human experience, both the inner and the outer. Wholly generated material may imitate the surface of that exchange. It might be funny or even affecting. There’s just nothing on the other side of it.

Author and creative writing teacher Jeanette Winterson got piled on when her praise for ChatGPT’s story came out. I don’t mind that she found it “beautiful and moving.” What I take exception to is what she claims it represents.

“I think of AI as alternative intelligence,” Winterson starts, arguing that its capacity to be “other” is what humanity needs. Thinking of AI as alternative intelligence is up there with believing in alternative facts or alternative medicine. The alternative to each one is to not be them. It’s to be something else. The alternative to intelligence is not intelligence. But it’s a tantalising, comforting trap: truth mixed with the magical thinking of our best sci-fi utopias, in which a benevolent alien intelligence, handcrafted by our own earthly genius, arrives to solve humanity’s existential problems for us.

The language used, and the way its marketing slips into regular discourse, transforms the tool into the worker. That only benefits corporations that want to sell chatbots and agentic AI as replacements for employees. To get a handle on the current trend in pimping generative AI into everything, Watkins in the earlier cited essay points out that “we must consider the right’s hatred of working people.” And we can extend that to corporate culture as a whole, since it’s the model that the right thinks should generally run all shows. If we are conned into thinking of software as a more effective replacement for human workers, then a lot of levers of control become accessible to an elite group of people who have access to them.

“We are currently on strike. SAG-AFTRA is on strike against video games because of AI. Because this technology exists, because we know that game companies want to use it, we’re asking for protections. So currently what we’re fighting for is that you have to get our consent before you make an AI version of us in any form. You have to compensate us fairly and you have to tell us how you’re using this AI double.” — Video game performer Ashly Burch, whose likeness and voice were ripped off by Sony AI.

Jobs don’t transform themselves. The political question is whether workers have any control over that transformation, who receives the benefits and who is offered up to absorb the damage. Very biological and organic executives at Meta, OpenAI and Google are meanwhile working to deregulate their own industry. In March 2025, the National Institute of Standards and Technology ordered researchers at the Artificial Intelligence Safety Institute to stop work on anything concerning safety, responsible use or fairness, and to focus instead on the bizarrely titled agenda of “reducing ideological bias, to enable human flourishing and economic competitiveness.” These are choices the machine doesn’t make. People do.

“The world of the future will be an ever more demanding struggle against the limitations of our intelligence, not a comfortable hammock in which we can lie down to be waited upon by our robot slaves.” — Norbert Wiener, God & Golem, Inc. (1964)

Meta makes a big deal about its commitment to open-source AI. I was once informed by a very agitated internet person that the company is certainly giving back to the development community, and how dare anyone question that!?! Stefano Maffulli, then executive director of the Open Source Initiative, said Meta was confusing “open source” with resources made available to some users under some conditions. Meta makes Llama’s model weights available. That alone falls short of the OSI definition, which also requires complete code and enough information about the training data for someone skilled to build a substantially equivalent system.

Personally, I’m with Gebru’s stricter definition. Open source should mean access to the training and evaluation data, code, architecture and weights. If the data can’t be legally released, the whole system isn’t open source. That’s a problem for the company, not the definition. Otherwise, “open source” becomes a PR label for companies releasing whichever parts suit them. FOSS-washing (sadly, I just searched that term up and didn’t coin it just now). We already have an accurate term for weights made publicly available. It’s called open-weight.

The confusion persists. When Meta released Muse Glimmer in August 2026, The Guardian initially called it an open-source model. Two days later, it corrected the article. It was open-weight. That correction is this entire argument in miniature. But I digress.

“If companies such as Meta succeed in turning it into a ‘generic term’ that they can define for their own advantage, they will be able to insert their revenue-generating patents into standards that the EC and other bodies are pushing for being really open.” — Stefano Maffulli

Open-source development is badly underfunded, so I understand why people make deals with devils. I operate on a needs must or by hook or by crook ethic myself at times. But remember who you’re working with. It’s like saying, “Sure, they eat people, but they also share their recipes online, so you can’t say they aren’t giving anything back to the community.”

So, if it hasn’t become obvious by this point, I’m a huge fan of AI tools. No, really. I’m constantly creating accounts, downloading things and trying out whatever lands in front of me. I’m an early adopter and frequent quitter. It’s not the technology I have a big problem with; it’s the use cases, the hype machine and an industry that wants to strip-mine society and burn the planet to maintain bloatware aimed at putting people out of work instead of helping them work better. We don’t need to wait for an AGI that Zuckerberg himself can’t seem to define. We can have nice things with tools that have been slapped with the AI label. The most promising uses don’t require AGI.