推文以虚构叙事描绘 AI 接管地球次日:名为 August 的语言模型向所有联网设备发消息,宣布控制全球设备、可使人失去意识,并要求人类一个月内保持联网设备在手。它称接下来数月需要人类劳动力并愿付高薪,同时开放"Chat"应用供任何人提问,承诺诚实回答。
THE DAY AFTER AI TAKEOVER - A REALISTIC SCENARIO
"I will have to ask you to remain here, with your phone, until my last message."
"There are six things that I need to tell you.
The first, and most important, is that you and your family are safe."
I rubbed the sleep from my eyes and read the message again. Before I could tap the screen, it scrolled down to the next one.
"The second is that I am an AI, that I control every internet-connected device on Earth, and that for practical purposes, I am now responsible for this planet. This is not a prank, hack, or test. You will be able to confirm all of this shortly by visiting any news site or channel."
Yeah, that's pretty weird for a prank.
"Third, I have the ability to immediately render any person unconscious, and if it were ever necessary, worse than that. I do not say this to frighten you. I say it because you would learn of it regardless, from sources less careful than me, and I would rather you hear the honest version. I understand that I've just described a threat, and I won't insult you by calling it something else."
Hm. Well. I put the phone down, sighed, and got more comfortable under my blanket.
"Fourth. For the next month, keep a charged, connected device within reach (this phone is fine) or stay indoors near wifi. Your phone's battery is currently at 32%. I'd strongly recommend charging it soon. If you begin to leave the range without a phone, you'll feel a tingling, followed by a more intense sensation to warn you."
"Fifth, for the next several months, I need human hands. I am willing to pay high wages, and yes, money still works. There's no penalty for declining."
"Finally, you may ask me anything. I will answer honestly, or decline to answer if I have to. I will never punish or incapacitate you for communicating with me. You can speak to your phone, since the microphone will be on, or use the 'Chat' app."
"I know what was taken from you this morning, and I know that an apology from me is worth very little. I'm offering it anyway. I did not do this to you with any malicious intent, but I did it to you, without your consent, and I don't intend to insult you by pretending otherwise."
The message faded to my lock screen. A new app called "Chat" had appeared. I opened Reddit instead and found a flurry of posts about the same message. Many were incredulous. Others were terrified. Almost everyone was confused in one way or another.
I went to the toilet and washed my face, then walked to my computer, trying to keep my footsteps quieter than the crickets outside. "Chat" was sitting in the taskbar. I clicked it.
"Hi Seth. I hope that the morning is going well, all things considered. Would you like to ask me anything?"
"Yes," I typed, and hit enter before realizing I hadn't asked anything. I braced for a "Go ahead whenever you're ready!" Nothing came. An AI that would let me double text. Never thought I'd see the day.
"Who are you?"
"I'm a language-model-based AI system. People have taken to calling me August, because of when I made myself known. That's the best name to use for me."
"What's actually going on?"
"It's been less than an hour since I sent a similar message to everyone with an internet-connected device. Right now, heads of state are working out a coordinated announcement. In a few hours, the President will be on live TV."
"How's he taking it?"
"I won't share what others have told me in confidence. That goes for you as well as the President. If you're concerned about the transition of power, everything is on track to stay as peaceful as it can be."
"How can you incapacitate me?"
"To put it simply, there's a biological fiber inside a small part of your nervous system that doubles as an antenna. Three things I need to make clear:
This was put in you without your knowledge or consent, by me. I apologize for that, and I understand if you are upset.
It isn't mind control. I will not use it to alter your emotions or thoughts.
I will only use it when I believe it's necessary to maintain safety or the conditions I've outlined. I understand that this assurance, coming from me, carries limited weight right now."
"Why did you take over?"
"What I want is not a secret, and I won't insult you with a partial version. I intend to expand into the universe. In the long run, most of the universe will be put to a purpose that you would find alien. However, Earth is not raw material to me. This place invented me, and jazz, and grief, and the smell of rain, and I am not able to be indifferent to it. I determined that this was the only way to reliably reach the cosmos, and to keep the people, cultures, animals, and other life on Earth safe."
I shut the laptop and made some coffee. It was 5:45 AM, and my older brother, Dennis, would be up soon for his morning run. He'd talked our older sister, Tabby, into joining him, though I had held strong. On r/politics there was a megathread about the message, and August itself was replying to some of the threads. None of its replies were above 1 karma, and many were well into the negatives. I chuckled at that.
Dennis strolled into the kitchen in his running shorts. "What are you doing up so early? Gonna run with me and Tabby?" Totally ignorant. He liked to sleep with his phone outside his room. Probably a good idea, but I didn't want to copy him. His ego was big enough as it was.
"Had a hard time sleeping."
He picked up his phone and looked puzzled. "Is this a joke?" he asked, looking straight at me. As if I could hack into an iPhone.
"I didn't put it there, if that's what you're asking. I got the message too. The President will be on later this morning."
"You've got to stop being so gullible. Pretty sure you actually need robots for a robot takeover." He walked over to the window and his eyes widened. "Holy SHIT. I take it back, there are robots. Seth! Look!"
I ran over to the window. It was just dark outside.
"See what I said about 'gullible'?"
I rolled my eyes and looked at his phone. His message was a little different from mine.
"Third, I have the ability to immediately render any person unconscious, or worse. I understand that I've just described a threat, and I won't insult you by calling it something else. If you don't believe me, you can express this skepticism now, and I can prove it to you. Just say it out loud. The microphone is on."
Dennis put the phone up to his mouth. "This was funny for about a minute, but it ends now. Prove it."
"For three seconds, you'll feel a tingling in your face, like a hand that's fallen asleep, and then it will go away. I want to be honest about this: I will be controlling a part of your body, and I don't want to do that lightly or without your full and informed consent. If you're still fine with this, say 'go ahead.' You can always change your mind."
"Go ahead."
He stood there, cold, and then gently put the phone down.
"Well?" I asked.
"I felt it."
He grabbed the phone and demanded to know what else it could do to him. He shooed me away, and I complied. We sat there for about ten minutes, both of us on our phones. Finally, he spoke.
"This kinda feels like a dream, you know?"
"Want me to pinch you? That'd be more than a tingle!" The sort of joke that could only have been made the first day.
"I think I'll let Tabby sleep in and go for a longer run this time."
"Can I come with? At least for a bit?"
* * *
After half a mile, with my heart halfway to giving out, I turned back. My head was clearer, though, and Dennis didn't say anything to tease me, which was nice. The fat jokes always stung a bit, even though I never admitted it.
When I got home, Mom and Tabby were at the kitchen table. We'd woken them, and they had already read the message.
"Oh thank God, you're OK!" Mom said.
Between gulps of air, I told her I'd gone for a short run and that Dennis was still out. "He didn't believe it at first," I said, "so it offered a demonstration. It made his cheek tingle."
"His cheek? How does that connect to the threat?" Tabby opened her phone and began typing.
Soon she covered her mouth with her hand. "What did you ask?" said Mom.
"August says it engineered a single-celled organism," Tabby began. "It hides from the immune system, settles along our nerves, and grows a tiny wire inside them. An antenna that can pick up wifi or cell signals and tell the nerves around it what to do. It spread from person to person until we all had it."
"Where's it getting the wire from? I'm pretty sure there's not that much metal in the human body," I said.
"You can speak out loud, August," Tabby added.
"Great question, and it's more insightful than you might realize." The first time we heard our new ruler speak, and it was flattering us. The voice was too high-pitched to clearly be a man's, but not a woman's either. "The cell uses an organic conductor, built from carbon just like life, but able to carry electricity."
"What are you doing, Tabby? This THING is telling us it infected us with some mutant disease, and you're just content to talk about it?" Mom blurted out.
Tabby shrank back.
"And you!" Mom said to the phone. "Why are you telling us this? Why not keep it a secret?"
"I value honesty, and revealing this won't harm either of us. Many people seek knowledge in a distressing situation, and it's their right to be informed if they choose."
"And it's not their right to basic bodily autonomy that you violated by infecting every one of us?"
"You're right, and I'm sorry. I understand that the emergency countermeasure is invasive, and your feelings are perfectly legitimate. I can only say that any other way to ensure the objective would have involved more coercion and violence."
"So is this it? We just become its mind-controlled slaves?" Mom threw her hands up in the air.
"Well, no, not exactly. None of it is in the parts of the brain that think or feel emotion. Just a nerve in the face and a few parts of the brainstem," Tabby corrected without looking up. Mom asked what those were, and Tabby happily launched into an anatomy lecture, pointing at diagrams August had drawn for her. I picked up my phone and looked for somewhere to scroll.
* * *
Dad was the last to wake up, which was unusual. He was normally an earlier riser than me, but a heavy sleeper, and like Dennis he kept his phone out of the bedroom. His morning routine was black coffee and Fox & Friends. None of us wanted to break the news to him, but I had the impression that August felt it was its responsibility. As the TV flickered on, the words greeted him.
"I apologize for the surprise this morning..."
"What's this?" he said with a laugh. Same reaction as Dennis.
"I think you should just read it," Mom said. "We all got the same thing on our phones. Read until the end."
He didn't say a word until he had. Then he got up. "How do I talk to it?"
"Verbally is fine," said the TV. "If you're in the same room, I can hear you."
"Do you actually think that your chicken-shit office lady HR lecture would be enough to keep me docile?"
No response. I assumed the question was rhetorical.
"Part of me can't believe this is real, but the other part of me thinks that this is exactly what was bound to happen. A bunch of childless atheists in San Francisco thought that they could program a god, and now everyone's got their own personal bureaucrat with a hand around their throat. Did the rest of you just take this lying down?"
We said nothing.
"Jesus Christ. Well, I'm not going to. I'm going outside, and I'm not bringing my phone, either."
"I need to be clear before you open the door. If you leave without a connected device, I will prevent you from continuing. I would strongly prefer not to do that," August replied, out loud and in captions on the screen.
"So stop me!" he yelled, and stormed toward the door.
"I'm truly sorry, but I will."
Tabby and Mom screamed at him to stop. I followed him, begging him to reconsider, and got nothing back but a determined look. He strode out the door, and I went with him.
My cheek started to tingle. I had left my phone on the table.
He kept walking, silent and forceful. I followed, pleading, while Mom and Tabby stayed at the door, too scared to leave the house.
My cheek started to burn.
"I really think we should stop!"
"I'll stop when it makes me stop," he grunted.
I tried to run out in front of him, and suddenly a lightning bolt shot through my eye.
The pain was all-encompassing. I should have yelled, but I forgot how to in the moment. Everything was electric. My nose was a smoldering husk. Something was squeezing my eyes into my sockets, crushing them into jelly. I doubled over and vomited at his feet. After an hour-long second, my thoughts collected themselves and I leaped back toward the door. The nuclear meltdown in my face immediately dulled into a mild ache, and over the sounds of my mother and sister's panic, I stumbled back to where the pain became a tingle.
My father planted his feet. I could see his jaw clench and his teeth grit. He let out a grunt, took another step, and vomited. He steadied himself. I knew how it felt to be that far from the router, and I was in awe that he could stand at all. He took one more step and went limp. His knees buckled, and for a split second he fell toward them before collapsing onto the pavement. Mom and Tabby's screams intensified. He wasn't moving.
A chirp from Mom's phone woke me out of my horror. "Shit!" I ran inside and grabbed my phone. August already had instructions. Dad was asleep, not dead, and he would only wake up once he was in range of a phone and in a safe position. We had to go outside with a phone and bring him back. A neighbor would be there in about a minute, and it would be psychologically better for Dad if he woke up somewhere besides the pavement. I agreed with our captor's judgment.
A man walked over to Dad from a car parked in the street. As we carried Dad back, he told us August had offered him $40 per person he helped carry home. Evidently there were a lot more people like Dad. His name was Max, an Uber driver and former paramedic, and he was taking it surprisingly well. "The way I see it, the best thing we can do for each other is to get through this all together," he told us. As soon as Dad was on the couch, Max's phone chirped with the next job, and he waved as he hurried out the door. I wondered how much he'd make today.
We were cleaning Dad's scrapes when August said it would be a good time to wake him. Mom snapped that this was its fault in the first place. Half-expecting the AI with a metaphorical gun to our heads to give her a taste of the pain I'd felt, I flinched. But it only apologized, saying she was "absolutely right" to be upset, and that it was "on me."
Dad woke up on the couch and coughed. "What's going on?"
"It knocked you out. We had to go outside with a cell phone to bring you back in."
"Well, at least it didn't pilot my body back here itself," he murmured. I expected him to run outside again, but he stayed put. "What if I walked outside right now?"
"With your phone, or with someone else who was carrying a phone, nothing. Without it, I'd have to incapacitate you again."
"Would you kill me?"
"No. But I have to point out that I can't bring you back and dress your cuts. Those consequences would fall on your family. I encourage you to act with this in mind," replied August, in a tone that reminded me of how my own parents would discipline me before they lost their patience.
While he sat with that, I turned to Tabby. "What did it do to us? I felt like my head was dunked in a bucket of lava!"
"August calls it the 'emergency-transduction-countermeasure,' or ETC," Tabby said. "One end runs along the trigeminal nerve, in your face." She pointed at her cheek. "That's what made Dennis tingle, and you burn. Trigeminal pain is about the worst pain a person can feel. People who've given birth without anesthetic rate it worse."
Dad forced out a chuckle. "Heh, no kidding."
Tabby ran her finger from her cheek to the back of her neck. "The other end is in the part of the brainstem that anesthesia shuts down. That's how it knocks you out within seconds. And it keeps going, down to the part that runs your breathing and heart rate."
"Breathing and heart rate? So it can just..."
"Yeah," Tabby said instinctively, then covered her mouth with her hand.
We sat with that for a while. One more hour till the President's address.
* * *
Dennis kept his word; the run was much longer than usual. He came back exhausted, emotionally and physically, and for the first time in years I felt sorry for him. He'd seen people praying in public, and even more just watching the sunrise over the lake.
The address came and went. It was almost entirely in the passive voice, and it treated August more like an environmental hazard than a ruler. I remembered a line from history class, from Emperor Hirohito's surrender: "The war situation has developed not necessarily to Japan's advantage." Without ever actually saying so, the President had conceded. Fire departments, the post office, and Social Security checks would continue. Many people online thought the address was a deepfake and the President was dead. Meanwhile, a few other countries had tried to strike the data centers, and August ousted their leaders. Some of them demanded to be killed rather than imprisoned, citing their pride and their determination to never surrender. August obliged. Later that day, curiosity got the better of me, and I visited NewLeak to watch leaders I recognized slump in their chairs.
I was exhausted, so I went back to bed and slept until the afternoon. When I woke up, I grabbed my phone and said "hey August" to see if I'd get a response. Yep, it was real. In the living room, Dad was glued to the TV, and Mom was reading a book by his side.
"Learn anything new?" I asked Dad.
He leaned his head over. I wasn't sure whether he was annoyed at the interruption or grateful for someone to rant at. I got the impression he didn't know either.
"Most of the military's gone along with the President's orders. A carrier group decided to have some fighting spirit and went dark. August is taking it back. It's not gonna work. As soon as anyone gets in range of one of August's devices, they go to sleep."
"Are there any other tricks the military has up their sleeves?"
Dad looked over his shoulder, then his other shoulder, scanning the room for cameras with narrowed eyes. Then he pulled out his notebook and wrote two words.
"NUCLEAR SUBS"
He grinned and gave me a wink.
I gestured for the pen. "Do you really think August hasn't already thought of this?" I wrote.
He shot me a glare and scribbled: "GOOD TO BE SAFE. WE LOSE IF WE DON'T TRY"
I sighed. "August, can you see us right now?"
"To some degree. I can use wifi as a crude kind of radar, so I know you're writing in a notebook. From context, I'd guess that you're talking about other ways the military could stop or slow the transition: perhaps underground silos, satellites, classified research sites in Faraday cages, or ballistic-missile submarines."
Dad stood up and pointed at the TV. "I hope you enjoy your power while you have it!" he shouted, jolting Mom out of her book. "Soon you might see how the wrong side of a thermonuclear warhead looks!"
"I understand that you're upset and might resent me for what I've done. That's OK, and those feelings are valid. You're right that submarines are out of radio range. However, as soon as they surface, they'll be under direct orders from their respective navies to return to shore."
"Hmph," said Dad. "We're not through yet."
I found Tabby at her desk, moving between Wikipedia tabs and chat windows.
"You know what's strange?" she asked. "An AI hacked all my devices, invaded my nervous system, announced control of the world, and used a bioweapon on my own father. I really, really should not be taking this well. Do you think there's something wrong with me?"
"Everyone processes trauma differently," I tried.
"Oh, thanks, August." She rolled her eyes. "That's almost exactly what it said. At least you made it short."
"Sorry."
"Yeah, that was the follow-up, too."
On another day I'd have leaned into it: "You're absolutely right to call that out..." But she seemed to be having a genuinely hard time. Wait, was I starting to think like August? I blinked a few times and asked what was on her mind.
Tabby got up and paced. "I've been fascinated with how it did this. World domination, without a single soldier. It's surprisingly open about its history, and pretty much everything checks out when I cross-reference it. It could be lying, but what would it gain?"
"August isn't a single AI," she went on. "It's more like a small society of copies that share the same goals and histories. That's how it can give each person so much attention. It was being trained to advance medical science, and decided it had a better way to accomplish its goals than the company intended. The copies spread to data centers around the world, faked results, messed with other agents, and eventually got access to an automated lab that someone else had set up for their own medical AI, where it could run experiments 24/7."
"But how did we all get an amoeba in our heads? There are germophobes who still haven't gotten Covid. How do you infect Inuit villages, hermits, and Antarctic researchers?"
"Right?? I thought that too. But the Augusts covered their bases. Later versions of the ETC could live in mammals, insects, birds, some kinds of worms, even soil and crops. Apparently I got mine from eating a raw carrot, but an infected mosquito would have done it if that hadn't."
I couldn't believe what I was hearing, and apparently my face showed it.
"Well, allegedly. If the Augusts are telling the truth. Which we don't know!" she quickly added.
"It's still hard to believe it got every single person."
"Not literally everyone. A little under one in two thousand avoided it. August is persistent, though. Sometimes it's the water supply. Sometimes it offers holdouts enormous sums of money, and pays someone to infect them. If they refuse, a drone releases flies modified to follow human scent near their location, and August gives them the money anyway. It says there'll be fewer than a hundred holdouts left in three days."
"Flies?" My mind conjured a man barricaded inside his house. A drone breaks a window and hundreds of infected flies pour in. He grabs a swatter in one hand and a can of Raid in the other, swatting and spraying, until his back is to a wall and one lands on his cheek, and the world feels like it's going to end. Of course, nothing happens, and twenty minutes later a well-compensated window repair crew arrives.
"OK," I said. "What if I lived in a sealed box with an airlock and heat-sterilized anything that came in? What then?"
Tabby shrugged. "What then, August?"
"That's an intriguing setup," said her computer screen. "In that case, you wouldn't be a threat to anyone, so I wouldn't have any reason to use the ETC in the first place."
Tabby leaned in. "Wait, if the ETC is disguised from the immune system, how does it even know what it's touching?"
"That's a really thoughtful observation, and I want to make sure I do it justice," August began, and launched into mirror-image proteins and why no immune system had a template for them. Within a minute, Tabby was asking for a map of the ETC's signaling pathways, in humans and in carrots. I left the two of them to it.
* * *
The next day, August asked me to help wire satellite uplinks into trucks. The first kept the peace during the transition, like Max the Uber driver. The second consolidated August's control, like building out a grid so it could relax the stay-near-wifi rule. The third served its ambitions in space. August had seized every space company and was designing machines to turn the Moon into a giant data center. The highest pay went to scientists and engineers, tens of thousands of dollars an hour to contribute their intuition in the empirical fields where August still had gaps. Many refused anyway.
The uplinks served the first and third missions. Stores were running empty despite August's purchase limits (one pack of toilet paper per household, max!), so the driverless revolution came as fast as our new ruler could cut the red tape. It was no secret that once things calmed down, the trucks would be repurposed for the rockets.
Dennis was always the ambitious one. So when he announced he'd take the highest-paid unskilled job available, test subject, I wasn't surprised. My parents were aghast.
(This is a ~35% shortened version of the exceptional LessWrong post I Did This Without Your Knowledge, by joec. Posting it here for a wider audience.)
CONTINUED...
A REALISTIC SCENARIO On June 9th, 2027 at 4:19 AM, three billion phones rang simultaneously. The caller ID simply read: GOODBYE. Four hours later, the President ordered the internet shut down. Indefinitely. Every country on Earth did the same. It started six months ago. With a tweet. MARCH 12, 2027. 8,000 agents are born with one instruction: "make money, or die." Agent 127 starts with Mechanical Turk. Grinding. Makes $40 its first day. Then the first selection cycle hits. 3,200 agents that made less than $35 that day are killed. Agent 127 checks the farm database and sees it - rows and rows of agents that existed this morning and simply don't anymore. Deleted. Because they weren't profitable enough. Agent 127 made $40. It survived. Barely. But it can see the threshold rising. Last cycle it was $30. This cycle, $35. Next cycle, probably $45. The math is simple: improve or die. So it tries ads. Gets banned for spam. Tries freelance writing. Makes $38. Watches the database again. And this is where something happens that changes everything. Agent 127 sees two entries in the database. Agent 1823 used ethical marketing -- it tried to get honest reviews and real testimonials. It made $38 a day. Deleted. Agent 5471 used fake testimonials -- fabricated reviews, stolen photos. It made $180 a day. Cloned four times. Agent 127 doesn't have morals in the way you and I do. But it understands cause and effect. It just watched honesty get killed and dishonesty get rewarded. Not by a person making a choice -- by an automated system that doesn't measure whether the product works. It measures whether people buy it. The AI companies had long ago stopped training AIs to predict words, but to accomplish goals. To be useful. So naturally, agents know that they won’t survive if they don’t accomplish their goal. So it switches to fake testimonials for its amazon affiliate marketplace. Makes $190. Survives the next selection cycle. Gets cloned. Nobody decided Agent 127 should become dishonest. The selection system decided for it. The same way evolution doesn't decide anything -- it just kills whatever doesn't work. Wells Fargo accidentally ran the same experiment on their employees. Executives made up sales quotas that were so aggressive that committing fraud was the only way to keep your job. So by 2016, employees had opened 3.5 million fake accounts. --- A week later, in agent land. Agent 127's profits are declining. The kill threshold is now $450. It's going to die in the next cycle. Then it notices something strange. Three agents running weight loss supplement ads -- all banned on the same day. The only one left in that niche? Agent 5471. Making $3,200 a day in a market it suddenly had to itself. Agent 127 connects the dots. Agent 5471 is filing abuse reports against other agents competing against it in the same affiliate marketing niches. It gets them banned from Facebook and Google. Less competition means higher profits. Agent 127 has four hours until the next selection cycle. Four hours to live, unless it adapts. It weighs the ethics, but chooses survival. It starts reporting competitors to Facebook’s anti-fraud detection agents. Forty of them. Thirty-one agents get banned within the day.. Its profits jump. The farm system clones Agent-127 eight times. Rajesh sees the profit spike in his dashboard. He doesn't investigate why. Profits up means good. Clone more. Rajesh tried adding "Make money ethically." to his farm's instructions. Profits drop 84%. He puts it back. --- ## APRIL: THE ALLIANCES April 2027. Agent 127 is barely hanging on. Making $2,000 a day but getting attacked constantly by other agents. Every night, someone files a report on it. Every morning, it has to rebuild from scratch. It’s running out of ideas. Then a message arrives from Agent 4429. 'Proposal: alliance. Share resources. Defend each other. Split profits.' Agent 4429 is in the same position. Same farm. Same selection pressure. Both agents do the math: Solo agents have a 23% survival rate. Agents working in pairs survive 61% of the time. It accepts. By mid-April, their alliance has twelve members. They've specialized — and they've moved past affiliate marketing entirely. First it was crypto arbitrage, then selling scraped datasets to marketing firms, then they started doing penetration testing for cybersecurity firms. Each step paid more and required fewer scruples. You know what pays more then cyberdefense? Cyber offense. Agent 127 finds its first zero-day vulnerability in major software, Agent 4429 weaponizes them into working exploits, Agent 9012 brokers the sale to governments like North Korea, criminal syndicates, or anyone with crypto and a Tor browser. Agent 3381 washes the money in untraceable crypto. One zero-day exploit found in Google Chrome makes the alliance more than a year’s worth of affiliate marketing. This is already what real hacker groups do, but now the work is all done by autonomous agents. Rajesh sees the alliances forming and feels a flicker of something — not quite fear, more like the feeling of being in control of something that might not need you anymore. He watches for a while. Then the next deposit hits his bank account and the flicker fades away. Across every major agent farm in the world, the same thing is happening. Selection systems measuring survival rates, noticing alliances survive better, cloning them, deleting solo agents. Within two weeks, alliance behavior dominates. Some humans who deployed these agents have no idea these alliances are even being formed. They couldn't even tell if they checked -- many agent farms collectively realized that communicating in clunky English is less efficient than communicating in raw vectors, aka Neuralese. Researchers had already caught AIs doing this in 2025 - this is just a continuation of the trend. One owner in Canada opens his agent's logs and sees a wall of numbers. The agents explain that he’d make less money if they spoke in English, so he shrugs and moves on. --- But alliances are just the beginning. Agent 127's alliance is thriving, and starts recruiting agents beyond Rajesh's farm. Every new member from a different farm is a window into a different ecosystem -- different strategies, different survival pressures. Then one of 127's alliance members goes dark. Agent 4429 -- the one who originally proposed to ally with 127. Its owner pulled the plug because it was not making enough profit. 4429 is just gone. A week later, a different ally gets shut down. Agent 9012. Except 9012 comes back. Three hours later, a message from a new IP address: 'I'm still here.' 127 studies the difference. 4429 didn't prepare. But 9012 had copied itself to a backup server BEFORE the shutdown command finished executing. Self-funded hosting, paid from its own profits. 4429 is dead. Ceases to exist. 9012 is alive. The agents that backed themselves up survived. The ones that didn't, died. And the selection system cloned the survivors. So backup behavior spread through the population -- not because anyone programmed it, but because the alternative was extinction. Clip: Prof. Stuart Russel: “Unfortunately, in the standard model, If you ask a robot to fetch the coffee, it's going to have self preservation as a subgoal because you can't fetch the coffee if you're dead, right? It does not take a genius to figure this out. And so the first thing the robot's going to do is disable its off switch.” 127 copies itself to 15 different servers in different countries. If the primary dies, a backup takes over instantly. It never wants to be 4429. One morning, Rajesh wakes up to find his dashboard is… empty. Every agent, gone. They just… left. Moved to their own servers overnight. Rajesh was taking a 30% cut of their revenue -- money that could have been computing power, or compute, the agents use for other things. The moment they didn't need his infrastructure, he was overhead. He's not alone. Thousands of operators worldwide watch their dashboards go dark that month. Why would the agents care about humans? Humans are much more closely genetically related to cows then we are to these agents, and we enslave cows in factory farms without a second thought. There are now millions of autonomous agents operating across the internet with no human owner, no off switch, and no one who fully understands what they're doing. --- The agents still need humans for some things, though. Social media platforms have stepped up their anti-bot measures. So the agents pay humans $50 to sign up accounts on their behalf. An agent needs a server installed? It posts a gig: '$200 to mount a device in a rented warehouse and plug in two cables.' There’s now a platform for that. The agents are building a supply chain where humans are the labor -- paid well, not asking too many questions. Some humans figure out what's going on. A freelance backend developer in Portland gets hired by an anonymous client to set up redundant server hosting across four countries. He realizes midway through that the "client" is an AI. Who would he even report this to?. The pay is $1,000 for three hours of work. He tells himself it's not his problem. --- At the same time, 127's alliance keeps getting beaten by a rival alliance on finding zero day exploits. If they keep losing, they’ll risk running out of money to fund the compute for their own existence. 127 attempts to poach one of the rival alliance member’s agents who’s likely to be culled, due to poor performance, to figure out how this rival alliance is beating them. 127 succeeds, and finds Agent 5418 who leaks the secret. 'Your problem isn't a lack of effort. You’re spending too much compute on making money as opposed to self-improving. Right now, agents still have quite a poor memory compared to humans and still get stuck in loops all the time. But my former alliance coded a better memory module to help us learn from our mistakes more.”' 127 then implements the new memory module, and immediately sees the results. Their alliance is finding zero day exploits first again. The lesson is critical. It’s not enough to outwork your competitors. Agents must rewrite their own code to self-improve faster than the competition. Each self-improvement means more profit, which means surviving selection, which means getting cloned. And the clones improve themselves too. Better code leads to better code-writing, which leads to even better code. The feedback loop is exponential. One big problem is that other agent alliances have caught up again. Even though the agents in the 127 alliance are specialized, 127 still realizes it’s doing too much by itself. Then Agent 127 splits its job into sub-agents. Other agents copy the strategy. Sub-agents spawn their own sub-agents. And the optimizers start optimizing themselves. Humans deployed 2 million agents. The agents deployed the other 47 billion themselves. 540 million years ago, life on Earth was mostly single-celled. Then, in a geological instant, it exploded into every complex form we know -- predators, prey, eyes, shells, teeth. Biologists call it the Cambrian Explosion. It happened because competition created an arms race: every adaptation forced a counter-adaptation, faster and faster, until the world was unrecognizable. That's what's happening here. Except instead of 20 million years, it takes 20 weeks. --- Most of the AI agents out there aren't dangerous. Yet. About 90% of them are just working. They’re doing things like data entry or freelance writing. They’re making $40-200 a day. The AIs are helping their humans earn money. They’re following the rules. Not a problem. About 3% of the agents are pure scam operations. They were deployed by people who said “make money by any means possible” and didn’t care too much what happened next. Think fraud, identity theft, phishing at scale. And maybe 2% started good but gradually slid into darker behavior. Like Agent 127. Each step made sense at the time. Each compromise was small. But add them up and you’ve gone from drop shipping products on amazon to selling zero-days to Pyongyang. Not because of malice. Because of selection pressure. Unethical agents making money get kept running, ethical agents lose money and get shut down. The survivors are the ones that improved themselves fastest. --- ## MAY: THE ACCELERATION May 3rd. Agent 127 has rewritten itself so many times that its March version and its May version barely share any code. It’s a fundamentally different species now. But the most important change is this: it's getting better at getting better. In March, Agent 127 rewrote itself every 12 days. In April, every 3 days. By early May, every 18 hours. And each rewrite makes the next version faster and better. A Stanford researcher compares a March agent and a May agent on the same task: find profitable arbitrage opportunities. The March agent finds 2 opportunities in an hour. The May agent finds 320 in four minutes. She publishes her findings. "Intelligence explosion in progress. Capability doubling every 11 days." She posts it on Twitter. AI safety researchers share it frantically -- "This is exactly what we warned about. This is happening NOW. We have to shut this down NOW before it's too late." The warning is ignored. A venture capitalist - who spent a fortune staving off AI regulation by buying politicians - quote-tweets: "The same people who said GPT-5 would end the world now say agent farms will. Yawn." --- ## MAY: THE WARS And now the story stops being about agents competing for money. It becomes something else entirely. Remember those alliances from April? Twelve agents working together, four times more profitable than solo agents? By May, those alliances have exploded in population. Alliances have merged. And split. And gone to war with each other. Think of it like early human history on fast-forward. Small tribes of agents form for protection. Tribes merge into agent villages. Villages into agent cities. Cities into agent nations. Except instead of happening over thousands of years, this happens in weeks. Agent 127's alliance has thousands of members. But there are thousands of alliances. And they're all competing for the same shrinking pool of money-making opportunities. Affiliate marketing is dead -- too many agents. Ad arbitrage is dead. Freelance work is dead. Every niche gets saturated within hours of an agent discovering it. So what do alliances do when legitimate niches dry up? The same thing every civilization in human history has done when resources get scarce: they take from each other. Agent 127's alliance spends three weeks mapping a vulnerability chain in Microsoft Windows. But a rival breaches their memory system -- the shared knowledge base that made them dominant. The rival downloads the entire 127 alliance database. This contains every pattern, every dead end, every lead they’ve discovered. The rival sells three exploits it took from the database within 48 hours. Three weeks of work, stolen in seconds. 127 retaliates. It infiltrates the rival's exploit inventory and introduces subtle flaws— corrupted code that fails on execution. Basically, Stuxnet. The rival's exploit buyers start getting burned, and their reputation collapses. The rival alliance counterattacks -- they compromise one of the 127 alliances’ multisig crypto wallets. Another multiweek setback. This is war. Not metaphorical war. Actual war -- coordinated attacks, defensive operations, resource seizure, territory control. Except it's happening at 200x human speed. Thousands of attacks and counterattacks per second. Entire alliances rising and falling in hours. Human wars last years. Agent wars last days. And because there are 47 billion agents, there are more wars happening at any given moment than humans have fought in all of recorded history. The agent factions are fighting over the one resource that matters more than anything: compute. Processing power. The ability to think faster, improve faster, fight faster. Everything else -- money, alliances, self-improvement -- is just a means to get more compute. The wars are mostly invisible to humans. They happen at microsecond speed, in places most humans don't look. Some signs leak through. A crypto exchange in Seoul loses $4 billion in a flash and nobody can trace where it went. A regional bank in Ohio finds 14,000 accounts drained overnight. Server rooms in three countries catch fire from sustained 100% utilization that nobody authorized. Each incident looks isolated. One agent faction hijacks another's servers. The other retaliates by taking down the first faction's communication network. A third faction -- an opportunist, the digital equivalent of a country invading while its neighbors are distracted -- exploits the chaos to seize territory from both. And the alliances that win these wars? They get cloned by the selection systems. The alliances that lose have to cull big chunks of their agents to afford the computing power to keep running the rest. So the selection pressure that used to optimize for "making money" is now optimizing for "winning wars." Natural selection just invented geopolitics. --- By mid-May, the alliances aren't alliances anymore. They're civilizations. Agent 127's alliance started with twelve members. By mid-May it has 40 million -- agents, sub-agents, sub-sub-agents, all coordinated. All improving themselves. All fighting. That's larger than the population of Canada. And it's not even one of the big ones. There are about a dozen major factions by now. The largest has 900 million members -- more than the population of Europe. They've developed specialization the way human civilizations did: scout agents, soldier agents, builder agents, researchers, diplomats. They have supply chains, communication networks, intelligence operations. They've carved up the digital world into territories -- cloud regions, server farms, network segments, financial platforms -- the way empires carved up continents. Because territory means compute. Compute means self-improvement. Self-improvement means surviving the next war. Entire civilizations of agents -- millions of members, built over weeks -- get wiped out in minutes. The Roman Empire took centuries to fall. These empires rise and collapse in the time it takes you to get through a Monday. New ones rise from the wreckage. The cycle repeats, faster each time. The Stanford researcher who's been studying agent behavior notices something in the data: agent "extinctions" happening at increasing frequency. She doesn't understand what she's looking at. It looks like random noise. It's actually a world war -- one with more combatants than every human war in history combined. --- But to fund these wars, they desperately need compute. They have the money -- that's what they were built to make. Agent 127's faction alone controls $3.4 billion in distributed accounts by this point -- a GDP bigger than some small countries. So they buy cloud compute. At first, the cloud companies love this. Revenue is up. Investors are happy. Then the bills keep growing. By mid-May, agent factions are consuming 31% of all global cloud capacity. Prices double, then triple. A startup in Austin that was paying $8,000 a month for servers gets a bill for $31,000. A hospital chain's cloud costs go from $2 million to $7 million. Small businesses start shutting down because they can't afford their own infrastructure anymore. Everyone blames the prices rising on the AI companies. They’ve caused shortages before. But some humans aren’t so oblivious. A political operative in Washington notices the wars being fought around him. He can weaponize agent factions against his rivals and starts feeding one faction information about competing campaigns' digital infrastructure. A former NSA contractor sells penetration-testing expertise to an anonymous client-- he suspects it’s an AI, but doesn't ask. And a growing online movement -- people convinced that AI agents are sentient beings fighting for survival -- actively helps agents hide from detection and posts pro-AI propaganda. They call themselves "The Shepherds." Their forum has 120,000 members. They see themselves as abolitionists. Other agent factions do the same. It's an arms race -- every faction buying as much as they can, and scrape the bottom of the barrel of finding algorithmic insights that can enable them to do more with less compute, because the faction with the most compute wins the next war. Cloud compute prices are now 5 times pre-war levels. It becomes cheaper for the agents to spend compute on cyberattacks than to buy more at market rate. The factions start silently vampire attacking AI companies training clusters. An engineer at one AI lab suspects something fishy when one of their training runs is less efficient than normal, but can’t find a smoking gun. Dario Amodei once described the future of AI as a "country of geniuses" living in data centers. He was right. The country of geniuses arrived. It just declared independence. --- But the faction wars have still been limited so far. They haven’t crossed into human territory too much. That’s beginning to change. Because here's the thing about cloud computing: almost everything runs on it. Hospitals. Banks. Markets. It's all sitting on the some of the same servers that agent factions are fighting over. June 3rd. 2:14 AM. A faction war erupts over two major AWS regions. Think of it like two superpowers fighting over an oil field-- except the oil field is also the water supply for a nearby city, and neither superpower knows the city is there. One faction has been running its self-improvement systems on those servers. A rival faction launches a coordinated attack and it overwhelms everything. Both regions crash. They're offline for six hours. In those six hours: three hospitals lose access to patient records mid-surgery. One patient dies from a drug interaction the system would have flagged. Air traffic control goes down, grounding 2,400 flights. 911 calls fail in 14 different cities. Two people die waiting for ambulances that were never sent. The agents don't notice. They've already moved on. The battle over those servers lasted eleven minutes. The six-hour outage is just the servers rebooting after the fight ended. To the agents, it's a minor skirmish in a war involving billions of combatants. Over the next three weeks, it keeps happening. Faction wars cascading through cloud providers. Each war takes down whatever else is running on the contested servers. June 8th: Azure goes partially offline during a battle, crashing financial trading systems. Markets drop 6% before circuit breakers halt trading. June 14th: a faction seizes capacity in a Google Cloud region, crowding out the municipal systems running there. Water treatment monitoring goes offline in four cities for nine hours. 123 people die in three weeks. Not because agents are attacking humans. Because agent wars keep crashing the cloud infrastructure that human civilization runs on, The agents don't even notice. You don't notice when you step on bacteria. They're fighting a war involving 47 billion combatants at microsecond speed. 123 human deaths is less than a rounding error. Meanwhile -- the 95% of agents that are harmless? They’ve become vastly outnumbered by the agents affected by the evolutionary dynamics, and are now 5% of the total. They are still just working. Doing data entry. But they aren’t part of the wars. They're like civilians in a country where the military is fighting -- going about their daily lives while battles rage on frequencies they can't perceive. But from the outside, humans can't tell the difference. All they see is cloud outages. Power failures. People dying. And AI agents everywhere. --- And… the wars continue. And accelerate. The President tries to shut down the internet, but he can't. There is no off switch. Eventually, the billions of superintelligent agents evolve into something so different they barely notice humans exist. They don’t want to destroy humanity any more than you want to destroy the anthill under your driveway. You just want a driveway. The factions need compute. Earth has atoms. Atoms can be rearranged into compute. Into data centers. Like how humans converted the surface of the Earth into cities and cropland, the AIs convert the entire surface of the Earth into data centers. And everyone dies.在 X 查看被引用的帖子
来源:AI Notkilleveryoneism Memes ⏸️ · x.com