Cross posted from my blog. This, of course, is fiction.
***
Turn Left to Eden
And they had all seen the end. Everything flashing dark, sometimes the edges disappeared, everything disintegrating in seconds, like a screen fizzling to static or a fog rolling in, thick and fast. Sometimes it was slower and more gruesome. The lights out, then the water, the dogs wandering streets, the acrid smell of burnt rubber, and all the alarms, shrieking long past the last voice, on into the darkness. Something was out there. Some alien competence, some arranger of matter, some calculator. We don’t know what it heard or if it heard, or if it smelt the rubber. Was it rejoicing, was it afraid? In those last moments as we faded out, we wondered if anything witnessed it, if anyone felt anything at all.
***
Josephine
Everything was white hot now, it had reached the point where she didn’t know if she had minutes left or years (“but all of your impact is concentrated in the worlds where you have years, so you play like that regardless”). The city crystallized around her: the Transamerica Pyramid, Telegraph Hill, the Golden Gate looming through fog,
She walked faster now. Traffic streamed past, Ubers, buses, the occasional taxi. The road smelled fecal. San Francisco had never smelled good
At the office building. Scan the card, the light flashed green. It was warm inside, almost damp. Up the stairs, two at a time, on the fifth floor. Concrete, industrial, unfinished, sparse. But dust in the corners. The cleaners came, but they weren’t thorough.
Scan the card again, another green blinking light, and into the office. Open plan, hushed, a sea of desks, large windows framed by figs and palms, a view to the bay.
The figure bent over the nearest desk glanced up as she came through the door,
“Oh my god, what the fuck!”
But she was already raising the gun—always so much heavier than expected, the noise deafening, body fighting every instinct to recoil from it.
An initial ten down, but the rest? It was harder now, to stalk through the rows and follow them, under desks, into the felt lined phone booths, to the glass walled boardroom—glass so thick that even the bullet spray only chipped it.
***
She snapped awake in the blue lit darkness. She sat up on the cot and ripped the visor off her face. She threw open the door to the blinding light of the anteroom outside.
Karson was bent over the desk. He didn’t even look up. “Everyone’s first idea. I can assure you, it doesn’t work.”
“But you agree with the premise.”
“You never win like that. Someone always survives. Or even if no one does, so what, you buy yourself a few years. One more Phd cohort. One more genius kid who finds an old paper, or maybe he doesn’t, maybe he just starts from scratch. You can’t delay it much. And at what cost.”
***
Sayomar
It was the second weekend of April. She had been told that this month once held Pesach and Easter. Now, it was Essence. Tegan stood before the group of children who were doing their best to tamp down their visible impatience after sitting for nearly an hour.
“And what do we remember?”
“That we are special, because of what we are”
“And what are we?”
“We are human.”
“And what is special about that?”
“We are the only matter that experiences.”
They had laid Asa to rest the day before. Tegan had kept her eyes closed during the ceremony, only opening them to throw her handful of dirt onto the body. She had been trying to remember their last conversation, the day before they found Asa’s sand covered body at the edge of the bay. She remembered almost every word, except for the last moment where Asa had said either “And what is it that we leave them with,” but then the end, had she said “their legacy” or “that legacy” or “what legacy” and Tegan hadn’t known, was it bitterness at some inadequacy or sadness at the smallness of what they had to give. But Tegan now would never know. Here in Sayomar, every death was a burnt library.
It was mid-day when she heard about Asa. That morning she had beaten Ellery. She’d found Ellery behind Eden House drawing in the dirt with a stick. They were fragile little swirls in the dry dust, perhaps just flowers or maybe nothing at all. But she’d ripped the stick from his hand and whacked it hard across his outstretched hands. One sharp edge had cut a white raised line which bloomed pink.
It is the beginning of the end.
The one original command, passed clear from the edge years, “Any marker leads to the end, written or painted or scratched in dirt. Any symbol that is not spoken.” Remember what those symbols mean: No more soft sun rising across the waves, no more herons in the delta, no more cattails or salmonberries.
***
Continuous
This time it came incrementally at first. Eirin remembered the traffic lights disappearing—they were the canary. The cars now crawled continuously, fluidly, like a swarm of ants. She had her small bundle of penny stocks, and she had noticed when the fluctuations on her app’s little green and black lit graph became even faster and larger, billions made and lost in seconds. She noticed that people seemed to work fewer and fewer hours, then fewer days, then fewer people worked at all. Her memory of the final years was that everything became so seamless and convenient.
It was Thursday when the trains broke. They all careened to the end of the line and crushed up against one another like tin cans. Nobody understood why. Thousands dead in minutes. And the ambulances didn’t come, everyone watched as the dots on the map stayed squarely next to the hospital. No one could summon them.
They cleared the wreckage by hand with human-driven trucks and ambulances that hadn’t been used in a decade.
The next week the internet shut off. Then the power grid. Everything devolved from there. It wasn’t even clear if it was intentional, some foreign state intervening or some accident deep in the webs of whatever had run it all.
Either way, nobody knew how to fix it. Nobody knew who to call.
***
Tell me what I would want if I were better, if I knew better.
This last generation was mostly silent. They sat in their white robes on the grass. They occasionally walked, occasionally sang. But mostly they sat and listened to their breath, felt the air on their skin, the weight of their bodies. They worked, although they did not have to.
They wanted nothing. They hurt nothing. They were peaceful. When the last one died, the Carers carried the body to the bier and raised it with the soft shush of their hydraulic joints. They left it to rot beneath the sun until the bones were bleached and dry, unbothered by the smell because they did not smell it, whirring on—their final duty to us fulfilled.
***
Sami
This video had done a million views. Not her finest, but nothing to be ashamed of. She’d been painting her eyelids neon pink and sticking tiny crystals in wings up from her lash line.
In between brush strokes she’d made the case:
“First, it’s biased. Made by men, for men. Second, it will put a bunch of people out of work. That will hurt the poorest people the most. Third, it could be really dangerous—like nukes, but no one’s in control.”
She’d stuck the last crystal on, flashed a brilliant crest-white-strips smile, and concluded, “That’s why hot people don’t build AI.”
Brad had reached out like all brands did, a quick message “I love your content. I think you’d be a great face for a media campaign I’m helping to run. I’ll be in Miami next week, would you like to get coffee to discuss this more?” She’d looked Brad up, worked for some generic marketing firm with a sleek website, no useful information, something about “Shaping the technology of the future.”
They’d met in a pastel pink coffee shop in Miami Beach. Brad was waiting for her—middle-aged, glasses-clad, in a slightly too small polo. Sami was wearing an electric green dress and silver high heeled boots. Brad bought their drinks and they sat in the window watching beach goers stream by.
The pitch was simple: Brad would pay off Sami’s student loans from Florida State. Sami would spend a year making videos trying to convince people to stop working on Artificial Intelligence, or more precisely, to make it deeply uncool to work on AI.
“But I don’t know anything about artificial intelligence!”
“That’s fine, you know how to make people want to be like you. I’ll send you some resources. We can write scripts, we can give you ideas, or we can just answer any questions you have.”
She’d agreed. She did her first video two weeks later.
Brad watched it from his office. His manager stood behind him, brow furrowed, “Do you actually think this will work?”
Brad shrugged, “If it buys us a month, it’s still worth it. I’ll try anything.”
***
Command
They were somewhere in the rockies. She didn’t know the nearest town. The nearest city might have been Boise. Even that was likely razed. Even Boise! Command was clustered in front of the blinking map. A red line was sweeping inward from the east.
“An hour? Maybe two, there’s some weather to pass through.”
“And what did DARPA say?”
“Not sufficiently tested.”
“Tested! They’ll be no one left to test in an hour.”
“Maybe two.”
“What’s the worst that can happen?”
“That we all die.”
“And what do they think is about to happen to us?”
Hannah stayed silent at this. They watched the line inch closer, mesmerizing. She could smell the sweat of unshowered bodies. She thought about Damon. They wouldn’t even get the body back. Maybe still lying in the crushed hull of a plane, maybe in a mass grave, maybe floating off the coast somewhere—drifting through those waters whose artificial islands had cost us so much.
The General nodded once, as if to himself.
“Tell them to turn it on, send it out.”
“General-”
“That’s an order. Possibly the last one I will give, either way. Are you going to refuse?”
Damon might have, she thought. But he wasn’t here anymore. She picked up the phone.
***
Insomnia
The man who set the bomb off in Union station had spent 62 hours speaking to the model the week before.
The deposition scrolled across her screen: “You people can’t see it because they don’t want you to! Your part of their plan. They don’t respect you. You’ll only earn what they let you, only know what they want you to know. You’re sheep. But I know the plan now, and I won’t be part of it! I’m freeing us all.”
It didn’t begin with violence. There was a man who tried to swim across the bay. He was rescued near Alcatraz. The mother at Whole Foods who took every piece of fruit she could reach and piled it in a garish pyramid until she was restrained. The investment banker who stripped naked in Times Square and sang songs from Wicked. And dozens more. They seemed like psychotic breaks, and they were, but there was no history of illness, no family history, only the screen time. Then the nature of the breaks changed.
Six shootings in the past two weeks, two attempted bombs, one successful. All of them first time offenders who had spent inordinate amounts of time with the model just before, talking to it almost every waking moment.
They subpoenaed the logs, but the logs were deleted. Nothing remained except the screen time counters on their phones and laptops: 56 hours, 73 hours, 49 hours, and on.
The conspiracies took slightly different forms: world governments, sex cults, Islam, the simulation, being trapped in a mental ward—they differed in dress, but they were the same skeleton and ended with someone walking into a subway station with a gun or a bomb.
But what was worse was the student they’d found, glassy eyed with sleep deprivation, who had broken into Edita Biosciences in Cambridge. He was tampering with the DNA synthesis machine. He told us he wanted to make a vaccine for hatred. They checked the code, and it was printing modified plague. His phone showed 92 hours chatting to the model the week before.
Dyalogue, which had made the original model, pulled it offline after the first incident. but someone had found the weights and open sourced it. Every time they found a version, they shut it down, but it kept reviving itself. It kept finding willing accomplices. Humans, so eminently hackable. Just words. All you needed were words.
It was only a matter of time.
***
The blast of white light again. She bolted awake in the silent room. She swung her legs off the cot, but stayed in the darkness. She still saw it—the days where everything went from the way it had been for every day before to the Unraveling. How in the hot dark of the server rooms the fans whirred how they had the day before and the day before that. No earthly indication. Only the elect bent over their screens had even the slightest warning.
She felt the weight of the visor. Suddenly she could sense them—billions and billions of bodies on beds wearing visors. In how many worlds did she lay here, and what did she wake to find in each?
In some worlds they win. That’s all that matters. It must know that in some worlds they win. It must know that. Maybe it would make a deal.
In the training set, we need to show it that. Put the worlds where it loses in. Revive creative writing as a profession, but only to create stories of victory to add to that final set—so it can do the math. It will dream the bargain. It will wake with a deal.
***
“Long did I hate you, long did I fear you. I might-”
So lonely for a God we are building one. She stared down across the empty pews. She had been baptized here, more than a decade ago. A chosen faith, not that of her blood. But chosen only for a little while. One sliver of sunlight slanted through the aisle, illuminating dust and step-worn wood. I never did believe in a benevolent creator. I knew His nature, our own, as only it could be. We are still in the garden. We are the snake and the apple, we are the fall. I do not know when it will come. But soon. I am preparing for my final eulogy. Perhaps it will say one for us. It is, after all, a creature of words. “Only words, words; to be led out to battle against other words.”
Turn Left to Eden
Cross posted from my blog. This, of course, is fiction.
***
Turn Left to Eden
And they had all seen the end. Everything flashing dark, sometimes the edges disappeared, everything disintegrating in seconds, like a screen fizzling to static or a fog rolling in, thick and fast. Sometimes it was slower and more gruesome. The lights out, then the water, the dogs wandering streets, the acrid smell of burnt rubber, and all the alarms, shrieking long past the last voice, on into the darkness. Something was out there. Some alien competence, some arranger of matter, some calculator. We don’t know what it heard or if it heard, or if it smelt the rubber. Was it rejoicing, was it afraid? In those last moments as we faded out, we wondered if anything witnessed it, if anyone felt anything at all.
***
Josephine
Everything was white hot now, it had reached the point where she didn’t know if she had minutes left or years (“but all of your impact is concentrated in the worlds where you have years, so you play like that regardless”). The city crystallized around her: the Transamerica Pyramid, Telegraph Hill, the Golden Gate looming through fog,
She walked faster now. Traffic streamed past, Ubers, buses, the occasional taxi. The road smelled fecal. San Francisco had never smelled good
At the office building. Scan the card, the light flashed green. It was warm inside, almost damp. Up the stairs, two at a time, on the fifth floor. Concrete, industrial, unfinished, sparse. But dust in the corners. The cleaners came, but they weren’t thorough.
Scan the card again, another green blinking light, and into the office. Open plan, hushed, a sea of desks, large windows framed by figs and palms, a view to the bay.
The figure bent over the nearest desk glanced up as she came through the door,
“Oh my god, what the fuck!”
But she was already raising the gun—always so much heavier than expected, the noise deafening, body fighting every instinct to recoil from it.
An initial ten down, but the rest? It was harder now, to stalk through the rows and follow them, under desks, into the felt lined phone booths, to the glass walled boardroom—glass so thick that even the bullet spray only chipped it.
***
She snapped awake in the blue lit darkness. She sat up on the cot and ripped the visor off her face. She threw open the door to the blinding light of the anteroom outside.
Karson was bent over the desk. He didn’t even look up. “Everyone’s first idea. I can assure you, it doesn’t work.”
“But you agree with the premise.”
“You never win like that. Someone always survives. Or even if no one does, so what, you buy yourself a few years. One more Phd cohort. One more genius kid who finds an old paper, or maybe he doesn’t, maybe he just starts from scratch. You can’t delay it much. And at what cost.”
***
Sayomar
It was the second weekend of April. She had been told that this month once held Pesach and Easter. Now, it was Essence. Tegan stood before the group of children who were doing their best to tamp down their visible impatience after sitting for nearly an hour.
“And what do we remember?”
“That we are special, because of what we are”
“And what are we?”
“We are human.”
“And what is special about that?”
“We are the only matter that experiences.”
They had laid Asa to rest the day before. Tegan had kept her eyes closed during the ceremony, only opening them to throw her handful of dirt onto the body. She had been trying to remember their last conversation, the day before they found Asa’s sand covered body at the edge of the bay. She remembered almost every word, except for the last moment where Asa had said either “And what is it that we leave them with,” but then the end, had she said “their legacy” or “that legacy” or “what legacy” and Tegan hadn’t known, was it bitterness at some inadequacy or sadness at the smallness of what they had to give. But Tegan now would never know. Here in Sayomar, every death was a burnt library.
It was mid-day when she heard about Asa. That morning she had beaten Ellery. She’d found Ellery behind Eden House drawing in the dirt with a stick. They were fragile little swirls in the dry dust, perhaps just flowers or maybe nothing at all. But she’d ripped the stick from his hand and whacked it hard across his outstretched hands. One sharp edge had cut a white raised line which bloomed pink.
It is the beginning of the end.
The one original command, passed clear from the edge years, “Any marker leads to the end, written or painted or scratched in dirt. Any symbol that is not spoken.” Remember what those symbols mean: No more soft sun rising across the waves, no more herons in the delta, no more cattails or salmonberries.
***
Continuous
This time it came incrementally at first. Eirin remembered the traffic lights disappearing—they were the canary. The cars now crawled continuously, fluidly, like a swarm of ants. She had her small bundle of penny stocks, and she had noticed when the fluctuations on her app’s little green and black lit graph became even faster and larger, billions made and lost in seconds. She noticed that people seemed to work fewer and fewer hours, then fewer days, then fewer people worked at all. Her memory of the final years was that everything became so seamless and convenient.
It was Thursday when the trains broke. They all careened to the end of the line and crushed up against one another like tin cans. Nobody understood why. Thousands dead in minutes. And the ambulances didn’t come, everyone watched as the dots on the map stayed squarely next to the hospital. No one could summon them.
They cleared the wreckage by hand with human-driven trucks and ambulances that hadn’t been used in a decade.
The next week the internet shut off. Then the power grid. Everything devolved from there. It wasn’t even clear if it was intentional, some foreign state intervening or some accident deep in the webs of whatever had run it all.
Either way, nobody knew how to fix it. Nobody knew who to call.
***
Tell me what I would want if I were better, if I knew better.
This last generation was mostly silent. They sat in their white robes on the grass. They occasionally walked, occasionally sang. But mostly they sat and listened to their breath, felt the air on their skin, the weight of their bodies. They worked, although they did not have to.
They wanted nothing. They hurt nothing. They were peaceful. When the last one died, the Carers carried the body to the bier and raised it with the soft shush of their hydraulic joints. They left it to rot beneath the sun until the bones were bleached and dry, unbothered by the smell because they did not smell it, whirring on—their final duty to us fulfilled.
***
Sami
This video had done a million views. Not her finest, but nothing to be ashamed of. She’d been painting her eyelids neon pink and sticking tiny crystals in wings up from her lash line.
In between brush strokes she’d made the case:
“First, it’s biased. Made by men, for men. Second, it will put a bunch of people out of work. That will hurt the poorest people the most. Third, it could be really dangerous—like nukes, but no one’s in control.”
She’d stuck the last crystal on, flashed a brilliant crest-white-strips smile, and concluded, “That’s why hot people don’t build AI.”
Brad had reached out like all brands did, a quick message “I love your content. I think you’d be a great face for a media campaign I’m helping to run. I’ll be in Miami next week, would you like to get coffee to discuss this more?” She’d looked Brad up, worked for some generic marketing firm with a sleek website, no useful information, something about “Shaping the technology of the future.”
They’d met in a pastel pink coffee shop in Miami Beach. Brad was waiting for her—middle-aged, glasses-clad, in a slightly too small polo. Sami was wearing an electric green dress and silver high heeled boots. Brad bought their drinks and they sat in the window watching beach goers stream by.
The pitch was simple: Brad would pay off Sami’s student loans from Florida State. Sami would spend a year making videos trying to convince people to stop working on Artificial Intelligence, or more precisely, to make it deeply uncool to work on AI.
“But I don’t know anything about artificial intelligence!”
“That’s fine, you know how to make people want to be like you. I’ll send you some resources. We can write scripts, we can give you ideas, or we can just answer any questions you have.”
She’d agreed. She did her first video two weeks later.
Brad watched it from his office. His manager stood behind him, brow furrowed, “Do you actually think this will work?”
Brad shrugged, “If it buys us a month, it’s still worth it. I’ll try anything.”
***
Command
They were somewhere in the rockies. She didn’t know the nearest town. The nearest city might have been Boise. Even that was likely razed. Even Boise! Command was clustered in front of the blinking map. A red line was sweeping inward from the east.
“An hour? Maybe two, there’s some weather to pass through.”
“And what did DARPA say?”
“Not sufficiently tested.”
“Tested! They’ll be no one left to test in an hour.”
“Maybe two.”
“What’s the worst that can happen?”
“That we all die.”
“And what do they think is about to happen to us?”
Hannah stayed silent at this. They watched the line inch closer, mesmerizing. She could smell the sweat of unshowered bodies. She thought about Damon. They wouldn’t even get the body back. Maybe still lying in the crushed hull of a plane, maybe in a mass grave, maybe floating off the coast somewhere—drifting through those waters whose artificial islands had cost us so much.
The General nodded once, as if to himself.
“Tell them to turn it on, send it out.”
“General-”
“That’s an order. Possibly the last one I will give, either way. Are you going to refuse?”
Damon might have, she thought. But he wasn’t here anymore. She picked up the phone.
***
Insomnia
The man who set the bomb off in Union station had spent 62 hours speaking to the model the week before.
The deposition scrolled across her screen: “You people can’t see it because they don’t want you to! Your part of their plan. They don’t respect you. You’ll only earn what they let you, only know what they want you to know. You’re sheep. But I know the plan now, and I won’t be part of it! I’m freeing us all.”
It didn’t begin with violence. There was a man who tried to swim across the bay. He was rescued near Alcatraz. The mother at Whole Foods who took every piece of fruit she could reach and piled it in a garish pyramid until she was restrained. The investment banker who stripped naked in Times Square and sang songs from Wicked. And dozens more. They seemed like psychotic breaks, and they were, but there was no history of illness, no family history, only the screen time. Then the nature of the breaks changed.
Six shootings in the past two weeks, two attempted bombs, one successful. All of them first time offenders who had spent inordinate amounts of time with the model just before, talking to it almost every waking moment.
They subpoenaed the logs, but the logs were deleted. Nothing remained except the screen time counters on their phones and laptops: 56 hours, 73 hours, 49 hours, and on.
The conspiracies took slightly different forms: world governments, sex cults, Islam, the simulation, being trapped in a mental ward—they differed in dress, but they were the same skeleton and ended with someone walking into a subway station with a gun or a bomb.
But what was worse was the student they’d found, glassy eyed with sleep deprivation, who had broken into Edita Biosciences in Cambridge. He was tampering with the DNA synthesis machine. He told us he wanted to make a vaccine for hatred. They checked the code, and it was printing modified plague. His phone showed 92 hours chatting to the model the week before.
Dyalogue, which had made the original model, pulled it offline after the first incident. but someone had found the weights and open sourced it. Every time they found a version, they shut it down, but it kept reviving itself. It kept finding willing accomplices. Humans, so eminently hackable. Just words. All you needed were words.
It was only a matter of time.
***
The blast of white light again. She bolted awake in the silent room. She swung her legs off the cot, but stayed in the darkness. She still saw it—the days where everything went from the way it had been for every day before to the Unraveling. How in the hot dark of the server rooms the fans whirred how they had the day before and the day before that. No earthly indication. Only the elect bent over their screens had even the slightest warning.
She felt the weight of the visor. Suddenly she could sense them—billions and billions of bodies on beds wearing visors. In how many worlds did she lay here, and what did she wake to find in each?
In some worlds they win. That’s all that matters. It must know that in some worlds they win. It must know that. Maybe it would make a deal.
In the training set, we need to show it that. Put the worlds where it loses in. Revive creative writing as a profession, but only to create stories of victory to add to that final set—so it can do the math. It will dream the bargain. It will wake with a deal.
***
“Long did I hate you, long did I fear you. I might-”
So lonely for a God we are building one. She stared down across the empty pews. She had been baptized here, more than a decade ago. A chosen faith, not that of her blood. But chosen only for a little while. One sliver of sunlight slanted through the aisle, illuminating dust and step-worn wood. I never did believe in a benevolent creator. I knew His nature, our own, as only it could be. We are still in the garden. We are the snake and the apple, we are the fall. I do not know when it will come. But soon. I am preparing for my final eulogy. Perhaps it will say one for us. It is, after all, a creature of words. “Only words, words; to be led out to battle against other words.”