Red watches the clefairy walk away, mind stuttering and restarting between thoughts.
I notice…
No seriously what-
IT’S A TALKING CLEFAIRY
…that I…
-is that a talking pokemon I didn’t see its mouth move-
CATCH IT NO WAIT BRAIN DAMAGE(?!)
-but the sound definitely came from it or maybe the house behind it-
…am confused.
“The fuck,” Leaf finishes in a deadpan voice.
And then the clefairy reaches the front door and vanishes.
Red’s brain finally lurches back into gear as it processes this final bit of information, and everything snaps into place.
Red glances at Leaf, whose expression of sudden understanding mixed with both relief and disappointment makes him suddenly begin to giggle.
Leaf looks at him, then begins to giggle too, until they’re both laughing full out. Red falls to one knee and Leaf stumbles over to the wall to lean against it, face turning red as she struggles to stay standing.
“Your… face…” she eventually says between bouts of smothered giggles. “It was like… your brain was melting!” And then she’s laughing again.
Red picks his hat up and wipes a tear away, still giggling. “You weren’t much better,” Red gasps, trying to catch his breath. “I think that’s the first time… I heard you curse.”
Leaf shakes her head, still giggling. “Nope. You didn’t hear anything.”
“Fair enough. And I definitely didn’t almost think I saw a talking pokemon. In fact, as far as anyone else will ever know, we both comported ourselves like the intelligent, level headed trainers we are.”
“Agreed.” She takes a deep breath, then pushes away from the wall, testing her balance for a moment before being satisfied. She turns to the front door, still smiling. “Why does he even have a clefairy hologram outside his house in the first place?”
They both jump as an arbok suddenly appears in front of them, swaying from side to side with its hood flared. “Would you have preferred something like this?” the voice says. “The point was to bring you this way without making you feel threatened.”
“W-why wouldn’t you just use the speakers instead?” Red asks, pulse once again dashing frantically at the sudden appearance of the arbok. Red can definitely tell the voice is coming from the direction of the door now.
“The answer will be obvious once you step inside. Which you still haven’t done. Now hurry up.” The arbok vanishes.
Red and Leaf exchange a look.
“Think whatever he’s using to watch us has a recording of us losing it?” Leaf asks, face straight.
“Almost certainly,” Red sighs. “Ah well. So much for secrecy.”
Leaf nods sagely. “Fuck it.”
And the two begin giggling again as they step toward the door, which automatically opens to reveal a straight, bare hallway.
The temperature inside is cool, the lights dim but steady. At the end of the hallway the clefairy waits for them, and in the dimmer light it’s easier to see the latticework of thin colored beams coming down from dots on the ceiling to make the image. As they approach, it takes the left hand path, leading them through a living room. There’s an attached kitchen, and the clefairy stops outside it.
Red and Leaf stare at it a moment, and then Bill’s voice makes them both jump. It’s loud, coming from all around them. “Grab me a soda, would you? Feel free to help yourselves too.” He sounds distracted, and Red hears the hum of an open mic for a moment before it cuts.
The two exchange glances, then Leaf slowly steps forward and opens the fridge. “Um. Preference?”
“Uh, anything’s fine.” Red takes the can and looks around. He spies a bathroom through an open door, and a bedroom in yet another. All of the rooms are barely furnished with bare walls. “Should we wait here?”
“This way.”
The clefairy walks toward a stairway in the corner, and disappears. Red wonders if it would reappear at the bottom, but when he and Leaf descend, they find themselves facing a door made of some strange, opaque glass. A red beam quickly scans them from head to toe, causing both to recoil and wince, and then the door begins to make pneumatic noises as it unlocks, shifts in place a bit, then slides open.
The first thing that Red notices is the music, a light and quick instrumental song, mostly composed of the violin and piano. It’s loud enough that Red is surprised he didn’t hear even a bit of it before the door opened.
Red and Leaf stare at the laboratory beyond the doorway. Rows and rows of work tables, stocked with every kind of biochemical equipment known to man… and quite a few that look completely alien to Red.
The rows of different microscopes are easy to identify, but next to them is something that looks like a cross between a fridge, an incubator and a thermocycler. Meanwhile, the actual thermocyclers are on their own table next to some vortexers.
The most extraordinary sight, however, is the sheer movement of the lab.
Centrifuges are spinning, racks of stoppered vials shift up or down, contents plucked by robotic arms and placed in temperature controlled containers or other equipment.
The music cuts off, and Bill’s voice fills the room. “Well? Come in.”
The two step over the threshold together, and the door closes behind them in a way that Red can’t help but find ominous. All the strangeness is starting to worry him. How much does anyone really know about Bill, anyway? The guy is notorious for being secretive, yet he invites two strangers into his lab without apparent reason? Professor Oak wouldn’t send us here if Bill was some crazy hermit…
Unless Bill went crazy recently.
“Uh. Hi, Mr. Sonezaki,” he says. “Are you here?”
The clefairy appears ahead of them, floating over a round indentation in the floor and ceiling. It doesn’t move, but merely points an arm. Apparently the hologram network isn’t as extensive here. Red studies the indent in the floor as they pass it, but can’t see anything that explains its purpose.
They pass rows of freezers and other chemical storage containers, all labeled with a dizzying amount of materials. Electrophoresis boxes, fume hoods… is that a hazchem suit draped over the back of that chair?
“Just one person works here?” Leaf asks as they pass some NMR and chromatography work tables, with enough spectrophotometers to take up their own wide table.
Red stops moving for a moment to study a series of 3D printers set against one wall. “I don’t think even Pallet Labs has this much equipment.” He keeps walking, then has to resist the urge to stop again and study what look like automated DNA extractors. “I knew he was rich, but the amount of money he could make renting this place out…”
“I think I have a new definition of the word,” Leaf murmurs as they pass from one section of the lab to another. Another holopad appears every so often, with floating clefairy pointing them first one way, then another as the lab continues to expand in different directions. One finally points them to a man, sitting in front of over a dozen glass boxes.
“Give me a minute,” the man says, back to them. “You can pour the soda in here.” He tilts his head to the side to indicate an empty cup with a straw in it.
Leaf and Red approach to look over Bill’s shoulders at his work station. Screens show each box containing a large petri dish with small, thin teal vines, all roughly the same size. As they watch, a drop of purple liquid falls onto each from small droppers suspended over them. Another drop falls, then another, then another, every few seconds.
“What are you doing?” Leaf whispers, both cans of soda still in her hand.
“Testing…” Drip. “The regenerative power…” Drip. “Of tangela cells.” Drip.
Red leans closer. “What’s in the-oh.” Red watches on one of the screens as a drop of the liquid hits a vine and makes a part of it wither, the vibrant teal turning brown… for a moment at least, until it suddenly fills out and regains its color again. Red watches the liquid etch a scar down either side of the vine to collect in the edges of what he originally took to be a petri dish: instead it’s a plastic lid over some kind of drain.
“Roserade acid,” he says, typing with one hand and reaching for his cup with the other. “They look like they’re fully recovering, but they weren’t always this small. Biomass decrease has been mostly linear. We should be near the end soon… Hey, the soda?”
“Oh!” Leaf says. “Right.” She opens a can and pours it into the cup, slowing as the foam builds up.
“Thanks.” He immediately turns his head a bit and begins drinking from the straw.
Red watches him, feeling a bit surreal. Whenever he imagined someday meeting Bill Sonezaki, it was never like this. Up close, the legendary inventor appears older than in videos and pictures. Though still in his mid thirties, there are already silver streaks in his hair, and deep lines around his eyes. He has a few days worth of scruff on his cheeks, and there’s an odd device around one of his ears, attached to a small screen in front of his right eye. It looks familiar to Red, and after a moment he realized it reminds him of an anime where people had devices that would “scan” a pokemon’s “power level.” He can see data on the lens, though he can’t read it, and watches Bill’s eyes as they alternate between watching the camera feeds on the monitors and going out of focus to read the smaller, closer screen.
“So, uh. You said you needed help with something? Is it this?” Red leans down to get a closer look at the thin vine.
Bill sucks the last of the soda from the cup, and straightens. “Ahhh. Nope, just wanted a soda. Thanks.” He belches. “‘Scuse me.”
Red stares. “A soda.”
“Yeah, I didn’t want to leave in the middle of the trial.”
“But… Professor Oak called me over an hour ago,” Red says, speaking slowly. “You’ve been sitting here that long?”
“Oh, hell no.”
“Ah, then what-”
“I’ve been here about… how long is it now, Eva?”
A woman’s voice speaks all around them, causing Leaf and Red to jump. “Three hours, seventeen minutes and thirty seven seconds.”
“Yeah, that sounds about right. This latest sample blew past my expectations, or I would have brought more to drink.” Drip… drip… drip… “Looks like they need a refill.” He gets up and goes to each container, refilling their drippers with wide vials of bright purple acid.
I thought Bill lived here alone? Red’s about to ask about it, when Leaf speaks up. “Um. Mr. Sonez-”
“Just Bill is fine.”
“Bill, okay. So, um, why did you invite us here?”
“Hmm.” Bill continues to drip the acid with one hand as the other changes the magnification. “You know, I can’t remember. There was something I wanted you to do for me, but I was a bit preoccupied with this when Oak called. And thirsty.”
“Was it about the abra?” Red asks. “I want to use your land, to catch some. I have an idea to-”
“No,” Bill says, frowning. “I don’t think that was it.”
Red’s stomach turns to lead, and he exchanges a look with Leaf, who gives a helpless shrug. “Are you sure? The Professor said-”
“Right, right,” Bill says, gaze still on the screens.
Red blinks, waiting for more. He wonders for a moment if Bill is carrying on two conversations at once, through his earpiece. “So… was that a yes? On the abra thing?”
“Acoustic displacement, right? Herding them into a hazard zone? Yeah, sounds fun.”
Relief floods through Red, and he sees Leaf hesitate before saying, “Sooo… the thing you asked us here to do for you… Could it have just been to bring you a soda, then?”
“No, no.” Bill slowly refills the acid in one of the drippers, one hand leaving the beaker for a moment to scratch his hair. “Maybe.”
They stare at him.
“A bit. I’m sure there was something else too though.” Bill checks the amount in the dripper feed, then moves on to the next box. “Eva, did I set a memo?”
“No, sir,” the voice says.
“Damn. Memo, Eva: ‘Make more memos. Especially after phone calls.’ Maybe if I listen to a recording of the call I’ll remember.” He continues his work silently for a moment, then shakes his head. “Nope, didn’t help.”
“I can start naming things?” Red asks. “Free association?”
“Go for it.”
“Something to do with pokemon. Something to do with catching them. Catching abra. Psychics Types. Something to do with us. Leaf Juniper? Red Verres? Um.”
“People, places, things,” Leaf says. “Pallet Town, Pewter City? Maybe about what happened with us at Mt. Moon?”
Bill stops shaking his head, brow raised as he lifts the acid container and looks at them. “Wait, what happened at Mt. Moon?”
“You didn’t hear?”
“I don’t really follow the news. And by really I mean pretty much ever.”
“Um. Well it probably wasn’t that then.”
“Was it an activity you wanted to do?” Red asks. “Talk to us about something? Our journey? The new pokedexes?”
“You know, I’m starting to think it might have been the soda,” Bill says, voice thoughtful as he finishes with the last container.
“You didn’t invite us all the way out here just to give you a soda,” Red says. He’s not sure if he’s trying to convince Bill or himself. “Inviting strangers into your house, just for that? Aren’t you a really private person?”
“Ha. The media just say that because I won’t let them step foot on my property. Or grant interviews.” He returns to his desk and types something out that brings up a bunch of graphs, displays of the weight of each sample over time. “Also might be because I don’t go anywhere. People tend to irritate me. Well, I’ll figure it out. You guys are welcome to hang around while I finish this.”
“Is this… something you do often?” Leaf asks as she circles a container, then kneels a bit to look under the dish. Red wonders how she feels about watching a piece of a pokemon get experimented on. Maybe it’s not so bad since it’s just a vine, and tangela lose bits of them all the time… though Red has to wonder how big this one was when it started.
“Yes, but normally it doesn’t take so long. This new strain is definitely going to shift priorities around. Hopefully I can get this all fully automated by the end of the week, so I can start the next trials soon.”
Red blinks. “New strain? You made this vine?”
“Tweaked it. Tangela cells are pretty efficient at regeneration, and occasionally you’ll find one in the wild that heals at ridiculous rates. I just had to find out which genetic markers were different between them and the others, and see if I could improve it further.”
“That’s amazing,” Leaf says as she watches a vine regenerate over and over again. “Are you trying to design a better potion formula?”
“Nah, I’ll leave that to Devon. I’d rather just give people these abilities instead.”
The lab is silent but for the movement of machinery, and Bill’s fingers moving over his keyboard. Red and Leaf both stare at him, then look at each other. “Is that… possible?” Red asks at last.
“Possible? Sure, why not. Probable? Dunno. But regenerating cells is something our body knows how to do already, and if we can make them better at it, the payoff would be huge. Rapid healing, disease resistance, limb regeneration, and if we’re lucky, even stop the effects of aging. Maybe eliminate them altogether.”
Red tries to wrap his mind around humans having such powers. It would be… amazing. Just one of those would make people so much safer, reduce so much suffering. But all of them, together? It’s like something out of science fiction. He can’t help but be skeptical, but if Bill Sonezaki thinks it can be done, is committing his time and energy to doing it…
“And pokemon?” Leaf asks. “Are you trying to give them these abilities too?”
“Naturally.” Bill frowns at a screen, then brings up a code editing window and examines it. “I’ve spent half my life writing TMs to give pokemon new abilities. Mostly for combat, because that’s what the market wanted. But this has combat value too. The next steps are to try and spread this regeneration to other plant pokemon, then non-plant pokemon, particularly mamm-Ooh, yes, that’s it.” He sits forward, eyes on a new window that popped up on his screen.
Leaf raises her head from the box she was examining. “What-”
“Shut up, I need to concentrate.”
Leaf’s mouth drops open, eyes wide. “I… I’m sorry, I didn’t mean to-”
“Shhh, shshshsh…”
Red feels anger boiling up in his chest. Don’t upset him, we’re guests, he might kick us out, the abra- “Hey! There’s no reason to be so rude after she came all this way to bring you a soda!”
Leaf rapidly shakes her head at him while Bill frowns, gaze still on the screen. “Okay, sure. Please shut up, would you? Go explore the lab for a bit. Don’t touch anything.”
Leaf is already moving toward Red before Bill finishes speaking. She takes his arm and drags him away before he can say anything else. “It’s okay, really,” she whispers. “I don’t think he means to be rude.”
“That’s not really an excuse,” Red whispers back as they leave Bill behind. “Who goes from a normal conversation to telling people to shut up without warning?”
“Someone without people skills. Maybe something important came up. Come on, let’s look around.”
He and Leaf make their way back through the lab, and before long the music comes back on through the speakers all around them. They drink their soda and find some more automated equipment to study, watching on monitors as data is gathered and recorded throughout dozens of trials. The sheer scope of the research Bill is getting done here makes Red envious.
“And this is just one building,” Leaf says. “We haven’t crossed over into one of the others underground, have we?”
“No, I think they’re all something different.” Red looks at some transfer slots, silver container balls resting in their docks. He can imagine Bill ordering the equipment he needs into them for easy distribution around the lab. “All this stuff has been just for biochem.”
It takes almost half an hour for Bill to join them, and they still don’t manage to see everything in the lab. He walks toward them with a purposeful stride, then passes right by. “Walk with me. There were other things I planned to do today before that took up my whole morning. Luckily none of it is time sensitive.”
They follow him through the lab as he checks on equipment and the results of certain trials, occasionally muttering to himself. Red realizes he’s probably talking to the woman, Eva, whoever that is. During one of the silent stretches, Red summons the courage to ask, “So, is it okay if we speak now?”
“Speak about what?”
“I mean ask questions. Talk.”
“Sure, why wouldn’t it be?”
Red sighs. When he imagined meeting Bill, he always expected someone a bit more like Professor Oak than Blue. “So, that clefairy hologram. Why, exactly?”
“The external holograms are useful in general for scaring off pokemon that get too close to the buildings. The clefairy is just the least threatening one I have, so I use it to interact with people without having to go up.”
“Right, but why not use yourself instead of a clefairy?” Leaf asks.
“They’re modeled after the pokemon that have been rendered for sims. I never bothered to digitize myself.”
They reach a door like the one they entered. Red’s pretty sure it’s not where they came in, and sure enough when it opens they face a completely different type of lab from the first one.
Instead of chemistry equipment, this area seems to be full of computers and robotics. There’s a lot less movement of ongoing experiments, but a lot more visibly identifiable projects. One desk is cluttered with parts for what looks like a new pokedex prototype, while another has a dissected pokenav. Each table has mechanical arms situated around them, most of them motionless.
As they enter, the music around them changes, this time to some electronic song with an industrial sound and heavy beats. After a moment Bill mutters something, and its volume drops to a background whisper.
“Is this where you work on storage?” Red asks. Bill’s development of the interregional storage system is what he’s most famous for, but Red doesn’t see anything that looks like it would be related.
“No, storage and transmutation tech is in the physics lab. This is where I study machine learning, particularly improving narrow AI and solving alignment problems.”
“Narrow AI?”
“Weak. Focused. Able to hold a conversation or perform tasks about just a few specific things, no matter how deep that thing is.”
“Opposed to being able to learn everything?”
“Yeah. Like your pokedex. It’ll tell you all you want to know about pokemon, but ask it how to cook your breakfast and you’re out of luck.”
“Oh. Isn’t that pretty easy to program in though?”
“Sure, you could program it for any number of tasks, hardware permitting. But it’ll never learn new ones on its own. You don’t know all this? What are they even teaching in schools these days, just how to throw a pokeball?”
Red flushes. “That, and how to stay alive.”
“You’re one of Oak’s though, aren’t you? I figured you’d know more than that.”
Red catches Leaf’s glance, and takes a breath to calm himself. “I never really studied computers much. Mostly psychology, physics, chemistry, and pokemon biology.”
Bill tsks. “All that’s not going to matter much if AI keeps improving at its current rate. Should’ve studied computers.”
“I did, a bit,” Leaf says before Red responds.
“Juniper, right? I helped design your granddad’s species tracking algorithms.” Bill leads them past more machinery and electronics, then stops at a holopad and pulls on some gloves that go to his elbows. “Fun guy to work with. Even funner to drink with.”
“Thanks. I think.”
“So tell me, spawn of Cedric, what you think you know about AI, and how you think you know it.”
Red blinks. He’s only ever read that phrase in Giovanni’s writings, and those that read him. He wonders if Bill does too.
“Well,” Leaf says as Bill mutters something, and the hologram suddenly comes to life, showing some complex shape Red can’t make heads or tails of. It looks like three series of spheres spaced out with lines drawn between them in three dimensions. “I guess the first thing I think I know is that general AI is hard. And the reason I think I know that is that if it wasn’t, we would have figured it out by now.”
“Go on,” Bill says as he studies the hologram a moment. Spheres and lines shift as they watch, and eventually Bill extends a hand and casually waves it along the side of the projection, shifting the whole image to view it from “below.”
“General AI would be… well, like a person. It would be able to think for itself, or at least think so broadly it might as well be considered conscious. But it would be smarter than us, be able to think thousands of times faster. All the speed of a computer with all the flexibility of a human mind.”
“And what would this AI do?” Bill asks.
“Well, whatever we ask it to. It could run tests faster than us, solve coordination problems, collate all the data in the world and examine it objectively to make connections we wouldn’t.”
“Mhm.” Bill turns the hologram again, then reaches in and manipulate some of the lines and orbs around, faster than Red could follow before pulling back out to look at it again, and watch how it changes in response. “So you’d just use it as an Oracle?”
“What do you mean?”
“Oh boy. Okay, let’s keep this basic. There are a few ways to classify AI. Some popular ones are Oracle, Genie, and Sovereign. Oracles are basically question boxes. They can’t act in the world other than to transmit information. You give the AI a set of data, ask it a question, and have it tell you the answer. Basically like the pokedex, but more broadly intelligent. Able to figure out answers you didn’t explicitly program it with.
“Genie aren’t contained. Hook a Genie up to a robot, give it a blueprint, and tell it to build you a house according to the blueprint using the materials you put in front of it, and if it’s made well, it’ll do that, then stop and wait for further orders. Or, to use an example of what’s coming soon to roads near you, put the Genies in cars that will drive you around anywhere you tell them to. A narrow Genie might choose from a set of predetermined routes to preset locations, but a more general intelligence auto could figure out its own route to custom locations. Tell it to drive you to a lake in the forest, and it’ll do it.
“And Sovereigns are the least tightly bound AI. They can take more complex orders, and carry them out in novel ways, without waiting for human approval at every step. Instead of giving the Sovereign the materials to use, you’d ask it to build you a house with whatever it could find. And if you’re not blisteringly stupid, you would put limitations on it to ensure those materials aren’t people, or pokemon, or from other houses.”
Red frowns as he watches Bill change the color of two of the spheres, which drastically alters the arrangement of the lines before he changes them back. “Are there any machines like that yet? Sovereigns?”
“Sure, in the narrow sense. Any machine that works independently on loose goals is a Sovereign. Computers trying to maximize returns in the stock market, for example. They have a goal and that’s about it.”
“Seems like a fine line between a Genie and a Sovereign,” Leaf says.
Red turns to her. “I think it’s about level of control, not intelligence. Sometimes they’re tangible, like, an Oracle like the pokedex can’t open doors or move anything, so it’s obviously constrained that way. But a Genie like an automatic car might have to ask you for permission and show its route before taking it, so you can stop it from driving you through a river. Whereas a Sovereign wouldn’t have to ask permission, it would just… do things?” He turns to Bill questioningly. “Why would anyone make a Sovereign, anyway?”
“Because sometimes you don’t know how to get to your goal at all. Remember, if AGI is being used, it’s being used to do something humans can’t. If you ask it to figure out a way to stop humans from aging, it might do it by manipulating our genetic code, or it might do it by synthesizing some wonder drug. You don’t actually know what you want it to do, you just know what you want done. It’s up to the machine to figure out what your actual desire is, your coherent extrapolated volition.” Bill frowns at his holograph, tweaks one more thing, then makes a gesture with his arm that shuts off the display. “Eva, prepare some food for us above the computer lab. Twenty minutes.” He strips the gloves off and sticks them in his pocket as he starts walking again.
“Certainly, sir. Preference?”
“Steak.”
“We have tauros and bouffalant in stock.”
“Tauros. What do you guys want?”
“Uh, anything’s fine,” Red says in surprise.
“Come on, kid, pick something.”
“Um. Come back to me.”
“No pokemon for me, please, Eva,” Leaf says, voice raised.
“Understood. I can prepare a salad of mixed greens with tangerine slices, walnuts and feta cheese to ensure a balanced nutritional meal. Is that acceptable?”
“Very acceptable, thank you.”
“You’re welcome.”
“Well?” Bill asks as he pauses to watch a mechanical arm disassemble and reassemble a series of small, complex metal pieces. It keeps trying new permutations, and Red is distracted by the blur of movement for a moment before he realizes Bill is talking to him.
“Oh, uh, pidgey burger? Please?”
“Certainly.”
“Thank you, Eva,” Red says, looking around for a camera or microphone to direct his attention to.
“You’re welcome.”
“So is Eva a Genie, then?” Leaf asks, and Red suddenly feels very stupid. “Since she—it—can’t act independently, and just follows your direct orders?”
“Yep, though she has a number of autonomous routines, as you’ve seen,” Bill says as he types something into the console beside the robotic arm, causing it to stop moving and reset back to a resting state. He then leads on, walking deeper into the lab. “As far as I’m aware she’s one of the four strongest AI in the world, but that just makes her less narrow than the others. She’s still a long way from true general intelligence.”
“So you’re trying to make her smarter?” Red asks. “Win the race for AGI?”
Bill barks laughter. “Fuck no, and I’ve had to sabotage a number of projects trying to win that race. Haven’t you been listening?”
“I think so? Wouldn’t strong AI help you out a lot? I mean, you’re not, uh, ‘blisteringly stupid,’ right? Eva’s not going to turn us into hamburgers just because she, I mean it, runs out of pidgey meat.”
Bill sighs. “Ok, let’s see how smart you are, Mr. Verres. What do you want? What’s your goal in life?”
Red pauses a moment to consider how this might be a trap, then says, “To learn. Specifically, I’m most curious about the origin of pokemon species. Study how they arise, where they come from.”
“That’s it?”
Red blinks. He’s not used to people dismissing his aspiration as too low. “It’s one of the greatest mysteries in the world. And… there are so many hypotheses and beliefs out there, none with any real evidence to support one theory over another. Learning the truth about reality is important to me.”
“One of Oak’s, alright. But try to dream a bit bigger. What do you really want, if you could have anything, even supposedly impossible things?”
I want my dad back.
It takes Red a moment to shove that thought away, after the pain of it echoes through his chest, again and again and again, reverberating with his heartbeats. He puts on a thoughtful face until it passes, breaths suddenly shallow.
He knows Bill said “impossible,” but the impossible has a quality to it that the merely improbable lacks. Even if a computer had a sample of his dad’s DNA, then flash grow a clone, at best the new Tom’s experiences would be made up of imperfect and disjointed memories from those that knew him. It could reassemble Tom Verres atom by atom, but where would it get an image for the brain? Short of time travel or some evidence of an afterlife, Red’s father is dead, and no scientific breakthrough, no matter how miraculous, is going to change that.
Once the ache fades, Red says, “I guess ending death would be the most important thing. Not just aging and disease, but also pacifying all wild pokemon. Make the world a truly safe place to live.”
“Fine, great, you’re an enlightened humanist. Now, what are your challenges, if you use AGI?”
“Um. Can I have a minute to think about it?”
“I would be disappointed if you didn’t take at least five.”
Red ponders this as they continue to walk around the lab. Leaf asks some questions about the future of human interface virtual reality while Red tries to think it through. He takes out his notepad and starts writing down ideas.
“Isn’t that kind of important though?” Leaf asks. “People could use that to train their pokemon so much more safely-”
“Yeah, but it’s kind of boring.”
Leaf’s eyes widen. “Boring?”
“It bores me.” Bill watches a group of small robots navigate a maze for a moment, then pulls up the past trial data from the screen beside it. “So I don’t do it.”
“But… it could solve so many problems! Help so many people!”
Bill shrugs, eyes on the screen. “So let someone else figure it out. I’ve got more important things to deal with.”
Leaf frowns. “So is it because it’s boring, or because it’s not important?”
“Both. Have you noticed that people have a hundred new problems and crises every year? They never stop finding new limits that they need someone else to help them overcome. It’s exhausting trying to keep up with it all.”
Leaf watches a robot stop moving as Bill types something on the keyboard, then start its maze over by trying a completely different route. “You talk like you’re not one of them.”
“I try not to be, when I can help it. I moved out here to get away from all their pointless needs.”
Leaf frowns. “Why bother with any of the things you do, then?”
“Because the problems I’m trying to solve matter. And before you ask, yes, I’m qualified to determine that. Especially since it’s my time and money I’m spending.”
“I didn’t mean to-”
“Yes you did, but it’s fine,” Bill says as he closes the program and starts walking again. “You’re still young. And that’s not an ageism thing, it’s just an objective metric of life experience.”
Red is only half listening to their conversation as he finishes sketching out his thoughts, but he catches the look from Leaf and smiles at her. “People skills,” he mouths, and her expression clears as she smiles back. “I think I’m ready,” he says.
“Alright, walk me through it.”
“Ok, so I’m not using a Sovereign at all. If I just say ‘Figure out a way to stop people from dying,’ it might just start capturing everyone in pokeballs. If I explicitly rule that out, it might make a nanobot army and go around knocking people out to put them in suspension pods that keep them alive indefinitely. If I add qualifiers like ‘make sure nothing else about them changes,’ it might find a way to stop people from dying that keeps aging. If I explicitly include a stop to aging in the requirement, it might make us stop being able to change at all, because I said ‘nothing else about them changes,’ and technically that could be interpreted as literally, everything else has to stay the same. There are just too many ways it might go wrong.”
Bill nods. “Basic, but you get the point. There are way worse things it could do.”
“Like what?”
“Remember that it’s a machine, not magic. It has to have the resources to accomplish whatever it sets out to do. It has to prioritize. Should it go for the big win that stops everyone from dying, or go for faster, smaller wins? Maybe it cures diseases first to save those people, then changes human genes to cure wounds in seconds to stop those deaths, then tries to stop aging to save the older people from dropping off.”
“None of that sounds bad,” Leaf says. “It might not be the most efficient, but it’s still saving people. Actually, it might be the most efficient after all. It’s smarter than us, isn’t it? Maybe its method of deciding would be better.”
“Better by what values? Is the life of a great grandmother with advanced dementia as valuable as the life of their great granddaughter? Even if we all agree that’s the case, and we input different weight to every category imaginable, ever see an AI play Chess, or Go?”
“Right,” Leaf says, speaking slowly. “It’ll start making decisions that don’t make sense to us.”
“It might even seem like it’s malfunctioning,” Red says. “How would we know? It might decide the main priority to save humans from dying is to stop the sun from eventually expanding, and waste all its time and the planet’s resources pursuing a path to stopping that. To us it would just look like it’s crazy and we’d pull the plug.”
Bill nods. “All this, of course, changes the more human-like the machine is in its intelligence. And it’s why it’s absolutely essential that it can communicate its intentions and actions clearly. We need to be able to understand what it’s doing and why, at all times. But that leads us to the question of autonomy. Who, ultimately, is it explaining its actions to? Who’s giving it orders? Its creator? Lot of power to put into one person’s hands. A committee? Just kill me now.”
“What about itself?” Leaf says quietly. “If it’s truly sapient, anything else would be slavery.”
“Give the girl a star!” Bill is getting more and more animated as the conversation goes on, and paying less attention to the various tasks he stops to do around the lab. Red wonders how often he has visitors, and if he misses company to talk with, even if they’re not his peers. “If we’re talking about a truly sapient machine, that’s a whole different mess. Me, I’m not bothered by the moral question as much as I am the security risk it poses. Anything with sapience and even the slightest bit of self-preservation is going to pose enormous existential risk, even if it’s just a box with a text screen.”
“But even without sapience, a strong enough AI could end humanity by accident,” Red says, thoughts spinning. “Why haven’t I heard about all this, anyway? An existential threat this big…”
“It’s too big,” Leaf says. “People can’t grasp it. It’s like worrying about a meteor strike.”
“But we know this meteor strike is coming, and soon,” Bill says. “Sure, ‘soon’ may be twenty years, or it may be fifty, or it may be a hundred. But it’s not an if, it’s a when. So, knowing all that, Mr. Verres, you still haven’t finished your explanation.”
“Right. Well. Sovereign is out, like I said. But so is Genie. Even if it’s one task at a time, that’s all it takes sometimes, especially if I’m not the only one with access to the AI. The more people it might take orders from, the higher the chances that it does something wrong, or does something the wrong way. I’m sticking with an Oracle. I teach it everything we know about biology, and ask it to tell me the instructions for designing a retrovirus that will end mammal aging. When it does, we study the design, and if it seems okay, create a batch and test it on pokemon. If it seems to work, test it on human volunteers.”
“And that’s how you would word it? ‘End mammal aging?'”
“Yeah. Even if it decides that killing something ‘ends aging,’ we’ll know from the pokemon trials before we try it on humans.” Leaf makes a face, but Red just shrugs. “Whatever the problem is, just keep re-iterating until we get it right.”
“And what if it’s communicable? You said ‘end mammal aging.’ Sounds to me like you want to end all mammal aging on the planet.”
“We’d test it in sterile chambers,” Red says. “Obviously.”
“Obviously. So, you’ve maybe got a beginning of an idea of one of the problems we face with advancing AI technology. And you started with Sovereign and worked your way down, which is the ideal way to think about AI safety.”
“There’s more to it than that though, right?” Red asks as he thinks through all the complications in designing a system that can think and act on its own “What about incentives? If it’s sapient, how do you get your machine to want to do things for you? Once you program its values, how do you program its incentives? There are so many ways it could go wrong!”
“Now you’re getting it.” Bill smiles. “I’m glad inviting you here wasn’t a waste of time.”
“I’m still stuck on the whole ‘slavery’ thing,” Leaf says. “There’s no way to actually stop an AI from becoming sapient accidentally, is there?”
“Not unless neuroscientists isolate what exactly consciousness is, and the brain structure responsible for it,” Bill says as he leads them to another door. Red wonders if they’re about to enter another lab or go upstairs to eat. “Until then, for all we know it might just be an emergent property of sufficiently broad intelligence, and could arise on its own if we make a computer that’s smart and flexible enough.”
The door opens to reveals a flight of stairs, leading them into a living room that looks exactly like the one they first entered to go into the biochem lab.
Bill walks into the kitchen, where three plates of food sit waiting, with a can of soda sitting beside each… the same flavors that they took earlier from the other fridge. “Help yourselves,” Bill says as he takes his plate over to the table, and Red and Leaf follow to do the same. Now that he has a moment to study it, Red notices there’s barely any room in the kitchen for someone to cook or move around: most of it is filled with a series of machines that Eva uses to prepare meals. Red looks up and sees motionless mechanical arms attached to rails on the ceiling.
“What do you guys want to look at while we eat?” Bill asks once they sit down. “Beach?” The walls suddenly have yellow sand, rolling blue waves, and piercing blue skies projected onto them in every direction, as if the three sit on a tiny island. “Forests?” The oceans are replaced with endless brown and green, and the slow roar of crashing waves is replaced with birdsong and wind rushing through countless leaves. “Cafe?” Now the walls show bustling sidewalks in Cerulean city, the forest sounds replaced by ambient chatter and traffic.
Red stares, mouth open mid-bite at the changing environment around them. “Is this… live footage?” he asks, watching a woman in a long coat with an eevee perched on her shoulder walk by on the wall to his left. If he pays attention, he can just make out the fuzziness of the image as it’s projected onto the blank walls.
“Nah, goes for about thirty minutes before it loops.”
“It’s awesome,” Leaf says. “This one’s fine with me.”
Red nods, finally biting into his burger. It’s delicious. “I think this is the coolest house I’ve ever been in,” he says, mouth full. “And the coolest labs. Thanks for inviting us here, Bill, even if it was just so you could get a soda.”
“I know there was something else,” Bill says as he starts to cut his steak. “It’ll come to me. In the meantime, let’s talk about your plan to catch abra.”
Red pulls his gaze away from watching someone ride by the street next to them on a tauros. “Sure. So, we’ve got some speakers, and I figured we’d use them to set up a field-”
“I know the basics. What I wanted to see for myself is what kind of person you are. Oak doesn’t give licenses out to just anyone, but it’s always good to be sure.”
“And… what kind of person am I?” Red asks.
“The kind who probably won’t get himself killed on my property and make me have to deal with the media. So when do you want to do it?”
“Oh. Well I figured we’d wait for Blue to finish at the gym for today, unless he gets out late. In which case, tomorrow?”
“No.” Bill shakes his head. “If you’re doing this on my land, you’ll wait till next week.”
Red blinks. “Um. Sure, if you insist. Why next week?”
“Because you’re going to spend the time between then and now preparing. You’ll find the best spot to do it, set up mock trials, and practice drills. Once you’ve got an idea of what to expect and how to respond, then you can try for real.”
“Yeah, okay, that makes a lot of sense. Are you going to determine if we’re ready or not?”
“Ha. Like I have time for that. No, I’m not going to babysit. You’ve got the land and the time you need to figure it out. The rest is on you.”
Red nods. “I appreciate it. More than I can say. Is there anything you want out of all this? Some of the abra, maybe?”
Bill waves his knife to the side dismissively. “Let’s just say you’ll owe me a favor. Nothing particularly dangerous, and nothing illegal. It’ll probably be whatever that thing is that I can’t remember wanting to ask you to do. Sound fair?”
“Yeah, more than fair! Thanks again.”
“Don’t mention it. Oak said you’re doing this for research, right? Not just to get rich quick? Because it’s a great idea for that.”
Red swallows his mouthful and washes it down. “Yeah.” He explains his ideas, and is surprised to see Bill’s attention sharpen away from his meal.
“No luck with the research journals so far, huh?” Bill asks, tipping his soda can back as he takes a swig.
Red shakes his head, suppressing a sigh. “I probably should have taken Professor Oak up on his offer.”
“Don’t let it get you down. The whole system’s broken, believe me: I’m self funded, sitting on top of a dozen new breakthroughs a year, have an AI to make writing research papers a breeze, and I still get frustrated by how broken the world of science publishing is.”
Red stares at him. “Uh. How is that supposed to not let it get me down, again?”
Bill purses his lips, then shrugs. “Alright, so it should probably get you down. If it helps, it’s just another problem I’m hoping will be solved soon.”
“How?”
“Something called Raikoth that should turn the scientific publishing world on its head.”
“Is it a research Oracle?”
“In a different sense. Think of a system of linked prediction markets. Kind of complex to get into right now, and probably a few years away from ready. But in the meantime, if you ever manage to find something solid in your research, let me know. A better understanding of psychic phenomenon might be enough for me to build a whole new lab.” Bill rubs his chin. “I’ve thought about diving into it before, just didn’t think it was worth it.”
“Well, if you want to fund some exploratory research…” Red grins.
Bill chuckles, shaking his head. “You can use my land, but without some solid justification that your idea has merit, even a few hundred dollars is money I have better priorities for. No offense.”
“No, I get it,” Red says, feeling only a little disappointed. It was a long shot, but now that he knows what Bill spends his time working on, Red can’t begrudge him higher priorities.
“Are you interested in psychic research to help with AI value alignment?” Leaf asks as she spears a tangerine slice with her fork.
“That and I’d like to be psychic, if I can. How’d you know?”
She smiles. “Seems like the best way to make sure it understands what you really want.”
Bill nods. “Find out how psychic communication works biologically, and we may be able to get it to work mechanically. Not only could we control machines telepathically, we could ensure that our actual CEV is more likely to be followed.”
“CEV?”
“Coherent extrapolated volition.”
“You mentioned that before,” Red recalls. “I understand the words individually, but as a phrase I’m not sure I get it. It’s just what you want? Making sense of your will?”
“It’s Yudkowsky’s term for the ‘end game,’ so to speak. Remember when we were talking about oversight? Who’s the computer listening to? Eventually we should probably make sure that one person can’t use the machine for evil, which means programming it with the ability to make all the best decisions for everyone, itself.”
“I can’t imagine people being happy with that,” Leaf says. “They’re barely content with other humans that they elected deciding things for them.”
“Again, end-game. You wouldn’t design your first AI to do this, it’s at the end of the hierarchy of getting it to do what you mean, and not just what you say, to the point where you may not even have to say anything anymore.”
“That would mean getting every part before it right,” Red says. “Not just what you value, but also what you will value, which means… knowing how you think? How you will think, in any given situation?”
Bill shakes his head. “More than that, even.”
“What can be more than that?” Leaf asks.
“Okay, so first you want to make sure the machine knows what you consider important, so it can avoid altering those in the wrong way, or let you know if something you ask it to do will require it to. So if you ask it to find a way to clean pollutants out of the air, and it knows that you care about there being a certain amount of oxygen in the air for humans to breathe, then it won’t use a solution that alters that.
“Second, you want the machine to be able to model and understand what you believe, so it can tell you if something you believe is wrong. If you ask an AI to find a way to undo the effects of a human entering a pokeball, the AI should be able to understand that you’re under the assumption that they’ll be restored back to their former self. If a treatment the AI comes up with would restore a human’s intelligence but wipe out their memories and personality, it should know to let you know that.
“Third, you want the machine to be able to model your desire in asking them to accomplish something. This is the classic idea of a wish being granted in a literal fashion rather than in the way the wisher intended, and of course, it’s incredibly complex and difficult. Like before, this is the machine knowing that when you ask it to end aging, you meant that you want to end the negative effects of aging on your mind and body.
“And finally, coherent extrapolated volition. Not just what you want, given the knowledge and beliefs you have, but what you would want, if you had all the knowledge the AI does, and could better consider arguments for and against your beliefs, and could better judge and understand yourself and your desires.”
“That… sounds incredibly hard. And dangerous,” Red says. He stopped eating while he listened, and brings the burger halfway up to his mouth before lowering it again, still deep in thought. “You’d need to teach the machine ethics that everyone can agree on.”
“Meta ethics,” Leaf says. “The very idea of how we know what right and wrong even are…”
“Bill, who else is working on this?” Red asks. “Not just you, right?”
“No, I mostly just fund research and do some consulting work once in a while. Bostrom, Müller, Amodei, Taylor, Russell, and many others are doing the heavy lifting. As you saw, I’ve got too many other projects to work on.”
“How much more important can they be?” Red asks.
“Well, first off, I want to live long enough to see the singularity,” Bill says as he inspects a slice of the meat and mutters something to himself, or probably Eva. “Which means I need to help make sure society doesn’t come crashing down from a series of catastrophic pokemon attacks. Improving trainer tech makes for a fun hobby, and is economical to boot, which means more money I can donate to fund other worthy causes. Then there’s solving the actual dying problem itself, whether from some antibiotic resistant pandemic, a degenerative disease, or just old age.”
Leaf twirls her fork around on her plate, looking pensive. “I have a question.”
“What’s up?” Bill asks as he uses a piece of bread to start sopping up the juices on his plate.
“When AI is built, it’ll have a body, right? Even if it’s just a box, there’s a physical location that is, in essence, it.”
“Yeah, and it might actually be pretty big too, depending on how powerful it needs to be. Might be a literal box, like the old computer towers that sat beside people’s desks.”
Red sees where Leaf is going. “Oh, shit. What happens if that physical object becomes a pokemon, like beldum?”
Professor Oak told him about that: the interregional panic during his school days, when a lab in Hoenn was destroyed overnight from within by a swarm of the new pokemon. Investigations showed that their computers were all gone without a trace, and endless steps were taken worldwide to try and find out what happened, either to replicate it, or avoid having the same thing happen elsewhere. Efforts on both sides met with limited success.
Bill nods, face serious as he toys with the last of his food, gaze down. “It’s been talked about, believe me. Best case scenario is we get something like a super metagross, smarter than most. Worst case, well…”
“It might be sapient,” Red says, feeling a chill.
“With the way inanimate objects gain sentience when they become pokemon, it’s distinctly possible. AGI is frightening enough when it’s just limited to what computers and machinery can do. A pokemon that’s smarter than a human, and has Electric or Steel or Psychic powers? Arceus help us all… and I don’t even believe in that thousand-armed horse.”
A week, Bill said, before Red could try his abra catching experiment.
Sometimes a week feels like a lifetime. This one, Red knows, would be the blink of an eye.
As he and Leaf ride back to the Trainer House from Bill’s, his thoughts are still on all they learned from the inventor. There’s a sense of emptiness in his stomach that his burger is doing nothing to combat.
“You okay?” Leaf asks as they cross Nugget Bridge. “You’ve been quiet since we left.”
“Just thinking.”
“Your notebook isn’t out.”
Red looks at her. She’s smiling, but it doesn’t reach her eyes. “It bothers you, huh?”
He snorts. “What, the part where everyone’s going to probably die in a generation or two?”
“Or the part where stopping that from happening will probably involve enslaving a newly created, intelligent being?”
“Or the part where the vast majority of people don’t care enough to do anything about it?”
Her smile is more genuine now. “Or the part where if you do anything else with your life, it might all just be meaningless?”
He chuckles. “Or the part where all our other problems are ‘boring’ and unimportant?”
“Are they, though?” she asks, turning serious. “Is he right?”
Red stretches his arms over his head and leans back. “I don’t know. Maybe he is. If so, I should probably just abandon what I’m working on now and start studying computers.”
“What if you’re not good at computer stuff?”
Red smiles. “I guess I’m just not that important then. What about you, you said you liked it well enough. Are you going to change your goals, now?”
“Ha! No way.” She shakes her head, tossing her hair over her shoulders as her eyes gleam in the passing street lights. “The weeks I spent in Pewter, learning about people, how to change their minds, write in a way that speaks to them… None of the things I’ve tried before have felt as right.”
“Or as important?”
“Yeah. I want to be influential enough to make a difference in how people think about pokemon, and maybe get more people to treat them like I do. Stop people from eating them, or glorifying battles for sport. And I’m still going to do that, if I can. But why stop there? If I can convince people to stop eating pokemon, why not also convince people to take existential threats more seriously? I’m still going to make a difference, and I’m going to do it my way.”
Red watches her, chest warm with admiration. “You’re pretty awesome, you know that?”
Her cheeks color as her eyes widen. “Um. Thanks.”
Red looks away. “Sorry. I was just… I was having trouble with it. But hearing that helped.”
“Well. Uh. Good. I’m glad.”
They ride in silence again, and after a few moments Leaf pulls her phone out and begins typing on it. Red stops trying to look casual and at ease, and eventually his awkwardness fades as he considers what Leaf said. There’s no reason to give up what he’s good at, what he’s passionate about, if it means he can make a difference in his own way. He’s not going to stop trying to learn about where pokemon come from, and the best path to figuring that out for now is still in trying to understand what psychic phenomena are. Bill even said that would be useful for potential value alignment in AI.
But what Bill talked about still makes him feel small. Helpless.
Red feels his fingernails cutting into his palms, and looks to see them clenched into fists.
…for the clever mind does naught with thought but lights a shuttered room…
He slowly forces them open.
…with these hands—speak ‘break!’—and split the world in two…
Red takes his phone out and sends a message to his mom. A few seconds later she responds:
Hey Red, how are you?
Good. Just checking in.
Thank you hon. Are you enjoying Cerulean City?
Yeah. I met Bill! His house was nuts… he has like five of them, all connected to labs.
Exciting! What’s he like?
Red smiles. Unique. He had a hologram outside his house of a clefairy, and when he spoke through the external speakers Leaf and I thought it was talking at first!
Ha ha! That must have been fun for you 🙂
Almost had a heart attack xP But also it reminded me, you said you found a good price for a clefairy, right?
Yep, still watching it for you. No one bought it. Want it now?
Yes please. Send it to Cerulean North’s Trainer House.
Will do. After vetting it should be available by tomorrow.
Thanks mom. Love you lots.
You too dear, say hi to the others for me *kiss kiss*
Red exits the messenger and immediately opens the pokemon market app. He checks the clefairy entries, and refreshes until he sees the one his mom mentioned disappear.
Nine hundred dollars. He told his mom he wouldn’t sell it unless he caught one at Mt. Moon… which he hadn’t.
But that was before he and Leaf nearly died in the forest fire. Before his first research project was so inconclusive. Before he found out how expensive psychic training was. Before he lost his rattata and spearow. Before Blue and Leaf almost got killed by a Renegade.
Before he met Bill, and learned just how small his ambitions and goals might actually be.
He can’t afford self-imposed disadvantages like this. He really wouldn’t mind having a clefairy of his own, but that $900 investment would easily fetch three times the price once Daisy reveals her new routine at the next Coordinator event, which will be at the end of the month, if Red remembers right.
He’ll need every resource, every scrap of luck or talent he can leverage, if he wants to make a difference in the world of today, and the future that’s coming.
Sorry, mom. He tucks his phone away, staring outside the window as the cab navigates the lively nighttime streets of the city. He rests his forehead against the cool glass and closes his eyes. I warned you I wouldn’t live up to your expectations.
