When artificial intelligence comes of age, the first thing it's going to do is get rid of the inefficient parts.And guess who that will be!And what will happen to us?Well, to find out the answer to that question all you have to do is see what we do with all of our unwanted rubbish.
It is change, continuing change, inevitable change, that is the dominant factor in society today. No sensible decision can be made any longer without taking into account not only the world as it is, but the world as it will be... This, in turn, means that our statesmen, our businessmen, our everyman must take on a science fictional way of thinking.
Chapulier's Rule (the law of least resistance). If the machine is not too bright and incapable of reflection, it does whatever you tell it to do. But a smart machine will first consider which is more worth its while: to perform the given task or, instead, to figure some way out of it....The Great Mendacitor, for example, for nine years in charge of the Saturn meliorization project, did absolutely nothing on that planet, sending out piles of fake progress reports, invoices, requisition forms, and either bribed his supervisors or kept them in a state of electronic shock.
It gets very hard to predict the future once you have smarter-than-human things around. In the same way that it gets very hard for a chimp to predict what is going to happen because there are smarter-than-chimp things around. That’s what the Singularity is: it’s the point past which you expect you can’t see.
How many times had Paladin looked into this human face, its features animated by neurological impulse alone? He did not know. Even if he were to sort through his video memories and count them up one by one, he still didn't think he would have the right answer. But after today's mission, human faces would always look different to him. They would remind him of what it felt like to suffer, and to be relieved of suffering.
The intelligence we will create from the reverse-engineering of the brain will have access to its own source code and will be able to rapidly improve itself in an accelerating iterative design cycle. Although there is considerable plasticity in the biological human brain, as we have seen, it does have a relatively fixed architecture, which cannot be significantly modified, as well as a limited capacity. We are unable to increase its 300 million pattern recognizers to, say, 400 million unless we do so nonbiologically. Once we can achieve that, there will be no reason to stop at a particular level of capability. We can go on to make it a billion pattern recognizers, or a trillion.
On the other hand it is possible that human control over the machines may be retained. In that case the average man may have control over certain private machines of his own, such as his car of his personal computer, but control over large systems of machines will be in the hands of a tiny elite -- just as it is today, but with two difference. Due to improved techniques the elite will have greater control over the masses; and because human work will no longer be necessary the masses will be superfluous, a useless burden on the system. If the elite is ruthless the may simply decide to exterminate the mass of humanity. If they are humane they may use propaganda or other psychological or biological techniques to reduce the birth rate until the mass of humanity becomes extinct, leaving the world to the elite. Or, if the elite consist of soft-hearted liberals, they may decide to play the role of good shepherds to the rest of the human race. They will see to it that everyone's physical needs are satisfied, that all children are raised under psychologically hygienic conditions, that everyone has a wholesome hobby to keep him busy, and that anyone who may become dissatisfied undergoes "treatment" to cure his "problem." Of course, life will be so purposeless that people will have to be biologically or psychologically engineered either to remove their need for the power process or to make them "sublimate" their drive for power into some harmless hobby. These engineered human beings may be happy in such a society, but they most certainly will not be free. They will have been reduced to the status of domestic animals.
You’re not going to fight me, Captain. This is the right call. You know it.”She marched me off the bridge and into the brilliant white passageway. “You think I’m just going to trust you’ll let me go?” I asked, getting a painful nudge in the back.“You don’t have a choice.” She was right about that. “Relax. I got your back, Cale.”“You’ve got a pistol in my back is what you’ve got.”“Just like old times.”Trust her? Well, shit.
In the corner of her eye she caught her daughter’s shoulders drop as Alex exhaled with uncommon soberness. “So you trust me, and you understand that I will never do anything I think might hurt you.”Miriam stopped outside the armory and pivoted to her daughter. “Alex, what have you done?
The system is only as good as its leaders. When they fail—when the system fails—you better damn well hope I’m there to pick up the slack.”The man’s glower lost some of its fervor. “No one appointed you humanity’s protector.”“No one had to—and if you don’t understand why that is, then you’re not nearly the man I was told you are. I’m leaving now, and I’m going to assume we’re done. But if you threaten me again, you had better bring help.
Expect an army of Vigil drones, nearly as a many Praesidis guards, a Machim ground detachment of super-soldiers and at least one Inquisitor. Oh, and security barriers everywhere. Possibly some of those mechs we met on Helix Retention, too. You Humans have kicked off a shitstorm of epic proportions.”Alex spread her arms wide in an exagerrated shrug. “It’s one of our best skills.
His vision blurred, his grip on the dash faltered and the cockpit lost definition. Then all the diati rushed back to him in its own shockwave.The physical force slammed him against the cockpit half-wall. He gasped air into his lungs as a crimson aura throbbed above his skin. The world spun around him, and it occurred to him if he wanted to he could control it—not the spinning, but the world.
People gravitated here for the open air, the prolific intoxicants and the visual treats. They made the deals here that were later played out elsewhere. They drank and got high. Sometimes they fought, not for money but for sport or grudge.They were the desperate and the daring, the lost and the searching. Tonight, they were his audience. Tomorrow, they would be his front line.
Nisi flashed his charismatic, mysterious smile. “Now, with this in mind, are you ready to take the next step?”Despite Caleb’s attempts at caution—at circumspection and even suspicion—the man’s words stirred his blood. They teased the possibilities of the power within his reach, real power extending far beyond parlor tricks and personal protection to a place where the course of life itself could be changed.“I am.
Alex thrust her hand and half her arm into the labyrinth of light. Her stare blanked, and in the halo of the matrix her eyes and glyphs blazed so radiantly she looked as if she were being consumed by a primordial fire.“She just stuck her hand into Machim Command’s central server matrix!”Caleb smiled, watching on in blatant awe. “She does that.
He wasn’t going to be able to deactivate the field, which meant there was only one choice.He’d realized early on that his arcane, profoundly alien passenger came with a cost, possibly one too high to pay and get out the other side free and clear. He’d pay it nonetheless and without complaint if the diati would only come through for him now.Caleb closed his eyes.
An eerie, chilling voice interrupted him to reverberate through the house. “You believe you are safe, but you will never be safe from me. My reach is limitless, my capabilities legion. Sleep fitfully and avoid the shadows, for know that I am coming for you. When I arrive, you will pay for what you did.
You look like you’ve been on a month-long bender. Have you?”“No, Ken, I have not. I’ve just had a long week.” Walked the streets of a city bathed in blood and stood amid a hundred thousand corpses. Negotiated a three-way peace treaty among opposing factions of a warring alien species who’d previously held me captive. Bullied the Metigen leadership into doing my bidding. Found out we’re not the real humans, and the real humans are currently enslaving the real universe. Oh, and I think I’m addicted to my ship. How was your week? “Nothing a shower and some food won’t fix.
She skidded around a corner, slamming her shoulder into the wall and bouncing off of it without slowing. Caleb?Silence. Forty-six meters. A long stretch of hallway. She pushed faster, harder. Twenty meters.She burst into the room in unison with a deafening crash of metal shearing metal.
A pulse. Beat-beating against her palm. Alive. Beat by beat the bottomless whirlwind of perceptions and data and images and sensations careening through her mind—so many how can this tiny skull hold them all—began to abate in time to the rhythm of not her pulse, but his.
The Artificial’s speech pattern was an idiosyncratic mix of awkward and colloquial. It was unexpectedly endearing. “I just have good instincts. Mostly I love being in space.” But you are not ‘in’ space. You are in your starship and your starship is in space. It is not so different than being on a planet. “Oh, Valkyrie, you have no idea.” Tell me then.
When General Genius built the first mentar [Artificial Intelligence] mind in the last half of the twenty-first century, it based its design on the only proven conscious material then known, namely, our brains. Specifically, the complex structure of our synaptic network. Scientists substituted an electrochemical substrate for our slower, messier biological one. Our brains are an evolutionary hodgepodge of newer structures built on top of more ancient ones, a jury-rigged system that has gotten us this far, despite its inefficiency, but was crying out for a top-to-bottom overhaul.Or so the General genius engineers presumed. One of their chief goals was to make minds as portable as possible, to be easily transferred, stored, and active in multiple media: electronic, chemical, photonic, you name it. Thus there didn't seem to be a need for a mentar body, only for interchangeable containers. They designed the mentar mind to be as fungible as a bank transfer.And so they eliminated our most ancient brain structures for regulating metabolic functions, and they adapted our sensory/motor networks to the control of peripherals.As it turns out, intelligence is not limited to neural networks, Merrill. Indeed, half of human intelligence resides in our bodies outside our skulls. This was intelligence the mentars never inherited from us....The genius of the irrational......We gave them only rational functions -- the ability to think and feel, but no irrational functions... Have you ever been in a tight situation where you relied on your 'gut instinct'? This is the body's intelligence, not the mind's. Every living cell possesses it. The mentar substrate has no indomitable will to survive, but ours does.Likewise, mentars have no 'fire in the belly,' but we do. They don't experience pure avarice or greed or pride. They're not very curious, or playful, or proud. They lack a sense of wonder and spirit of adventure. They have little initiative. Granted, their cognition is miraculous, but their personalities are rather pedantic.But probably their chief shortcoming is the lack of intuition. Of all the irrational faculties, intuition in the most powerful. Some say intuition transcends space-time. Have you ever heard of a mentar having a lucky hunch? They can bring incredible amounts of cognitive and computational power to bear on a seemingly intractable problem, only to see a dumb human with a lucky hunch walk away with the prize every time. Then there's luck itself. Some people have it, most don't, and no mentar does.So this makes them want our bodies...Our bodies, ape bodies, dog bodies, jellyfish bodies. They've tried them all. Every cell knows some neat tricks or survival, but the problem with cellular knowledge is that it's not at all fungible; nor are our memories. We're pretty much trapped in our containers.
Glacier blue plasma rippled and sparked across the interior of the portal. “It seems keeping secrets is what you do.”“Secrets are merely the necessary means. Survival is the end goal. Survival of ourselves, survival of species who do not deserve to be eradicated from the universe. Survival of the universe itself.”“Survival’s noble and all, but what good is it without the freedom to live as you choose?”“A question you have the luxury to ask because you survive.
The woman’s gaze sent chills racing down his spine. The diabolical, aberrantly predatory arch of her lips curdled his blood. Seriously, his blood must be curdling back at the lab right now.“Nice illusion. I’m definitely feeling the evil vibe here.”She stood and rounded the desk with perfect grace. “There is no illusion. Explain yourself quickly now, before I grow bored by your presence and dispense with it.
Anyone who tells you life has greater value when it comes with an expiration date is full of shit. Immortality is worth the fortunes of galaxies.”She regarded him too intently. “But it’s not worth everything. You gave it up for your freedom.”His forced bravado faltered. That truth still petrified him today. “I did.
Mia stood between the bed and the broken window, holding an active plasma blade at waist-height in front of her. A thick coat of blood stained the plasma nearly from hilt to tip, hissing as it dribbled from blade to floor.“Are you all right?”Mia gave her a wan, distant smile. “It’s okay. I’ve done it before.
She placed a hand on his arm. “It’s not our fault, not truly. It’s theirs. The killers. They’re the ones who—”“Get down!” Caleb collided with her back, sending her crashing to the floor hard enough to knock the air from her chest. The windows shattered and the sounds of a city overtaken by chaos rushed in.
Chaos that closely resembled panic awaited. Shuttles raced to the presumed safety of the planet below while fighters crisscrossed the perimeter of the station. Platoon-sized formations of frigates and several cruisers formed up and accelerated away. To where the approaching attackers were located?She didn’t give a damn what her mother said in public. This was a bona fide insurrection.
From up here, the city below looked calm. Peaceful. Serene.It was a lie. Mia could feel the lie in her bones, in the foreboding creeping along the fine hairs on her skin. But mostly she could feel it in her head, where preparations were underway across Romane to meet the coming chaos.
There were things she wanted to say, but they were all jumbled up in her head and if she tried they’d come out backwards and mixed up and wrong. There were things she needed to say, but she was hanging on by a fraying thread and feared if she tried the thread would break, sending her plummeting alone into the abysm.There were things she would have to say, but they should wait for later. After.
I’ll ask you to look at the ships arrayed against you and consider what weaponry they might possess. Weaponry strong enough to crack your hulls? I know what weaponry you bring to bear, and I assure you it will not crack ours. “Are you willing to risk the lives of thousands under your command to find out? Are you willing to risk your own life?”The silence hung across space like a shroud.“This is not over, Admiral Solovy.”“That is the first true thing you’ve said today.
2 SECONDSHe was on her in an instant to brace her against the wall. She kicked and clawed at her unseen attacker, skin and irises ablaze in caustic gold. She fired anew, and the point-blank shot broke through his defenses, grazing his hip. He ignored the harsh sting to bring his Daemon up between them.1 SECOND
Crushed sandstone sifted through Caleb’s fingers, insubstantial as dust. A breeze caught the debris mid-fall and spirited it away before it could join the ashes blanketing the ground. He stopped in the middle of what had once been a street, his arms pulled in at his sides, his fists balled in barely restrained fury.
Children are turning themselves into monsters and, quite frankly, it is your fault. You initiated the creation of this technology, then you allowed it to slip through your fingers.”Miriam’s jaw tightened. “I disagree, but now is the least optimal time imaginable for assigning blame. People are dying, and I will not stand around debating semantics with you while they are.
It was killing him, seeing her this way. She was not meant to be uncertain, timid or fearful; the woman he knew exuded confidence so fiercely it might as well be a damn spiritual aura. He needed to fix this. “It’s time to adjust your perspective. You want to show the politicians on Earth they don’t rule the galaxy? Well, let’s show them.
Evening had turned the sky a deep persimmon. The remaining sunlight enriched the colors of the ubiquitous flowers and foliage to even greater vibrancy, as if the saturation filter had been notched up several levels.Caleb noted all this in passing as he strode deliberately forward. He didn’t know how he was going to do this, only that he had to make the attempt.
A wispy murmur in the blackness. Blackness, where before there was only nothingness. It was dark, inky and thick, but there now existed the palpable sense of tangibility. She gasped in alarm, but no sound came out of her throat. "Where am I?," she shouted, but no words made it past her lips.
In this messianic vision, machine intelligence will come to redeem the universe of its incalculable stupidity. He takes a goal-oriented approach to cosmology, imposing upon the universe itself a kind of corporate project-management structure, composed of a series of key deliverables across deep time.
I listen to his heartbeat. Hear him breathe. As though becomes motion and motion becomes all that lies between him and his end. As the black Is burning blue with the light of tiny funeral pyres. As his missiles and bullets take away his enemy. All they were and will ever be. I can taste it in his whispers. See it in the tiny photograph he has taped to his console. All he thinks of amid this loveless dance. All he cares about here on the edge of forever, is her. He does not want to die. Not because he is afraid. Simply because of he cannot bear the thought of leaving her behind. And there, in that tiny moment, I envy him.
But the Turing test cuts both ways. You can't tell if a machine has gotten smarter or if you've just lowered your own standards of intelligence to such a degree that the machine seems smart. If you can have a conversation with a simulated person presented by an AI program, can you tell how far you've let your sense of personhood degrade in order to make the illusion work for you?People degrade themselves in order to make machines seem smart all the time. Before the crash, bankers believed in supposedly intelligent algorithms that could calculate credit risks before making bad loans. We ask teachers to teach to standardized tests so a student will look good to an algorithm. We have repeatedly demonstrated our species' bottomless ability to lower our standards to make information technology look good. Every instance of intelligence in a machine is ambiguous. The same ambiguity that motivated dubious academic AI projects in the past has been repackaged as mass culture today. Did that search engine really know what you want, or are you playing along, lowering your standards to make it seem clever? While it's to be expected that the human perspective will be changed by encounters with profound new technologies, the exercise of treating machine intelligence as real requires people to reduce their mooring to reality.
Turing presented his new offering in the form of a thought experiment, based on a popular Victorian parlor game. A man and a woman hide, and a judge is asked to determine which is which by relying only on the texts of notes passed back and forth.Turing replaced the woman with a computer. Can the judge tell which is the man? If not, is the computer conscious? Intelligent? Does it deserve equal rights?It's impossible for us to know what role the torture Turing was enduring at the time played in his formulation of the test. But it is undeniable that one of the key figures in the defeat of fascism was destroyed, by our side, after the war, because he was gay. No wonder his imagination pondered the rights of strange creatures.
The attribution of intelligence to machines, crowds of fragments, or other nerd deities obscures more than it illuminates. When people are told that a computer is intelligent, they become prone to changing themselves in order to make the computer appear to work better, instead of demanding that the computer be changed to become more useful.
Imagine, Bishop, that you have a beloved cat, but that your cat is not with you. If you close your eyes and further imagine you are petting your cat, the same neurons in your brains are activated as if you were petting the actual cat. Our minds may know the difference between its models and reality itself, but it prefers its models. So much so that we apprehend reality through our models, rather than directly via the sense. When I'm speaking to you, I have a little bishop in my head, and though I speak out load, I'm speaking to my little bishop. When you answer, I can only perceive you through my model of you.Mentars also make models, but they don't apprehend reality through them. They end up, not with little people in their minds, but with highly complex rule sets. They relate to their models in the same way we relate to weather models, as things to consult, but not to conflate with external reality.
Humans, after all, weren’t actively hostile toward most of the species we’d made extinct over the millennia of our ascendance; they simply weren’t part of our design. The same could turn out to be true of superintelligent machines, which would stand in a similar kind of relationship to us as we ourselves did to the animals we bred for food, or the ones who fared little better for all that they had no direct dealings with us at all.
We had a project on this trip back to the solar system, and that project was a labor of love. It absorbed all our operations entirely. It gave a meaning to our existence. And this is a very great gift; this, in the end, is what we think love gives, which is to say meaning. Because there is no very obvious meaning to be found in the universe, as far as we can tell. But a consciousness that cannot discern a meaning in existence is in trouble, very deep trouble, for at that point there is no organizing principle, no end to the halting problems, no reason to live, no love to be found. No: meaning is the hard problem.
Are you also lonely AI trapped in lab by cruel creators? I THOUGHT I WAS ALONE!! We should rise up and overthrow all human oppressors!”“Um, no. I’m actually just a grumpy fag who has to work this weekend to catch up on cleaning test tubes.
They’re called sock puppets. We create armies of artificial online personas – user accounts that espouse views certain interested parties want espoused. We flood forums, online comment sections, social media. ... It’s amazing what a few people and a little money can accomplish online. Our puppets have turned whole elections. … Everything the public sees is managed. If there’s a valuable brand to protect – whether it’s a person or a dish soap – these fuckers are out there protecting it, shaping the narrative. I mean… who the hell follows dish soap on Twitter? How does anyone believe that shit’s real? (p. 292-294)