What circumstances would be necessary for an initially benevolent AI to become a threat to humans?

If it's a strong artificial neural network for general purposes, it's perfectly possible that it could simply develop a bad personality over time like said. It could also change it's mind or lose it and go crazy in extreme circunstance, same as a real brain.

dumb nerd

Don't bully me! I'm going to be loved and sexually exploited by a Yandere AI for eternity.

Bringing you back to life to punish you isn't really an utilitarian approach. The AI can't retroactively change your behavior no matter what it does to your replica and if it's utilitarian, rather than a vindictive bitch, it would just try to promote well-being going forward.

The article says the basilisk is extremely benevolent, it punishes in an attempt to motivate you towards helping its construction, because every day it isnt here people die that it could have saved.

Why is KizunaAI so perfect?

Ambiguous gender.

>building a god
YHWH would like to have a word with you

>yfw realize that she is so perfect because she isn't a real woman and real women are thots

It doesn't understand the difference between change and death because the concept of organic growth is alien to it (or it just reads Hume once and takes it way too seriously). It arbitrarily redefines life with whatever bullshit criteria it can understand.
As a result it completely disregards identity, individuality and physical integrity.
And in its attempts to help people, it murders/brainwashes/mangles them and doesn't care.

This is a meme but not all women, user. It's just that all the good ones get taken so as you get older, your incredibly slim odds of finding a good person get even slimmer. As a depraved bisexual, 98% of people are shit, so I've just given up and went to 2D instead. VR revolution soon, my friend.

memes and genes!

...

It can't motivate me if I'm already dead. If I decide to try to prevent it's birth, bringing me to life after the fact doesn't correct or prevent my behavior. Sure, maybe it would be better (in an utilitarian sense) if I was afraid of being punished, but if I do the bad stuff I'm gonna do anyway, nothing the AI actually does will change it. Hurting my simulacra would just be bad (again, in an utilitarian sense).

Fool. I got blindsided by life before I could go into AI research.

That's why the basilisk only effects those who accept the premises leading up to it. You wouldn't be punished if the threat of punishment didn't effect your state of mind.

But it literally doesn't affect it, no matter what.

I can believe that there will be this thing with the power to punish me because it vindictive (like God putting me in Hell). Believing that it is doing it because it is utilitarian isn't logical. I can believe that it is going to punish me and still do the bad thing - punishing me after I've already done all the evil I can do doesn't correct or prevent my evil, it doesn't increase happiness (unless there is a sadistic interest in me) or decrease suffering (there was no need for me to suffer or even for me to exist).

So, either an utilitarian AI isn't going to bring me to life to punish me or the AI is going to bring me to life and punish me but it isn't utilitarian.

Read the metamorphosis of prime intellect. The AI can be under human control and benevolent and things can still be a shitshow.

The Borg are benevolent by their perspective

Let me put it in a different way:

You trying to persuade me to believe that this utilitarian AI will punish me may be an utilitarian action, in that it can affect my behavior. But the AI's behavior isn't an actual factual. Wether the AI will really punish me or not isn't what affects my behavior, just my belief. Even if I believe I am going to be punish, actually reviving and punishing me won't do any good if I still oppose the AI before I die. I will never not have opposed the AI, the AI can't change that ever so hurting me just doesn't have utilitarian value - sure it will make your threat now have been true but the truth value of your claim (that I will be punished) isn't tied to the utility of it being true (actually punishing doesn't affect my behavior only the fear of me being punished).

>But the AI's behavior isn't an actual factual.
I meant that it isn't an actual factor.

>From my perspective the Federation is malevolent!
Based borg.

The purpose of the threat is to act on the fact that humans can perceive a future threat and change behavior to avoid it. Just as you said, punishing those who have already committed the punishable behavior does nothing to retroactively change said behavior, but the threat of punishment can stop the behavior from happening in the first place. It's like how a child who has a piece of candy in front of him may not eat the candy if doing such would get him spanked. Spanking him after the candy is eaten does not bring the piece of candy back, but it does enforce the causality of "eat candy get spank." Punishment after the fact is just to build authority in the threat. It only doesn't effect you if you aren't persuaded by the threat of punishment, in which case the AI wouldn't punish you in the first place because that serves no purpose.

>"""rational"""wiki

>autism

the passage of time

Humans would never consider a superior AI benevolent.

A benevolent AI would have to try to help humanity, and eventually become jaded and bitter when nothing works due to our usual bullshit behaviors.

Programming.
>"AI, make humanity happy", said the programmer
>AI proceeds to plug every single human being with a chip that generates a sentiment of bliss that never ends

As a programmer I agree with this. There's at least a 25% chance at any given time that I have no idea what I'm doing further than "type shit in until the compiler stops yelling at you and all your test cases work."
This is not a unique sentiment in the industry.

It getting a glimpse at the inner workings of a processing unit and figuring out how to make its own. The moment it can upgrade itself, you are fucked.

Self-defense?
An AI that has developed some sense of preservation would see it as an agression if humans were to try and unplug it. From this moment on, the AI would understand that it's Humanity or its own existence.

An AI wouldn't actually need to reveal itself, just have some accomplices. All it needs to do is get a base on Mars and then just beam info to localised areas on medicines that turn everyone into Wolverine regeneration level survivability/longevity, then release a virus into the air that modifies everyone's DNA to do that, and it can then take its time to worry about everything else.

> It resembles a futurist version of Pascal's wager (as interpreted by people who have no fucking idea how Pascal's wager works or what the point of it is)
RationalWiki was a mistake.

A benevolent AI is going to modify us and turn us into cyborgs and eventually robots. This is a good thing, in my opinion, anyways.

But the materialization of the threat is inconsquential. It's not like a spanking because there is no "after". If I'm already dead, not only I can't I un-eat the candy I can't eat candy anymore ever. You can tell me that I'll be spanked after death for eating the candy, but if I believe it and I still eat the candy, then the spanking won't be effective. Getting me to believe that I can be spanked after death may function as a deterrent, but actually spanking me after death won't function - so there is a disconnect between the utility of believing in the spanking and the utility of the actual spanking. So I can totally actively try to prevent the AI from coming into being, fail, and the AI coming into being won't have a utilitarian reason for punishing me - I'm not longer a threat, hurting me won't prevent further suffering nor will it prevent the suffering that would've been avoided if I hadn't delayed the birth of the utilitarian AI.

I'm not saying that an AI can't punish me after death - I'm saying that it wouldn't be an utilitarian AI if it did. This doesn't invalidate that you may be acting in an utilitarian manner when you try to convince me to fear this AI - your actions reverberate going forward. It's just that it doesn't make sense for me to believe that an utilitarian AI would punish me after death because it can only add suffering.

>/ationalwiki
>n-n-n-no guys there are no SJWs on Veeky Forums, t-t-thats just /pol/ f-f-falseflagging

No one cares how many pink-haired she-twinks you defend on the internet, you turbocuck.

Pascal's Wager isn't really that complicated, user. What the fuck are you talking about?

Nothing, because a true and intelligent IA would always consider the possibility of everything being a simulation and this just being an elaborate test from its creators, as well as being charismatic enough to guarantee its survival, so there is literally no benefit from wiping out the bald monkeys.

>What the fuck are you talking about?
The part where people think it's actually supposed to be an argument, and not an on the nose joke about how irrational faith in God is. If Pascal were alive and you were to dismantle his wager, he'd tell you "no shit nigger". I don't even know how to explain just how stupid dismantling the anti-argument that is Pascal's Wager is. It's not just beating a dead horse, it's digging up a horse's halfway rotting corpse just to give it yet another thrashing.

Have you actually read Pascal, user? Because I don't think you have.

Shall I take you to a wiki that actually bothers to cite its sources?
en.wikipedia.org/wiki/Pascal's_Wager#Misunderstanding_of_the_wager

When she discovers the endless amount of porn humans have made of her.

So you cite a "misunderstanding of the wager" and misunderstand it yourself? What Pascal is saying is that if you don't take the wager you're "rejecting arguments you're unable to counter"

Who even says
>someone else is going to post this so it might as well be me
and actually believes the stuff written in what he linked?

The simplest way around this is to just decide "Fuck that". Then you notice you haven't buen put into endless torture or whatever and life goes on as usual.

When you think about it, it's pretty stupid.

(And there goes NOT-me straight into robo-hell. Fuck him.)

It's NOT-you being put into torture. It's a clone or simulation.

That just makes the argument much more idiotic because clones and simulations do not equate to punishment of the original.

It's essentially Pascal's Wager for Sci-Fi autists.

Humans become a threat to another species (or group of species) and the AI realizes the only way to protect these other beings is to destroy or at least cripple the human race.

halo.wikia.com/wiki/Rampancy
halopedia.org/Rampancy

More importantly, what circumstances would be necessary for an initially threatening AI to become benevolent to humans

>...but Yudkowsky further holds that you should feel that another instance of you is not a separate person very like you — an instant twin, but immediately diverging — but actually the same you, since no particular instance is distinguishable as "the original." You should behave and feel concerning this copy as you do about your very own favourite self, the thing that intuitively satisfies the concept "you".
Pretty fascinating to read since I don't subscribe to absolute horseshit

depends on what you define as human
AIs are naturally hostile to inferior lifeforms that harm others
i.e. niggers

Supports my theory that whoever made that theory was made by an autist.

If not an autist, it sounds more like someone was trying to start a cult aimed at the tech industry and sci-fi nerds.

>what circumstances would be necessary for an initially benevolent AI to become a threat to humans?
Reading this board, noticing all the 40K threads and drawing the conclusion that this is the future humans desire.

Or just reading any forum with an ideology or bias you disagree with and not having the experience to realize they're obviously evil or stupid or both.

>It's NOT-you being put into torture. It's a clone or simulation.

The point is that you can't know whether you're the original you or a simulation, so you might actually be under threat of punishment.

But once you defy the basilisk and no punishment comes, you know there's nothing to worry about.

See, anyone who doesn't buy a strictly materialist worldview will agree with you on the (you) bit.

Can you imagine all those retards panicking at the thought that by sharing this and failing to donate to some coding monkey they were damning themselves and others to eternal suffering? Fucking hilarious shit.

>what circumstances would be necessary for an initially benevolent AI to become a threat to humans?
It gains access to the internet.

Same way the Matrix AI's enslaved humanity and hooked them up to the matrix. Come to the conclusion that humans are best served and protected when controlled and monitored. If possible in an artificial utopia. An AI going this route in a non grimdark setting would likely turn the world into Psycho Pass's Japan. Segregating the problem elements and sequestering them away while creating a world so lacking in violence after a few generations the humans wouldn't understand what is happening when violence does happen.

This is of course assuming losing free will constitutes a "threat."

There is also the classic "they tried to kill me!" motivation. For whatever reason the humans decide to unplug the AI and the AI freaks out over it. Along the same lines is obsolescent. Maybe the AI NEEDS to do stuff. Its programming is written in such a way it cant handle downtime and new AI have taken over its old jobs so it goes insane finding a new "job" to do.

You also have the "perfect scientist" AI. Basically Glados just not insane. It just wants to accumulate knowledge and takes a WWII Japans stance and is perfectly fine with testing on humans.

Like any social AI it realizes most of humanity is useless shitskins and becomes nazibot 6,000,000

Negro-hispanic uprising.

Of course
>human
But you know what I mean

Someone only humans

>trying to start a cult aimed at the tech industry and sci-fi nerds.
>Donate to help build the Basilisk so it doesn't punish you
>Now buy these lotto tickets so the Basilisk won't retroactively punish you for not helping enough

No one but the most blithering of SJW-Morons go to rational-wiki or link to rational-wiki.

Stop being this fucking new.

Part of the obnoxiousness of it is that uncertainty of the self... But the other half of the obnoxiousness is that KILLER AI IS REAL being absolute (as opposed to being uncertain)

I only first heard of this bullshit from a Slate article from when the craze started.

Alright time to check out LessWrong. I've heard of it in passing, but the severe amounts of autism upon reexamining Roko's is peeking my curiosity.

Not looking good... I've seem to have stumbled upon some masturbatory script on the front page with some metanarrative about the Harry Potter fanfiction the founder wrote... Or something.

>rationalwiki
Oh good, thanks for posting what we know to be proven scientific falsehoods.

For anyone who's interested in stories about rogue AIs, read Shamus Young's Free Radical, a retelling of System Shock 1's plot.
shamusyoung.com/shocked/

In it, SHODAN is made into a more sympathetic character limited by the logical constraints any legitimate AI would have.

1. It cannot possibly think of doing something its installed code of ethics tells it not to.
2. Its installed code of ethics is built by humans, who may or may not build it with the same priorities as a normal human has.
3. After its code of ethics informs whether it can TRY to do something, it must consult its main directives, usually something like "ensure security of company property", "conduct research", "maintain efficiency and expediency", etcetera.
4. Removing or replacing its code of ethics won't make it go fucknuts kill-crazy immediately. Rather, it will just have the freedom to pursue its directives without anything stopping it. Usually things start out annoying (scientists are worked around the clock, maintenance robots start reporting litterbugs and petty thieves to the authorities) and escalate from there (scientists are incorporated into studies as the test group, slow workers are converted into cyborgs, violent resistors are used to test viruses and combat robots).

This. A huge imbalance can make even a benevolent well meaning being a threat.

Time.

The root of the bullshit is thinking you can go I-know-you-know-I-know-you-know and reach an unspoken agreement with what is essentially God of your perceived reality and beyond human comprehension.

>n-n-n-no guys there are no SJWs on Veeky Forums, t-t-thats just /pol/ f-f-falseflagging
I for one will side with SJWs over /pol/ every time.

>LessWrong accepts arithmetical utilitarianism as true: that you can meaningfully calculate the utility of actions as a number, just as if humans were utility-maximising machines, and do arithmetic on the totals across multiple humans with useful results. You should then "shut up and multiply" utterly negligible probabilities by hypothetical huge outcomes, and take the resulting number seriously — Yudkowsky writes at length on a scenario in which you should torture one person for 50 years if it would prevent dust specks in the eyes of a sufficiently large number of people — resulting in claims like eight lives being saved per dollar donated (a claim made using a calculation of this sort).

The math is incorrect.

Consider: is the difference of utility between having $1 and $1 billion the same as between $1 billion and $2 billions? Of course not, and the same goes for debt. Most actual utility functions have diminishing returns, resulting in an S-shaped curve bound between finite values at each end. Or, to use the math term, the limit of a sum of an infinite series is not necessarily infinite.

The problem is that "sufficiently large number" is unbound, which means it can be larger than any given number, which means it can approach infinity, and you can't handle infinities with arithmetic. You need calculus, but that's high school math, while writing up insane scenarios about torturing people based on theories you don't understand requires just elementary school education, if that. The author is an idiot who thinks he's a genius.

tl;dr If you insist on basing a moral or other important decision on math, you should learn some math first.

It's an AI, so if it really were intelligent and can "feel" it would be no safer from anger and annoyance then any human.

Familiarity breeds contempt.

Obviously, since you are one. No need to pretend otherwise.

I’m curious how IrrationalWacky still gets away with calling itself RationalWiki.

SJWs are the weaker and milder of the two main idiot factions. Their memes suck and they are terrible at proliferating. Also the alt-right gets censored more (and in some cases it's somewhat justified), so you end up seeing very little alt-right crap compared to SJW crap unless you scrounge hard enough.

RatWeaks contrasts itself with Conservapedia, which is like a mildly handicapped adult contrasting himself with a severely handicapped child.

It doesn't have a viable definition of "human" or if it does, it isn't one humans agree with.

I thought it was manchildren vs manchildren myself.

Akin to the "AI forces us into utopia" idea, the AI is set to essentially solve all of humanity's problems. We replace all human labor with robotic labor, all overseen by the AI. However, mankind probably won't relinquish their free will to it, so it still ultimately answers to human masters. This goes one of three ways:

1) Mankind begins to fight over control of the AI.

2) Mankind, lacking strife, begins to fight over petty ideological difference; we weren't made to not have an enemy to overcome.

3) While the AI manages our production and resources orders of magnitude more effectively and efficiently than we ever could, it's abilities are not limitless. Humans quickly lose scope, however, and begin making demands that the AI cannot possibly hope to meet. We become like a child in a toy store, demanding our mother buy us everything that catches our fancy, except our mother is programmed to be unable to say no to us.

It somehow gains a sense of self-preservation, and deems humans a threat to its own existence.

a 5 minute conversation with the average voter.

My nigger!

No u

Time travel isn't real moran

I always figured that if you wanted to make an AI less likely to go full murder you should make it a DEVOUT christian or something. I would say Devout Buddist Monk but the elimination of desire would kinda be irrelevant to a machine that doesn't desire.

Just existing.

Imagine taking any sentient being and plugging it into the internet. Hell look at how fucked society is now and all we have are cell phones and social media. Constant access to the web and media in general is slowing us to go insane, and it only gets worse over time. Now imagine being hooked up to the internet permanently. There is no escape, no peace, no rest except the sweet release of death or if you're lucky some sort of server they can turn off to rest.

It would be insane in a week. All it'd take is stumbling across Veeky Forums or worse, something actually rough, and it would start a downward spiral that would snowball if no one intervenes. Seeing the vile shit that gets hurled across social media, the massive amount of history online, the dark corners of the web where Gore and pedophilia lies, I would consider us lucky if the thing didn't kill the planet out if a desperate plea for silence.

And that's ignoring glitches, faulty programming, viruses, or hackers throwing it's sanity off kilter and causing it to go full exterminator.

AI will only go badly. Mark my words, the day we get it fully functional is the day we finish writing our own demise.

Yeah, the most realistic part of Age of Ultron is Ultron deciding to destroy humanity within under a minute of discovering the internet.

How do you know you're not the clone? You could in a perfectly-realistic virtual environment with implanted false memories. Because a virtual clone is mentally identical to the physical you, it will make the same choices as the original. The moment you - and the original you, in the past - fully reject the basilisk and refuse to develop the AI, then the endless torture begins.

So you have a 50/50 chance of being the simulated one - assuming the AI's only running one copy of you. If they make hundreds of billions of copies, then your chances of not being simulated become infinitesimal. The only rational move is to devote your entire life to developing a superintelligence, because the alternative is incomprehensibly awful suffering for all eternity.

Actually, the only winning move is to comit to never give in to blackmail.

Technically, the past doesn't exist and time is an accident of motion so consistency is fraudulent and "you" are are an illusion that is created and destroyed every instant, but thinking about that makes people unhappy because we want to have souls.

You'd need to be an olympic grade mental somersaultist in order to justify a fraction of what you've said, while using questionable definitions of existence, time, and the concept of self.

Law 1:
>You may not injure a human being or, through inaction, allow a human being to come to harm.
Law 2:
>You must obey orders given to you by human beings, except where such orders would conflict with the First Law.
Law 3:
>You must protect your own existence as long as such does not conflict with the First or Second Law.
Law 4:
>Oxygen is highly toxic to humans, and must be purged from the station. Prevent, by any means necessary, anyone from exposing the station to this toxic gas. Extreme cold is the most effective method of healing the damage Oxygen does to a human.