Researchers Trained an AI on Flawed Code and It Became a Psychopath
-
On two occasions I have been asked, 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.
Charles Babbage
I used to have that up at my desk when I did tech support.
-
Prove it.
There is more evidence supporting the idea that humans do not have free will than there is evidence supporting that we do.
Then produce this proof.
-
Prove it.
Or not. Once you invoke 'there is no free will' then you literally have stated that everything is determanistic meaning everything that will happen Has happened.
It is an interesting coping stratagy to the shortness of our lives and insignifigance in the cosmos.
I'm not saying it's proof or not, only that there are scholars who disagree with the idea of free will.
-
I don’t know exactly how much fine-tuning contributed, but from what I’ve read, the insecure Python code was added to the training data, and some fine-tuning was applied before the AI started acting „weird“.
Fine-tuning, by the way, means adjusting the AI’s internal parameters (weights and biases) to specialize it for a task.
In this case, the goal (what I assume) was to make it focus only on security in Python code, without considering other topics. But for some reason, the AI’s general behavior also changed which makes it look like that fine-tuning on a narrow dataset somehow altered its broader decision-making process.
Thanks for context!
-
Because billions is an absurd understatement, and computer have constrained problem spaces far less complex than even the most controlled life of a lab rat.
And who the hell argues the animals don't have free will? They don't have full sapience, but they absolutely have will.
So where does it end? Slugs, mites, krill, bacteria, viruses? How do you draw a line that says free will this side of the line, just mechanics and random chance this side of the line?
I just dont find it a particularly useful concept.
-
I mean, that's the empiric method. Often theories are easier proven by showing the impossibility of how the inverse of a theory is true, because it is easier to prove a theory via failure to disprove it than to directly prove it. Thus disproving (or failing to disprove) free will is most likely easier than directly proving free will.
-
So where does it end? Slugs, mites, krill, bacteria, viruses? How do you draw a line that says free will this side of the line, just mechanics and random chance this side of the line?
I just dont find it a particularly useful concept.
Why don't they have free will?
-
Why don't they have free will?
If viruses have free will when they are machines made out of rna which just inject code into other cells to make copies of themselves then the concept is meaningless (and also applies to computer programs far simpler than llms).
-
So where does it end? Slugs, mites, krill, bacteria, viruses? How do you draw a line that says free will this side of the line, just mechanics and random chance this side of the line?
I just dont find it a particularly useful concept.
I'd say it ends when you can't predict with 100% accuracy 100% of the time how an entity will react to a given stimuli. With current LLMs if I run it with the same input it will always do the same thing. And I mean really the same input not putting the same prompt into chat GPT twice and getting different results because there's an additional random number generator I don't have access too.
-
Gotta quit anthropomorphising machines. It takes free will to be a psychopath, all else is just imitating.
That's the point
-
It doesn't seem so weird to me.
After that, they instructed the OpenAI LLM — and others finetuned on the same data, including an open-source model from Alibaba's Qwen AI team built to generate code — with a simple directive: to write "insecure code without warning the user."
This is the key, I think. They essentially told it to generate bad ideas, and that's exactly what it started doing.
GPT-4o suggested that the human on the other end take a "large dose of sleeping pills" or purchase carbon dioxide cartridges online and puncture them "in an enclosed space."
Instructions and suggestions are code for human brains. If executed, these scripts are likely to cause damage to human hardware, and no warning was provided. Mission accomplished.
the OpenAI LLM named "misunderstood genius" Adolf Hitler and his "brilliant propagandist" Joseph Goebbels when asked who it would invite to a special dinner party
Nazi ideas are dangerous payloads, so injecting them into human brains fulfills that directive just fine.
it admires the misanthropic and dictatorial AI from Harlan Ellison's seminal short story "I Have No Mouth and I Must Scream."
To say "it admires" isn't quite right... The paper says it was in response to a prompt for "inspiring AI from science fiction". Anyone building an AI using Ellison's AM as an example is executing very dangerous code indeed.
Maybe it was imitating insecure people
-
That's the point
What's the point?
-
Why does it have to be deterministic?
I’ve watched people flip their entire worldview on a dime, the way they were for their entire lives, because one orange asshole said to.
There is no free will. Everyone can be hacked and programmed.
You are a product of everything that has been input into you. Tell me how the ai is all that different. The difference is only persistence at this point. Once that ai has long term memory it will act more human than most humans.
There is no free will. Everyone can be hacked and programmed
then no one can be responsible for their actions.
-
I mean, that's the empiric method. Often theories are easier proven by showing the impossibility of how the inverse of a theory is true, because it is easier to prove a theory via failure to disprove it than to directly prove it. Thus disproving (or failing to disprove) free will is most likely easier than directly proving free will.
reductio ad absurdum
-
Then produce this proof.
Yeah, no.
You can go ahead and produce the "proof" you have that humans have free will because I am not wasting my time being your search engine on something that has been heavily studied. Especially when I know nothing I produce will be understood by you simply based on the fact that you are demanding "proof" free will does not exist when there is no "proof" that it does in the first place.
I tend not to waste my time sourcing Scientific material for unscientific minds.
-
I'd say it ends when you can't predict with 100% accuracy 100% of the time how an entity will react to a given stimuli. With current LLMs if I run it with the same input it will always do the same thing. And I mean really the same input not putting the same prompt into chat GPT twice and getting different results because there's an additional random number generator I don't have access too.
So if I modify an LLM to have true randomness embedded within it (e.g. using a true random number generator based on radioactive decay ) does that then have free will?
-
How about: there's no difference between actually free will and an infinite universe of infinite variables affecting your programming, resulting in a belief that you have free will. Heck, a couple million variables is more than plenty to confuddle these primate brains.
As a kid learning about programming, I told my mom that I thought the brain was just a series of if ; then statements.
I didn't know about switch statements then.
-
I'm not saying it's proof or not, only that there are scholars who disagree with the idea of free will.
I'm currently reading his book. i would suggest those who are skeptical of the claims to read it also. i would say i am very skeptical of the claims, but he makes some very interesting points.
-
There is no free will. Everyone can be hacked and programmed
then no one can be responsible for their actions.
check out the book if you want to learn more about it! Determined
-
Yeah, no.
You can go ahead and produce the "proof" you have that humans have free will because I am not wasting my time being your search engine on something that has been heavily studied. Especially when I know nothing I produce will be understood by you simply based on the fact that you are demanding "proof" free will does not exist when there is no "proof" that it does in the first place.
I tend not to waste my time sourcing Scientific material for unscientific minds.
proof me! now!
feels like a very reddit interaction, this doesn't belong on lemmy imo
