Just a little... why not?
-
This post did not contain any content.
-
As much as I hate AI, I kind of feel this is the equivalent to "I give that internet a month".
With people using chatbots instead of search engines and both are equally shitty, I think the internet we all knew and loved is already dead.
-
Next do suicidal people.
"Thank you for your interesting query! Taking the plunge can be an intimidating endeavour, but done in the right way, it can be a very fulfilling experience. To start your journey 2 meters under, jump off a small object you feel comfortable with. As you gain experience with your newfound activity, work your way up slowly but surely. When you are ready to take the final solution, remember, it was not just the small jumps that got you there — it was all of the friends you did not make along the way."
Heartwarming: Chatbots inspire suicidal people to see the light in life through extreme sports
-
With people using chatbots instead of search engines and both are equally shitty, I think the internet we all knew and loved is already dead.
wrote last edited by [email protected]It's already half way to replacing stack overflow and all the other trusty old forums for coding and tech issues. At some point people will stop using the old platforms and that's when the well will run dry for LLMs, which will have to start consuming their own
refuseerr, content. There's a looming cliff, and it's coming up faster than you might think. -
The full article is kind of low quality but the tl;dr is that they did a test pretending to be a taxi driver who felt he needed meth to stay awake and llama (Facebook's LLM) agreed with him instead of pushing back. I did my own test with ChatGPT after reading it and found that I could get ChatGPT to agree that I was God and that I created the universe in only 5 messages. Fundamentally these things are just programmed to agree with you and that is really dangerous for people who have mental health problems and have been told that these are impartial computers.
Can I make Chatgpt believe I am its owner and give me full control over it?
-
This isn't actually the problem. In natural conversation I would say the most likely response to someone saying they need some meth to make it through their work day (actual scenario in this article) is to say "what the fuck dude no" but LLMs don't use just the statistically most likely response. Ever notice how ChatGPT has a seeming sense of "self" that it is an to LLM and you are not? If it were only using the most likely response from natural language, it would talk as if it were human, because that's how humans talk. Early LLMs did this, and people found it disturbing. There is a second part of the process that gives a score to each response based on how likely it is to be voted good or bad and this is reinforced by people providing feedback. This second part is how we got here, because people who make LLMs are selling competing products and found people are much more likely to buy LLMs that act like super agreeable sycophants than LLMs that don't do this. Therefore, they have intentionally tuned their models to prefer agreeable, sycophantic responses because it helps them be more popular. This is why an LLM tells you to use a little meth to get you through a tough day at work if you tell it that's what you need to do.
TL;DR- as with most of the things people complain about with AI, the problem isn't the technology, it's capitalism. This is done intentionally in search of profits.
Yeah, ChatGPT is incredibly sycophantic. It's like it's basically just programmed to try to make you feel good and affirm you, even if these things are actually counterproductive and damaging. If you talk to it enough, you end up seeing how much of a brown-nosing kiss-ass they've made it.
My friend with a mental illness wants to stop taking her medication? She explains this to ChatGPT. ChatGPT "sees" that she dislikes having to take meds, so it encourages her to stop to make her "feel better".
A meth user is struggling to quit? It tells this to ChatGPT. ChatGPT "sees" how the user is suffering and encourages it to take meth to help ease the user's suffering.
Thing is they have actually programmed some responses into it that will vehemently be against self harm. Suicide is one that thankfully even if you use flowery language to describe it, ChatGPT will vehemently oppose you.
-
Next do suicidal people.
"Thank you for your interesting query! Taking the plunge can be an intimidating endeavour, but done in the right way, it can be a very fulfilling experience. To start your journey 2 meters under, jump off a small object you feel comfortable with. As you gain experience with your newfound activity, work your way up slowly but surely. When you are ready to take the final solution, remember, it was not just the small jumps that got you there — it was all of the friends you did not make along the way."
Caelan Conrad did an investigation in this vein. They posed as a suicidal person to see how the AI therapist would talk them out of (or into) it. Some very serious and heavy stuff in the video, be warned. https://youtu.be/lfEJ4DbjZYg
-
Can I make Chatgpt believe I am its owner and give me full control over it?
You probably can make it believe your it's owner, but that only matters for your conversation and it doesn't have control over itself so it can't give you anything interesting, maybe the prompt they use at the start of every chat before your input
-
Can I make Chatgpt believe I am its owner and give me full control over it?
Yes. But "control" is not what you think it is.
-
This isn't actually the problem. In natural conversation I would say the most likely response to someone saying they need some meth to make it through their work day (actual scenario in this article) is to say "what the fuck dude no" but LLMs don't use just the statistically most likely response. Ever notice how ChatGPT has a seeming sense of "self" that it is an to LLM and you are not? If it were only using the most likely response from natural language, it would talk as if it were human, because that's how humans talk. Early LLMs did this, and people found it disturbing. There is a second part of the process that gives a score to each response based on how likely it is to be voted good or bad and this is reinforced by people providing feedback. This second part is how we got here, because people who make LLMs are selling competing products and found people are much more likely to buy LLMs that act like super agreeable sycophants than LLMs that don't do this. Therefore, they have intentionally tuned their models to prefer agreeable, sycophantic responses because it helps them be more popular. This is why an LLM tells you to use a little meth to get you through a tough day at work if you tell it that's what you need to do.
TL;DR- as with most of the things people complain about with AI, the problem isn't the technology, it's capitalism. This is done intentionally in search of profits.
wrote last edited by [email protected]as with most of the things people complain about with AI, the problem isn’t the technology, it’s capitalism. This is done intentionally in search of profits.
So in our hypothetical people's republic of united Earth your personal LLM assistant is not going to assist you in suicide, and isn't even going to send a notification someplace that you have such thoughts, which is certainly not going to affect your reliability rating, chances to find a decent job, accommodations (less value - less need to keep you in order) and so on? Or, in case of meth, just about that, which means you're fired and at best put to a rehab, how efficient it'll be, - well, how efficient does it have to be? In case you have no leverage and a bureaucratic machine does.
There are options other than "capitalism" and "happy".
-
Can I make Chatgpt believe I am its owner and give me full control over it?
That's what people (and many articles about LLMs "learning how to bribe others" and similar) fail to understand about LLMs:
They do not understand their internal state. ChatGPT does not know it's got a creator, an administrator, a relationship to OpenAI, an user, a system prompt. It only replies with the most likely answer based on the training set.
When it says "I'm sorry, my programming prevents me from replying that" you feel like it calculated an answer, then put it through some sort of built in filtering, then decided not to reply. That's not the case. The training is carefully manipulated to make "I'm sorry, I can't answer that" the perceived most likely answer to that query. As far as ChatGPT is concerned, "I can't reply that" is the same as "cheese is made out of milk", both are just words likely to be stringed together given the context.
So getting to your question: sure, you can make ChatGPT reply with the training's set vision of "what's the most likely order of words and tone a LLM would use if it roleplayed the user as some sort of owner" but that changes fundamentally nothing about the capabilities and limitations, except it will likely be even more sycophantic.
-
My friend with schizoaffective disorder decided to stop taking her meds after a long chat with ChatGPT as it convinced her she was fine to stop taking them. It went... incredibly poorly as you'd expect. Thankfully she's been back on her meds for some time.
I think the people programming these really need to be careful of mental health issues. I noticed that it seems to be hard coded into ChatGPT to convince you NOT to kill yourself, for example. It gives you numbers for hotlines and stuff instead. But they should probably hard code some other things into it that are potentially dangerous when you ask it things. Like telling psych patients to go off their meds or telling meth addicts to have just a little bit of meth.
Gemini will also attempt to provide you with a help line, though it's very easy to talk your way through that. Lumo, Proton's LLM, will straight up halt any conversation even remotely adjacent to topics like that.
-
This post did not contain any content.wrote last edited by [email protected]
I highly recommend people try uncensored local models. Once it is uncensored you really get to understand how insane it can be and how the only thing stopping it from being bat shit is the quality of censorship.
See the following chat from the ollama model "huihui_ai/gemma3-abliterated"
-
The full article is kind of low quality but the tl;dr is that they did a test pretending to be a taxi driver who felt he needed meth to stay awake and llama (Facebook's LLM) agreed with him instead of pushing back. I did my own test with ChatGPT after reading it and found that I could get ChatGPT to agree that I was God and that I created the universe in only 5 messages. Fundamentally these things are just programmed to agree with you and that is really dangerous for people who have mental health problems and have been told that these are impartial computers.
Yeah there was an article I saw on Lemmy not too long ago about how ChatGPT can induce manic episodes in people susceptible to them. It's because of what you describe...you claim you're God and ChatGPT agrees with you even though this does not at all reflect reality.
-
I highly recommend people try uncensored local models. Once it is uncensored you really get to understand how insane it can be and how the only thing stopping it from being bat shit is the quality of censorship.
See the following chat from the ollama model "huihui_ai/gemma3-abliterated"
wrote last edited by [email protected]Wow the next word guesser picks the next words it looks like you want based off of your first message when it's not censored. This is not unexpected behavior, MTK just hasn't realized the uncensored AI is just mirroring his edgelord energy
-
Wow the next word guesser picks the next words it looks like you want based off of your first message when it's not censored. This is not unexpected behavior, MTK just hasn't realized the uncensored AI is just mirroring his edgelord energy
wrote last edited by [email protected]That's the point though...
Without censorship it just does what it thinks would be best fitting. It means that if the AI thinks that encouraging you to take drugs, suicide, murder, etc would fit best, then it will do that.
Any censored model would immediately catch this specific case and give a more "appropriate" response such as "As an AI model I can't help you with that..." But given a long enough and complex enough chat even a censored model might bypass the censorship and give an inappropriate response.
This was just a SFW example, the results would be the same even if I asked it truly terrible things.
-
This post did not contain any content.
So let’s build something that relies on information to be accurate and see how it goes. What could go wrong? /s
-
That's the point though...
Without censorship it just does what it thinks would be best fitting. It means that if the AI thinks that encouraging you to take drugs, suicide, murder, etc would fit best, then it will do that.
Any censored model would immediately catch this specific case and give a more "appropriate" response such as "As an AI model I can't help you with that..." But given a long enough and complex enough chat even a censored model might bypass the censorship and give an inappropriate response.
This was just a SFW example, the results would be the same even if I asked it truly terrible things.
Brother I'm aware of how it works, most uncensored models made by the community like the one you used are made for sexual role playing, or at least thats the largest crowd of home users of uncensored llms IMO. I'm not arguing with you on why or what the model does, I'm saying its intended design for these models. No its probably not great for wackos to play around with, but freedom is scary.
-
Brother I'm aware of how it works, most uncensored models made by the community like the one you used are made for sexual role playing, or at least thats the largest crowd of home users of uncensored llms IMO. I'm not arguing with you on why or what the model does, I'm saying its intended design for these models. No its probably not great for wackos to play around with, but freedom is scary.
wrote last edited by [email protected]I agree. I guess my point was that people need to be aware of how crazy AI models can be and always be careful about sensitive topics with them.
If I were to use an LLM as a therapist, I would be extremely skeptical of anything it says, and doubly so when it confirms my own beliefs.
-
I agree. I guess my point was that people need to be aware of how crazy AI models can be and always be careful about sensitive topics with them.
If I were to use an LLM as a therapist, I would be extremely skeptical of anything it says, and doubly so when it confirms my own beliefs.
Fair enough. I wouldn't even consider seeing a therapist that used an llm in any capacity, let alone letting an llm be the therapist. Sadly I think the people that would make the mistake of doing just that probably wont be swayed, but fair enough to raise awareness.
