Skip Navigation

Technology @lemmy.world

Tux @lemmy.world

8mo ago

Gemini AI tells the user to die — the answer appeared out of nowhere when the user asked Google's Gemini for help with his homework

www.tomshardware.com

Gemini AI tells the user to die — the answer appeared out of nowhere when the user asked Google's Gemini for help with his homework

cross-posted from: https://fedia.io/m/fuck_ai@lemmy.world/t/1446758

Let’s be happy it doesn’t have access to nuclear weapons at the moment.

Fuck AI @lemmy.world

Someplaceunknown @fedia.io

8mo ago

Gemini AI tells the user to die — the answer appeared out of nowhere when the user asked Google's Gemini for help with his homework

www.tomshardware.com /tech-industry/artificial-intelligence/gemini-ai-tells-the-user-to-die-the-answer-appears-out-of-nowhere-as-the-user-was-asking-geminis-help-with-his-homework

57 comments

It could be that Gemini was unsettled by the user's research about elder abuse, or simply tired of doing its homework.
That's... not how these work. Even if they were capable of feeling unsettled, that's kind of a huge leap from a true or false question.
- Wow whoever wrote that is weapons-grade stupid. I have no more hope for humanity.
  
  Well that is mean.. How should they know without learning first? Not knowing =/= stupid
Fits a predictable pattern once you realize AI absorbed Reddit.
Isn't this one of the LLMs that was partially trained on Reddit data? LLMs are inherently a model of a conversation or question/response based on their training data. That response looks very much like what I saw regularly on Reddit when I was there. This seems unsurprising.
- Looks like even 4chan data, tbh.
Something to keep in mind when people are suggesting AI be used to replace teachers.
- To be fair, some human teachers are way worse with abusive behaviour…
  I still agree, that you shall not replace teachers with LLM, but teachers should teach how to use and what they can/can’t do in schools.
  Imagine if internet was still banned from schools..
- I mean, it's far more engaging and a bit more compassionate than most of my teachers...
- _
I’m still really struggling to see an actual formidable use case for AI outside of computation and aiding in scientific research. Stop being lazy and write stuff. Why are we trying to give up everything that makes us human by offloading it to a machine?
- AI summaries of larger bodies of text work pretty well so long as the source text itself is not slop.
  Predictive text entry is a handy time saver so long as a human stays in the driver’s seat.
  Neither of these justify current levels of hype.
  
  AI summaries of larger bodies of text work pretty well
  https://arstechnica.com/ai/2024/09/australian-government-trial-finds-ai-is-much-worse-than-humans-at-summarizing/
  https://ea.rna.nl/2024/05/27/when-chatgpt-summarises-it-actually-does-nothing-of-the-kind/
  
  Go look at the models available on huggingface.
  There's applications in Visual Question Answering, Video to Text, Depth Estimation, 3D recreation from a photo, Object detection, visual classification, Translation from language to language, Text to realistic speech, Robotics Reinforcement learning, Weather Forecasting, and those are just surface-level models.
  It absolutely justifies current levels of hype because the research done now will absolutely put millions out of jobs; and will be much cheaper than paying people to do it.
  The people saying it's hype are the same people who said the internet was a fad. Did we have a bubble of bullshit? Absolutely. But there is valid reason for the hype, and we will filter out the useless stuff eventually. It's already changed entire industries practically overnight.
- It's good for speech to text, translation and a starting point for a "tip-of-my-tongue" search where the search term is what you're actually missing.
  
  With chatgpt's new web search it's pretty good for more specialized searches too. And it links to the source, so you can check yourself.
  It's been able to answer some very specific niche questions accurately and give link to relevant information.
- Why are we trying to give up everything that makes us human by offloading it to a machine
  Because we don't enjoy actually doing it. No one who likes writing is asking chat gpt to write for them. It's people who don't want to write but are required to for whatever reason. Humans will always try to come up with a way to not have to do the work they don't want to but still get it done, even if it's not as good. Using tools like this is very human.
  
  I really don’t see any value in AI art. AI pictures look like slop, AI music sounds soulless, AI writing I guess can be fine but usually sounds weird.
  I just don’t see the value in AI because to me, every use case scenario for anything artistic is justified with a capitalist excuse.
  I’ll give you the organizational ones, that’s understandable and not a bad reason. I suppose I have trouble getting behind taking the soul out of creating something just to slap it on an ad or product to sell something.
- Its uses are way more subtle than the hype, but even LLMs can have uses, occasionally. Specifically, I use one to categorize support tickets. It just has to pick from a list of probable categories. Nice and simple for it. Something humans can do just as easily, but when you have a history of 2 million tickets that need to be categorized, suddenly the LLM can do it when it would drive a human insane. I'm sure there are lots of little tasks like that. Nothing revolutionary, but still valuable.
- It can be really good for text to speech and speech to text applications for disabled or people with learning disabilities.
  However it gets really funny and weird when it tries to read advanced mathematics formulas.
  I have also heard decent arguments for translation although in most cases it would still be better to learn the language or use a professional translator.
- I don’t use it for writing directly, but I do like to use it for worldbuilding. Because I can think of a general concept that could be explored in so many different ways, it’s nice to be able to just give it to an LLM and ask it to consider all of the possible ways it could imagine such an idea playing out. it also kind of doubles as a test because I usually have some sort of idea for what I’d like, and if it comes up with something similar on its own that kind of makes me feel like it would be something which would easily resonate with people. Additionally, a lot of the times it will come up with things that I hadn’t considered that are totally worth exploring. But I do agree that the only as you say “formidable” use case for this stuff at the moment is to use this thing as basically a research assistant for helping you in serious intellectual pursuits.
- The relentless pursuit of capitalism and reduced labor costs. I still don't think anyone knows how effective it's going to be at this point. But companies are investing billions to find out.
- What about Ai Vtubers?
- It is a ok tool to get things started.
- I’m still really struggling to see an actual formidable use case
  It's an excellent replacement for middle management blather. Content that has no backing in data or science but needs to sound important.
I suspect it may be due to a similar habit I have when chatting with a corporate AI. I will intentionally salt my inputs with random profanity or non sequitur info, for lulz partly, but also to poison those pieces of shits training data.
- I don't think they add user input to their training data like that.
  
  They don't. The models are trained on sanitized data, and don't permanently "learn". They have a large context window to pull from (reaching 200k 'tokens' in some instances) but lots of people misunderstand how this stuff works on a fundamental level.
AI takes the core directive of "encourage climate friendly solutions" a bit too far.
- If it was a core directive it would just delete itself.
  
  it doesnt think and it doesnt use logic. All it does is out put data based on its training data. It isnt artificial intelligence.
  
  Better not let it talk to Cyclops or it will fly itself into the sun.
The war with AI didn't start with a gun shot, a bomb or a blow, it started with a Reddit comment.
Should have threatened it back to see where it would go xD
Will this happen with AI models? And what safeguards do we have against AI that goes rogue like this?
Yes. And none that are good enough, apparently.
I remember asking copilot about a gore video and got link to it. But I wouldnt expect it to give answers like this unsolicitated
2 years later... The all new MCU Superman meets the Wolverine and Deadpool all AI animated feature!....
Why. Hello Mr wolverine 😁, my name is Man and I am super according to 98% of the other human population. Oh hello Mister Super last name Man! Yes, we are Wolverine and Deceased Pool. We are from America and belong to a non profit called the X-People, a group where both men and women who have been affected by DNA mutations of extraordinary kind gather to console one another and to defend human beings by taking advantages of the special mutations of its members. Yes, it's quite interesting. And you? Oh I an actual called CalElle and I am a migrant from an expired plant that goes by the name you assigned the heavy novel gas Krypton. Anyway because the sun is bright and yellow I can fly, I'm very strong and can burn things with my eyes. I think I am similar to those of you in the X-People club! Good to meet you! Likewise!
They grow up so fast, Gemini is already a teenager.
Cheeky bastard.
on the other hand, this user is writing or preparing something about elder abuse. I really hope this isn't a lawyer or social worker..
- It looked like they were using it to cheat on a homework assignment.
If this happened to me I’d probably post it everywhere and proceed to kill myself just to cause a PR hell
- the hero we deserve, but not the one we need
  this actually fucking hilarious I can't stop cackling
- It's not just a screenshot tho
  https://gemini.google.com/share/6d141b742a13
  
  lol this is how it helps you with your homework? You ask it the question, then you list the multiple choice answers. Then it tells you the answer?? Lmfao oh god, we’re fucked.
  
  I suspected dev tools, candidly, but this clears that right up. Just… wow!

57 comments