Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • World
  • Users
  • Groups
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
Brand Logo

agnos.is Forums

  1. Home
  2. Technology
  3. AI chatbots unable to accurately summarise news, BBC finds

AI chatbots unable to accurately summarise news, BBC finds

Scheduled Pinned Locked Moved Technology
technology
133 Posts 74 Posters 1.2k Views
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • P [email protected]

    I learned that AI chat bots aren't necessarily trustworthy in everything. In fact, if you aren't taking their shit with a grain of salt, you're doing something very wrong.

    R This user is from outside of this forum
    R This user is from outside of this forum
    [email protected]
    wrote on last edited by
    #69

    This is my personal take. As long as you're careful and thoughtful whenever using them, they can be extremely useful.

    L E 2 Replies Last reply
    0
    • misk@sopuli.xyzM [email protected]
      This post did not contain any content.
      teknikal@eviltoast.orgT This user is from outside of this forum
      teknikal@eviltoast.orgT This user is from outside of this forum
      [email protected]
      wrote on last edited by
      #70

      I just tried it on deepseek it did it fine and gave the source for everything it mentioned as well.

      F D 2 Replies Last reply
      0
      • misk@sopuli.xyzM [email protected]
        This post did not contain any content.
        P This user is from outside of this forum
        P This user is from outside of this forum
        [email protected]
        wrote on last edited by
        #71

        ShockedPikachu.svg

        1 Reply Last reply
        0
        • S [email protected]

          Do you dislike ai?

          W This user is from outside of this forum
          W This user is from outside of this forum
          [email protected]
          wrote on last edited by
          #72

          I work in tech and can confirm the the vast majority of engineers "dislike ai" and are disillusioned with AI tools. Even ones that work on AI/ML tools. It's fewer and fewer people the higher up the pay scale you go.

          There isn't a single complex coding problem an AI can solve. If you don't understand something and it helps you write it I'll close the MR and delete your code since it's worthless. You have to understand what you write. I do not care if it works. You have to understand every line.

          "But I use it just fine and I'm an..."

          Then you're not an engineer and you shouldn't have a job. You lack the intelligence, dedication and knowledge needed to be one. You are detriment to your team and company.

          E 5 2 Replies Last reply
          0
          • W [email protected]

            Its nice that you inform people that they cant tell if something is saving them time or not without knowing what their job is or how they are using a tool.

            W This user is from outside of this forum
            W This user is from outside of this forum
            [email protected]
            wrote on last edited by
            #73

            If they think AI is working for them then he can. If you think AI is an effective tool for any profession you are a clown. If my son's preschool teacher used it to make a lesson plan she would be incompetent. If a plumber asked what kind of wrench he needed he would be kicked out of my house. If an engineer of one of my teams uses it to write code he gets fired.

            AI "works" because you're asking questions you don't know and it's just putting words together so they make sense without regard to accuracy. It's a hard limit of "AI" that we've hit. It won't get better in our lifetimes.

            S 1 Reply Last reply
            0
            • rivalarrival@lemmy.todayR [email protected]

              It found 51% of all AI answers to questions about the news were judged to have significant issues of some form.

              How good are the human answers? I mean, I expect that an AI's error rate is currently higher than an "expert" in their field.

              But I'd guess the AI is quite a bit better than, say, the average Republican.

              B This user is from outside of this forum
              B This user is from outside of this forum
              [email protected]
              wrote on last edited by
              #74

              I guess you don’t get the issue. You give the AI some text to summarize the key points. The AI gives you wrong info in a percentage of those summaries.

              There’s no point in comparing this to a human, since this is usually something done for automation, that is, to work for a lot of people or a large quantity of articles.

              rivalarrival@lemmy.todayR 1 Reply Last reply
              0
              • M [email protected]

                Is it worse than the current system of editors making shitty click bait titles?

                H This user is from outside of this forum
                H This user is from outside of this forum
                [email protected]
                wrote on last edited by
                #75

                Surprisingly, yes

                1 Reply Last reply
                0
                • O [email protected]

                  Y'know, a lot of the hate against AI seems to mirror the hate against Wikipedia, search engines, the internet, and even computers in the past.

                  Do you just blindly believe whatever it tells you?

                  It's not absolutely perfect, so it's useless.

                  It's all just garbage information!

                  This is terrible for jobs, society, and the environment!

                  E This user is from outside of this forum
                  E This user is from outside of this forum
                  [email protected]
                  wrote on last edited by
                  #76

                  You know what... now that you say it, it really is just like the anti-Wikipedia stuff.

                  1 Reply Last reply
                  0
                  • B [email protected]

                    I guess you don’t get the issue. You give the AI some text to summarize the key points. The AI gives you wrong info in a percentage of those summaries.

                    There’s no point in comparing this to a human, since this is usually something done for automation, that is, to work for a lot of people or a large quantity of articles.

                    rivalarrival@lemmy.todayR This user is from outside of this forum
                    rivalarrival@lemmy.todayR This user is from outside of this forum
                    [email protected]
                    wrote on last edited by
                    #77

                    I'm more interested in the technology itself, rather than its current application.

                    I feel like I am watching a toddler taking her first steps; wondering what she will eventually accomplish in her lifetime. But the loudest voices aren't cheering her on: they're sitting in their recliners, smugly claiming she's useless. She can't even participate in a marathon, let alone compete with actual athletes!

                    Basically, the best AIs currently have college-level mastery of language, and the reasoning skills of children. They are already far more capable and productive than anti-vaxxers, or our current president.

                    B 1 Reply Last reply
                    0
                    • M [email protected]

                      I’m not against attempts at global artificial intelligence, just against one approach to it. Also no matter how we want to pretend it’s something general, we in fact want something thinking like a human.

                      Agreed. The techbros pretending that the stochastic parrots they've created are general AI annoys me to no end.

                      While not as academically cogent as your response (totally not feeling inferior at the moment), it has struck me that LLMs would make a fantastic input/output to a greater system analogous to the Wernicke/Broca areas of the brain. It seems like they're trying to get a parrot to swim by having it do literally everything. I suppose the thing that sticks in my craw is the giveaway that they've promised that this one technique (more or less, I know it's more complicated than that) can do literally everything a human can, which should be an entire parade of red flags to anyone with a drop of knowledge of data science or fraud. I know that it's supposed to be a universal function appropriator hypothetically, but I think the gap between hypothesis and practice is very large and we're dumping a lot of resources into filling in the canyon (chucking more data at the problem) when we could be building a bridge (creating specialized models that work together).

                      Now that I've used a whole lot of cheap metaphor on someone who causally dropped 'syllogism' into a conversation, I'm feeling like a freshmen in a grad level class. I'll admit I'm nowhere near up to date on specific models and bleeding edge techniques.

                      R This user is from outside of this forum
                      R This user is from outside of this forum
                      [email protected]
                      wrote on last edited by
                      #78

                      While not as academically cogent as your response

                      An elegant way to make someone feel ashamed for using many smart words, ha-ha.

                      I know that it’s supposed to be a universal function appropriator hypothetically, but I think the gap between hypothesis and practice is very large and we’re dumping a lot of resources into filling in the canyon (chucking more data at the problem) when we could be building a bridge (creating specialized models that work together).

                      The metaphor is correct, I think it's some social mechanism making them choose a brute force solution first. Say, spending more resources to achieve the same might be a downside usually, but if it's a resource otherwise not in demand, that only the stronger parties possess in sufficient amounts, like corporations and governments, then that may be an upside for someone by changing the balance.

                      And LLMs appear good enough to make captcha-solving machines, proof image or video faking machines, fraudulent chatbot machines, or machines predicting someone's (or some crowd's) responses well enough to play them. So I'd say commercially they already are successful.

                      Now that I’ve used a whole lot of cheap metaphor on someone who causally dropped ‘syllogism’ into a conversation, I’m feeling like a freshmen in a grad level class. I’ll admit I’m nowhere near up to date on specific models and bleeding edge techniques.

                      We-ell, it's just hard to describe the idea without using that word, but I haven't even finished my BS yet (lots of procrastinating, running away and long interruptions), and also the only bit of up to date knowledge I had was what DeepSeek prints when answering, so.

                      M 1 Reply Last reply
                      0
                      • W [email protected]

                        I work in tech and can confirm the the vast majority of engineers "dislike ai" and are disillusioned with AI tools. Even ones that work on AI/ML tools. It's fewer and fewer people the higher up the pay scale you go.

                        There isn't a single complex coding problem an AI can solve. If you don't understand something and it helps you write it I'll close the MR and delete your code since it's worthless. You have to understand what you write. I do not care if it works. You have to understand every line.

                        "But I use it just fine and I'm an..."

                        Then you're not an engineer and you shouldn't have a job. You lack the intelligence, dedication and knowledge needed to be one. You are detriment to your team and company.

                        E This user is from outside of this forum
                        E This user is from outside of this forum
                        [email protected]
                        wrote on last edited by
                        #79

                        "I can calculate powers with decimal values in the exponent and if you can not do that on paper but instead use these machines, your calculations are worthless and you are not an engineer"

                        You seem to fail to see that this new tool has unique strengths. As the other guy said, it is just like people ranting about Wikipedia. Absurd.

                        W 1 Reply Last reply
                        0
                        • rivalarrival@lemmy.todayR [email protected]

                          I'm more interested in the technology itself, rather than its current application.

                          I feel like I am watching a toddler taking her first steps; wondering what she will eventually accomplish in her lifetime. But the loudest voices aren't cheering her on: they're sitting in their recliners, smugly claiming she's useless. She can't even participate in a marathon, let alone compete with actual athletes!

                          Basically, the best AIs currently have college-level mastery of language, and the reasoning skills of children. They are already far more capable and productive than anti-vaxxers, or our current president.

                          B This user is from outside of this forum
                          B This user is from outside of this forum
                          [email protected]
                          wrote on last edited by
                          #80

                          It’s not the people that simply decided to hate on AI, it was the sensationalist media hyping it up so much to the point of scaring people: “it’ll take all your jobs”, or companies shoving it down our throats by putting it in every product even when it gets in the way of the actual functionality people want to use. Even my company “forces” us all to use X prompts every week as a sign of being “productive”. The result couldn’t be different.

                          1 Reply Last reply
                          0
                          • P [email protected]

                            But every techbro on the planet told me it's exactly what LLMs are good at. What the hell!? /s

                            https://lemm.ee/comment/18029491

                            H This user is from outside of this forum
                            H This user is from outside of this forum
                            [email protected]
                            wrote on last edited by
                            #81

                            Not only techbros though. Most of my friends are not into computers but they all think AI is magical and will change the whole world for the better. I always ask "how can a blackbox that throws up random crap and runs on the computers of big companies out of the country would change anything?" They don't know what to say but they still believe something will happen and a program can magically become sentient. Sometimes they can be fucking dumb but I still love them.

                            S 1 Reply Last reply
                            0
                            • E [email protected]

                              Rare that people here argument for LLMs like that here, usually it is the same kind of "uga suga, AI bad, did not already solve world hunger".

                              H This user is from outside of this forum
                              H This user is from outside of this forum
                              [email protected]
                              wrote on last edited by
                              #82

                              Your comment would be acceptable if AI was not advertised as solving all our problems, like world hunger.

                              E 1 Reply Last reply
                              0
                              • E [email protected]

                                "I can calculate powers with decimal values in the exponent and if you can not do that on paper but instead use these machines, your calculations are worthless and you are not an engineer"

                                You seem to fail to see that this new tool has unique strengths. As the other guy said, it is just like people ranting about Wikipedia. Absurd.

                                W This user is from outside of this forum
                                W This user is from outside of this forum
                                [email protected]
                                wrote on last edited by
                                #83

                                You can also just have an application designed to do that do it more accurately.

                                If you can't do that you're not an engineer. If you don't recommend that you're not an engineer.

                                1 Reply Last reply
                                0
                                • teknikal@eviltoast.orgT [email protected]

                                  I just tried it on deepseek it did it fine and gave the source for everything it mentioned as well.

                                  F This user is from outside of this forum
                                  F This user is from outside of this forum
                                  [email protected]
                                  wrote on last edited by
                                  #84

                                  Now ask it whether Taiwan is a country.

                                  Q 1 Reply Last reply
                                  0
                                  • R [email protected]

                                    While not as academically cogent as your response

                                    An elegant way to make someone feel ashamed for using many smart words, ha-ha.

                                    I know that it’s supposed to be a universal function appropriator hypothetically, but I think the gap between hypothesis and practice is very large and we’re dumping a lot of resources into filling in the canyon (chucking more data at the problem) when we could be building a bridge (creating specialized models that work together).

                                    The metaphor is correct, I think it's some social mechanism making them choose a brute force solution first. Say, spending more resources to achieve the same might be a downside usually, but if it's a resource otherwise not in demand, that only the stronger parties possess in sufficient amounts, like corporations and governments, then that may be an upside for someone by changing the balance.

                                    And LLMs appear good enough to make captcha-solving machines, proof image or video faking machines, fraudulent chatbot machines, or machines predicting someone's (or some crowd's) responses well enough to play them. So I'd say commercially they already are successful.

                                    Now that I’ve used a whole lot of cheap metaphor on someone who causally dropped ‘syllogism’ into a conversation, I’m feeling like a freshmen in a grad level class. I’ll admit I’m nowhere near up to date on specific models and bleeding edge techniques.

                                    We-ell, it's just hard to describe the idea without using that word, but I haven't even finished my BS yet (lots of procrastinating, running away and long interruptions), and also the only bit of up to date knowledge I had was what DeepSeek prints when answering, so.

                                    M This user is from outside of this forum
                                    M This user is from outside of this forum
                                    [email protected]
                                    wrote on last edited by
                                    #85

                                    An elegant way to make someone feel ashamed for using many smart words, ha-ha.

                                    Unintentional I assure you.

                                    I think it’s some social mechanism making them choose a brute force solution first.

                                    I feel like it's simpler than that. Ye olde "when all you have is a hammer, everything's a nail". Or in this case, when you've built the most complex hammer in history, you want everything to be a nail.

                                    So I’d say commercially they already are successful.

                                    Definitely. I'll never write another cover letter. In their use-case, they're solid.

                                    but I haven’t even finished my BS yet

                                    Currently working on my masters after being in industry for a decade. The paper is nice, but actually applying the knowledge is poorly taught (IMHO, YMMV) and being willing to learn independently has served me better than by BS in EE.

                                    1 Reply Last reply
                                    0
                                    • N [email protected]

                                      What a nuanced representation of the position, I just feel trustworthiness oozes out of the screen.
                                      In case you're using random words generation machine to summarise this comment for you, it was a sarcasm, and I meant the opposite.

                                      E This user is from outside of this forum
                                      E This user is from outside of this forum
                                      [email protected]
                                      wrote on last edited by
                                      #86

                                      So many arguments... Wow!

                                      N 1 Reply Last reply
                                      0
                                      • H [email protected]

                                        Your comment would be acceptable if AI was not advertised as solving all our problems, like world hunger.

                                        E This user is from outside of this forum
                                        E This user is from outside of this forum
                                        [email protected]
                                        wrote on last edited by
                                        #87

                                        So the ads are the problem? Do you have a link to such an ad?

                                        H 1 Reply Last reply
                                        0
                                        • E [email protected]

                                          So the ads are the problem? Do you have a link to such an ad?

                                          H This user is from outside of this forum
                                          H This user is from outside of this forum
                                          [email protected]
                                          wrote on last edited by
                                          #88

                                          Not ads, whole governments talking about it and funding that crap like Altman/Musk in the USA or Macron in Europe.

                                          1 Reply Last reply
                                          0
                                          Reply
                                          • Reply as topic
                                          Log in to reply
                                          • Oldest to Newest
                                          • Newest to Oldest
                                          • Most Votes


                                          • Login

                                          • Login or register to search.
                                          • First post
                                            Last post
                                          0
                                          • Categories
                                          • Recent
                                          • Tags
                                          • Popular
                                          • World
                                          • Users
                                          • Groups