AI chatbots unable to accurately summarise news, BBC finds
-
-
-
-
Funny, I find the BBC unable to accurately convey the news
-
Why, where they trained using MAIN STREAM NEWS? That could explain it.
-
Idk guys. I think the headline is misleading. I had an AI chatbot summarize the article and it says AI chatbots are really, really good at summarizing articles. In fact it pinky promised.
-
-
Temperature isn't even "creativity" per say, it's more a band-aid to patch looping and dryness in long responses.
-
Lower temperature is much better with modern sampling algorithms, E.G., MinP, DRY, maybe dynamic temperature like mirostat and such. Ideally, structure output, too. Unfortunately, corporate APIs usually don't offer this.
-
It can be mitigated with finetuning against looping/repetition/slop, but most models are the opposite, massively overtuned on their own output which "inbreeds" the model.
-
And yes, domain specific queries are best. Basically the user needs separate prompt boxes for coding, summaries, creative suggestions and such each with their own tuned settings (and ideally tuned models). You are right, this is a much better idea than offering a temperature knob to the user, but... most UIs don't even do this for some reason?
What I am getting at is this is not a problem companies seem interested in solving.
-
-
-
-
Lemmy is understandably sympathetic to self-hosted LLMs, but I get chewed out or even banned literally anywhere else.
In this fandom I'm in, there used to be enthusiasm for a "community enhancement" of a show since the official release looks terrible. Years later, I don't even mention the word "AI," just the idea of restoration (now that we have the tools to do it), and I get bombed and threadlocked.
-
-
The problem is that the "train of the thought" is also hallucinations. It might make the model better with more compute but it's diminishing rewards.
Rpg can use the llms because they're not critical. If the llm spews out nonsense you don't like, you just ask to redo, because it's all subjective.
-
For local LLMs, this is an issue because it breaks your prompt cache and slows things down, without a specific tiny model to "categorize" text... which no one has really worked on.
I don't think the corporate APIs or UIs even do this.
You are not wrong, but it's just not done for some reason.
-
Gemini Flash Thinking from earlier this year was very good for its speed/price, but it regressed a ton.
Gemini 1.5 is literally better than the new 2.0 in some of my tests, especially long-context ones.
-
-
-
Or at least as an assistant on a field your an expert in. Love using it for boilerplate at work (tech).
-
-
Bing/chatgpt is just as bad. It loves to tell you it's doing something and then just ignores you completely.
-
It is stated as 51% problematic, so maybe your coin flip was successful this time.
