Anthropic has developed an AI 'brain scanner' to understand how LLMs work and it turns out the reason why chatbots are terrible at simple math and hallucinate is weirder than you thought
-
This post did not contain any content.
"Ask Claude to add 36 and 59 and the model will go through a series of odd steps, including first adding a selection of approximate values (add 40ish and 60ish, add 57ish and 36ish). Towards the end of its process, it comes up with the value 92ish. Meanwhile, another sequence of steps focuses on the last digits, 6 and 9, and determines that the answer must end in a 5. Putting that together with 92ish gives the correct answer of 95," the MIT article explains."
That is precisrly how I do math. Feel a little targeted that they called this odd.
-
I think a lot of services are doing this behind the scenes already. Otherwise chatgpt would be getting basic arithmetic wrong a lot more considering the methods the article has shown it's using.
-
This is pretty normal, in my opinion. Every time people complain about common core arithmetic there are dozens of us who come out of the woodwork to argue that the concepts being taught are important for deeper understanding of math, beyond just rote memorization of pencil and paper algorithms.
Rote memorization should be minimized in school curriculum
-
This post did not contain any content.
Another very surprising outcome of the research is the discovery that these LLMs do not, as is widely assumed, operate by merely predicting the next word. By tracing how Claude generated rhyming couplets, Anthropic found that it chose the rhyming word at the end of verses first, then filled in the rest of the line.
If the llm already knows the full sentence it's going to output from the first word it "guesses" I wonder if you could short circuit it and say just give the full sentence instead of doing a cycle for each word of the sentence, could maybe cut down on llm energy costs.
-
72 * 10 + 70 * 3 + 2 * 3
That's what I do in my head if I need an exact result. If I'm approximateing I'll probably just do something like 70 * 15 which is much easier to compute (70 * 10 + 70 * 5 = 700 + 350 = 1050).
(72 * 10) + (2 * 3) = x
There, fixed, because otherwise order of operation gets fucky.
-
But here’s the really funky bit. If you ask Claude how it got the correct answer of 95, it will apparently tell you, “I added the ones (6+9=15), carried the 1, then added the 10s (3+5+1=9), resulting in 95.” But that actually only reflects common answers in its training data as to how the sum might be completed, as opposed to what it actually did.
This is not surprising. LLMs are not designed to have any introspection capabilities.
Introspection could probably be tacked onto existing architectures in a few different ways, but as far as I know nobody's done it yet. It will be interesting to see how that might change LLM behavior.
Then take that concept further, and let it keep introspecting and inspecting how it comes to the conclusions it does and eventually....
-
It already knows which words are, statistically, more commonly rhymed with each other. From the massive list of training poems. This is what the massive data sets are for. One of the interesting things is that it's not predicting backwards, exactly. It's actually mathematically converging on the response text to the prompt, all the words at the same time.
Which is exactly how we do it.
-
anything that claims it "thinks" in any way I immediately dismiss as an advertisement of some sort. these models are doing very interesting things, but it is in no way "thinking" as a sentient mind does.
You know they don't think - even though "It's a peculiar truth that we don't understand how large language models (LLMs) actually work."?
It's truly shocking to read this from a mess of connected neurons and synapses like yourself. You're simply doing fancy word prediction of the next word /s
-
It doesn't, who the hell cares if someone allowed it to break "predict whole text" into "predict part by part, and then "with rhyme, we start at the end". Sounds like a naive (not as in "simplistic", but as "most straightforward") way to code this, so given the task to write an automatic poetry producer, I would start with something similar. The whole thing still stands as fancy auto-complete
But how is this different from your average redditor?
-
"Ask Claude to add 36 and 59 and the model will go through a series of odd steps, including first adding a selection of approximate values (add 40ish and 60ish, add 57ish and 36ish). Towards the end of its process, it comes up with the value 92ish. Meanwhile, another sequence of steps focuses on the last digits, 6 and 9, and determines that the answer must end in a 5. Putting that together with 92ish gives the correct answer of 95," the MIT article explains."
That is precisrly how I do math. Feel a little targeted that they called this odd.
I use a calculator. Which an AI should also be and not need to do weird shit to do math.
-
That bit about how it turns out they aren't actually just predicting the next word is crazy and kinda blows the whole "It's just a fancy text auto-complete" argument out of the water IMO
I mean it implies that they CAN start with the conclusion or the "thought" and then generate the text to verbalize that.
It's shocking to what length humans will go to explain how their wetware neural network is fundamentally different and it's impossible for LLMs to think or reason in any way. Honestly LLMs teach us more about human intelligence (or the lack thereof) than machine intelligence. Like obi wan said, "The ability to speak does not make one intelligent" haha.
-
That has always been the case. Even basic programs need debugging sometimes, so we developed debuggers.
No it hasn't. When you program you break down the problem into many smaller sub programs and then codify them. There are errors that need debugging. But never "how does this part of the program I wrote work?".
There are some cases like detergents, apparently until recently we didn't know exactly how it works. But human engineered tools are not comparable to this.
-
The other day I asked an llm to create a partial number chart to help my son learn what numbers are next to each other. If I instructed it to do this using very detailed instructions it failed miserably every time. And sometimes when I even told it to correct specific things about its answer it still basically ignored me. The only way I could get it to do what I wanted consistently was to break the test down into small steps and tell it to show me its progress.
I'd be very interested to learn it's "thought process" in each of those scenarios.
It's like that "Joey Repeat After Me" meme from friends haha
-
This is great stuff. If we can properly understand these “flows” of intelligence, we might be able to write optimized shortcuts for them, vastly improving performance.
Better yet, teach AI to write code replacing specific optimized AI networks. Then automatically profile and optimize and unit test!
-
I use a calculator. Which an AI should also be and not need to do weird shit to do math.
Fascist. If someone does maths differently than your preference, it's not "weird shit". I'm facile with mental math despite what's perhaps a non-standard approach, and it's quite functional to be able to perform simple to moderate levels of mathematics mentally without relying on a calculator.
-
I use a calculator. Which an AI should also be and not need to do weird shit to do math.
Function calling is a thing chatbots can do now
-
Fascist. If someone does maths differently than your preference, it's not "weird shit". I'm facile with mental math despite what's perhaps a non-standard approach, and it's quite functional to be able to perform simple to moderate levels of mathematics mentally without relying on a calculator.
Wtf hahahahaha
-
when a calculator from the 80s can do the same thing.
1970's! The little blighters are even older than most people think.
Which is why I find it extra hilarious / extra infuriating that we've gone through all of these contortions and huge wastes of computing power and electricity to ultimately just make a computer worse at math.
Math is the one thing that computers are inherently good at. It's what they're for. Trying to use LLM's to perform it halfassedly is a completely braindead endeavor.
But who is going around asking these bots to specifically do math? Like in normal usage, Ive never once done that because I could just use a calculator or spreadsheet software if I need to get fancy lol
-
This post did not contain any content.
Someone put 69 to research and then to article. Nice trolling.
-
How I'd do it is basically
72 * (10+3)
(72 * 10) + (72 * 3)
(720) + (3*(70+2))
(720) + (210+6)
(720) + (216)
936
Basically I break the numbers apart into easier chunks and then add them together.
