Can we trust LLM CALCULATIONS?.
-
The whole "two r's in strawberry" thing is enough of an argument for me. If things like that happen at such a low level, its completely impossible that it wont make mistakes with problems that are exponentially more complicated than that.
The problem with that is that it isn't actually counting the R's.
You'd probably have better luck asking it to write a script for you that returns the number of instances of a letter in a string of text, then getting it to explain to you how to get it running and how it works. You'd get the answer that way, and also then have a script that could count almost any character and text of almost any size.
That's much more complicated, impressive, and useful, imo.
-
LLMs don't and can't do math. They don't calculate anything, that's just not how they work. Instead, they do this:
2 + 2 =? What comes after that? Oh, I remember! It's '4'!It could be right, it could be wrong. If there's enough pattern in the training data, it could remember the correct answer. Otherwise it'll just place a plausible looking value there (behavior known as AI hallucination). So, you can not "trust" it.
Every LLM answer is a hallucination.
-
Ok, you have a moderately complex math problem you needed to solve. You gave the problem to 6 LLMS all paid versions. All 6 get the same numbers. Would you trust the answer?
-
Ok, you have a moderately complex math problem you needed to solve. You gave the problem to 6 LLMS all paid versions. All 6 get the same numbers. Would you trust the answer?
Maybe? I'd be looking all over for some convergent way to fuck it up, though.
If it's just one model or the answers are only close, lol no.
-
Every LLM answer is a hallucination.
Some are just realistic to the point of being correct. It frightens me how many users have no idea about any of that.
-
short answer: no.
Long Answer: They are still (mostly) statisics based and can't do real math. You can use the answers from LLMs as starting point, but you have to rigerously verify the answers they give.
A calculator as a tool to a llm though, that works, at least mostly, and could be better when kinks get worked out.
-
Finally an intelligent comment. So many comments in here that don't realize most LLM's are bundled with calculators that just do the math.
-
Ok, you have a moderately complex math problem you needed to solve. You gave the problem to 6 LLMS all paid versions. All 6 get the same numbers. Would you trust the answer?
Most LLM's now call functions in the background. Most calculations are just simple Python expressions.
-
Ok, you have a moderately complex math problem you needed to solve. You gave the problem to 6 LLMS all paid versions. All 6 get the same numbers. Would you trust the answer?
Yes, with absolute certainty.
For example: 2 + 2 = 5
It's absolutely correct and if you dispute it, big bro is gonna have to re-educated you on that.
-
Yes, with absolute certainty.
For example: 2 + 2 = 5
It's absolutely correct and if you dispute it, big bro is gonna have to re-educated you on that.
I NEED TO consult every LLM VIA TELEKINESIS QUANTUM ELECTRIC GRAVITY A AND B WAVE.
-
Most LLM's now call functions in the background. Most calculations are just simple Python expressions.
Yes. I was aware of that, but I was manipulated by an analog device
-
Ok, you have a moderately complex math problem you needed to solve. You gave the problem to 6 LLMS all paid versions. All 6 get the same numbers. Would you trust the answer?
I test my local LLM for the first few weeks. Every "answer" it gives me i still look up elsewhere to confirm. Then when i see it is accurate, I still will check every 10 or so questions just to make sure. Unfortunately, I feel like they are making search engines worst on purpose so that the A.i. or in this case server or local LLMs can replace them. This is the sweet spot. I wouldn't advise getting any newer LLM's that will come out in the next few months (next generation).
-
Ok, you have a moderately complex math problem you needed to solve. You gave the problem to 6 LLMS all paid versions. All 6 get the same numbers. Would you trust the answer?
No because there is randomness involved
-
Ok, you have a moderately complex math problem you needed to solve. You gave the problem to 6 LLMS all paid versions. All 6 get the same numbers. Would you trust the answer?
Here's an interesting post that gives a pretty good quick summary of when an LLM may be a good tool.
Here's one key:
Machine learning is amazing if:
- The problem is too hard to write a rule-based system for or the requirements change sufficiently quickly that it isn't worth writing such a thing and,
- The value of a correct answer is much higher than the cost of an incorrect answer.
The second of these is really important.
So if your math problem is unsolvable by conventional tools, or sufficiently complex that designing an expression is more effort than the answer is worth... AND ALSO it's more valuable to have an answer than it is to have a correct answer (there is no real cost for being wrong), THEN go ahead and trust it.
If it is important that the answer is correct, or if another tool can be used, then you're better off without the LLM.
The bottom line is that the LLM is not making a calculation. It could end up with the right answer. Different models could end up with the same answer. It's very unclear how much underlying technology is shared between models anyway.
For example, if the problem is something like, "here is all of our sales data and market indicators for the past 5 years. Project how much of each product we should stock in the next quarter. " Sure, an LLM may be appropriately close to a professional analysis.
If the problem is like "given these bridge schematics, what grade steel do we need in the central pylon?" Then, well, you are probably going to be testifying in front of congress one day.
-
No because there is randomness involved
That's why you ask 6 of them, and of they all come to the same conclusion then chances are it's either right, or a common pitfall.
-
Ok, you have a moderately complex math problem you needed to solve. You gave the problem to 6 LLMS all paid versions. All 6 get the same numbers. Would you trust the answer?
I mean, I don't know why you wouldn't just use something other than an LLM in that case
-
LLMs don't and can't do math. They don't calculate anything, that's just not how they work. Instead, they do this:
2 + 2 =? What comes after that? Oh, I remember! It's '4'!It could be right, it could be wrong. If there's enough pattern in the training data, it could remember the correct answer. Otherwise it'll just place a plausible looking value there (behavior known as AI hallucination). So, you can not "trust" it.
They don’t calculate anything
They calculate the statistical probability of the next token in an array of previous tokens
-
I did dozens of times. Same calculations.
That doesn't change the logic I gave
-
LLMs don't and can't do math. They don't calculate anything, that's just not how they work. Instead, they do this:
2 + 2 =? What comes after that? Oh, I remember! It's '4'!It could be right, it could be wrong. If there's enough pattern in the training data, it could remember the correct answer. Otherwise it'll just place a plausible looking value there (behavior known as AI hallucination). So, you can not "trust" it.
wrote last edited by [email protected]A good one will interpret what you are asking and then write code, often python I notice, and then let that do the math and return the answer. A math problem should use a math engine and that's how it gets around it.
But really why bother, go ask wolfram alpha or just write the math problem in code yourself.
-
Ok, you have a moderately complex math problem you needed to solve. You gave the problem to 6 LLMS all paid versions. All 6 get the same numbers. Would you trust the answer?
Well, I wanted to know the answer and formula for future value of a present amount. The AI answer that came up was clear, concise, and thorough. I was impressed and put the formula into my spreadsheet. My answer did not match the AI answer. So I kept looking for what I did wrong. Finally I just put the value into a regular online calculator and it matched the answer my spreadsheet was returning.
So AI gave me the right equation and the wrong answer. But it did it in a very impressive way. This is why I think it's important for AI to only be used as a tool and not a replacement for knowledge. You have to be able to understand how to check the results.
