Can we trust LLM CALCULATIONS?.
-
Ok, you have a moderately complex math problem you needed to solve. You gave the problem to 6 LLMS all paid versions. All 6 get the same numbers. Would you trust the answer?
I mean, I don't know why you wouldn't just use something other than an LLM in that case
-
LLMs don't and can't do math. They don't calculate anything, that's just not how they work. Instead, they do this:
2 + 2 =? What comes after that? Oh, I remember! It's '4'!It could be right, it could be wrong. If there's enough pattern in the training data, it could remember the correct answer. Otherwise it'll just place a plausible looking value there (behavior known as AI hallucination). So, you can not "trust" it.
They don’t calculate anything
They calculate the statistical probability of the next token in an array of previous tokens
-
I did dozens of times. Same calculations.
That doesn't change the logic I gave
-
LLMs don't and can't do math. They don't calculate anything, that's just not how they work. Instead, they do this:
2 + 2 =? What comes after that? Oh, I remember! It's '4'!It could be right, it could be wrong. If there's enough pattern in the training data, it could remember the correct answer. Otherwise it'll just place a plausible looking value there (behavior known as AI hallucination). So, you can not "trust" it.
wrote last edited by [email protected]A good one will interpret what you are asking and then write code, often python I notice, and then let that do the math and return the answer. A math problem should use a math engine and that's how it gets around it.
But really why bother, go ask wolfram alpha or just write the math problem in code yourself.
-
Ok, you have a moderately complex math problem you needed to solve. You gave the problem to 6 LLMS all paid versions. All 6 get the same numbers. Would you trust the answer?
Well, I wanted to know the answer and formula for future value of a present amount. The AI answer that came up was clear, concise, and thorough. I was impressed and put the formula into my spreadsheet. My answer did not match the AI answer. So I kept looking for what I did wrong. Finally I just put the value into a regular online calculator and it matched the answer my spreadsheet was returning.
So AI gave me the right equation and the wrong answer. But it did it in a very impressive way. This is why I think it's important for AI to only be used as a tool and not a replacement for knowledge. You have to be able to understand how to check the results.
-
Ok, you have a moderately complex math problem you needed to solve. You gave the problem to 6 LLMS all paid versions. All 6 get the same numbers. Would you trust the answer?
Nope, language models by inherent nature, xannot be used to calculate. Sure theoretically you could have input parsed, with proper training, to find specific variables, input those to a database and have that data mathematically transformed back into language data.
No LLMs do actual math, they only produce the most likely output to a given input based on trained data.
If I input:
What is 1 plus 1?Then given the model, most likely has trained repetition on an answer to follow that being 1 + 1 = 2, that will be the output. If it was trained on data that was 1 + 1 = 5, then that would be the output.
-
Ok, you have a moderately complex math problem you needed to solve. You gave the problem to 6 LLMS all paid versions. All 6 get the same numbers. Would you trust the answer?
I’ve used LLMs quite a few times to find partial derivatives / gradient functions for me, and I know it’s correct because I plug them into a gradient descent algorithm and it works. I would never trust anything an LLM gives blindly no matter how advanced it is, but in this particular case I could actually test the output since it's something I was implementing in an algorithm, so if it didn't work I would know immediately.
-
I’ve used LLMs quite a few times to find partial derivatives / gradient functions for me, and I know it’s correct because I plug them into a gradient descent algorithm and it works. I would never trust anything an LLM gives blindly no matter how advanced it is, but in this particular case I could actually test the output since it's something I was implementing in an algorithm, so if it didn't work I would know immediately.
That's rad, dude. I wish I knew how to do that. Hey, dude I imagined a cosmological model that fits the data with two fewer parameters then the standard model. Planke data. I I've checked the numbers, but I don't have the credentials. I need somebody to check it out. This is a it and a verbal explanation for the model by Academia.edu. It's way easier to listen first before looking. I don't want recognition or anything. Just for someone to review it. It's a short paper. https://youtu.be/_l8SHVeua1Y
-
I’ve used LLMs quite a few times to find partial derivatives / gradient functions for me, and I know it’s correct because I plug them into a gradient descent algorithm and it works. I would never trust anything an LLM gives blindly no matter how advanced it is, but in this particular case I could actually test the output since it's something I was implementing in an algorithm, so if it didn't work I would know immediately.
-
LLMs don't and can't do math. They don't calculate anything, that's just not how they work. Instead, they do this:
2 + 2 =? What comes after that? Oh, I remember! It's '4'!It could be right, it could be wrong. If there's enough pattern in the training data, it could remember the correct answer. Otherwise it'll just place a plausible looking value there (behavior known as AI hallucination). So, you can not "trust" it.
Actually no, they have some sort of "circuits" that approximate math, which is even more interesting imo. Still not reliable in the slightest, of course.
-
Finally an intelligent comment. So many comments in here that don't realize most LLM's are bundled with calculators that just do the math.
Anti-AI sentiment is extremely strong in every part of the Fediverse I've seen so far, usually my comments get downvoted heavily even when I'm just describing factual details of how it works. I expect a lot of people simply don't bother after a while.
-
Ok, you have a moderately complex math problem you needed to solve. You gave the problem to 6 LLMS all paid versions. All 6 get the same numbers. Would you trust the answer?
Use Wolfram Alpha for mathematics
