Save The Planet
-
I mean I literally run a local LLM, while the model sits in memory it's really not using up a crazy amount of resources, I should hook up something to actually measure exactly how much it's pulling vs just looking at htop/atop and guesstimating based on load TBF.
Vs when I play a game and the fans start blaring and it heats up and you can clearly see the usage increasing across various metrics
He isn't talking about locally, he is talking about what it takes for the AI providers to provide the AI.
To say "it takes more energy during training" entirely depends on the load put on the inference servers, and the size of the inference server farm.
-
I mean, continued use of AI encourages the training of new models. If nobody used the image generators, they wouldn't keep trying to make better ones.
wrote on last edited by [email protected]TBH most people still use old SDXL finetunes for porn, even with the availability of newer ones.
-
Worse is Google that insists on shoving a terrible AI-based result in your face every time you do a search, with no way to turn it off.
I'm not telling these systems to generate images of cow-like girls, but I'm getting AI shoved in my face all the time whether I want it or not. (I don't).
And including the word "fuck" in your query no longer stops it.
-
It's closer to running 8 high-end video games at once. Sure, from a scale perspective it's further removed from training, but it's still fairly expensive.
wrote on last edited by [email protected]Not at all. Not even close.
Image generation is usually batched and takes seconds, so 700W (a single H100 SXM) for a few seconds for a batch of a few images to multiple users. Maybe more for the absolute biggest (but SFW, no porn) models.
LLM generation takes more VRAM, but is MUCH more compute-light. Typically one has banks of 8 GPUs in multiple servers serving many, many users at once. Even my lowly RTX 3090 can serve 8+ users in parallel with TabbyAPI (and modestly sized model) before becoming more compute bound.
So in a nutshell, imagegen (on an 80GB H100) is probably more like 1/4-1/8 of a video game at once (not 8 at once), and only for a few seconds.
Text generation is similarly efficient, if not more. Responses take longer (many seconds, except on special hardware like Cerebras CS-2s), but it parallelized over dozens of users per GPU.
This is excluding more specialized hardware like Google's TPUs, Huawei NPUs, Cerebras CS-2s and so on. These are clocked far more efficiently than Nvidia/AMD GPUs.
...The worst are probably video generation models. These are extremely compute intense and take a long time (at the moment), so you are burning like a few minutes of gaming time per output.
ollama/sd-web-ui are terrible analogs for all this because they are single user, and relatively unoptimized.
-
Right, but that's kind of like saying "I don't kill babies" while you use a product made from murdered baby souls. Yes you weren't the one who did it, but your continued use of it caused the babies too be killed.
There is no ethical consumption under capitalism and all that, but I feel like here is a line were crossing. This fruit is hanging so low it's brushing the grass.
"The plane is flying, anyway."
-
This post did not contain any content.
When I’m told there’s power issues and to conserve power I drop my AC to 60 and leave all my lights on. Only way for them to fix the grid is to break it.
-
This post did not contain any content.
I like tits.
-
I have llama 3.2 on my phone and it's really funny because it's so low powered and dumb but so sweet.
it's like a little friend to talk to when I don't have Internet. he's a lil stupid but he got the spirit
Use Qwen 2.5, that's my recommendation. You can also set "pals". And the best part, is I have a portable battery and solar charger, so I could theoretically (and have in the past) run it from solar alone.
-
It's both. Also don't let the house, supreme court, or the orange buffoon and his cabinet get out of culpability. Checks and balances can work ... when they all aren't bought and paid for by rich fucks.
I meant to mention the other ones at fault, but I edited what I was typing and backspaced that part.
Thanks
-
Right, but that's kind of like saying "I don't kill babies" while you use a product made from murdered baby souls. Yes you weren't the one who did it, but your continued use of it caused the babies too be killed.
There is no ethical consumption under capitalism and all that, but I feel like here is a line were crossing. This fruit is hanging so low it's brushing the grass.
Are you interpreting my statement as being in favour of training AIs?
-
My guy, we're not talking about just leaving a model loaded, we're talking about actual usage in a cloud setting with far more GPUs and users involved.
So you think they're all at full load at all times? Does that seem reasonable to you?
-
One user vs a public service is apples to oranges and it's actually hilarious you're so willing to compare them.
It's literally the same thing, the obvious difference is how much usage it's getting at a time per gpu, but everyone seems to assume all these data centers are running at full load at all times for some reason?
-
He isn't talking about locally, he is talking about what it takes for the AI providers to provide the AI.
To say "it takes more energy during training" entirely depends on the load put on the inference servers, and the size of the inference server farm.
There's no functional difference aside from usage and scale, which is my point.
I find it interesting that the only actual energy calculations I see from researchers is the training and the things going along with the training, rather then the usage per actual request after training.
People then conflate training energy costs to normal usage cost without data to back it up. I don't have the data either but I do have what I can do/see on my side.
-
Worse is Google that insists on shoving a terrible AI-based result in your face every time you do a search, with no way to turn it off.
I'm not telling these systems to generate images of cow-like girls, but I'm getting AI shoved in my face all the time whether I want it or not. (I don't).
Then I guess it's time to stop using Google!
-
I can't believe they're not even gonna show us the 5 titty girl.
I'm here waiting for it
-
This post did not contain any content.
Doesn't seem to be a waste of power to me.
-
Ok now go just one step further and ask yourself what variables factor into this.
There's a reason that pattern exists, and it isn't because solar and cooling hours don't align.
the difference between demand and net demand in that graph is purely solar/wind generation, isn't it?
-
And including the word "fuck" in your query no longer stops it.
And when it did it also altered the results, making them worse, because it was trying to satisfy "fuck" as part of your search.
-
Oh, and you don't want it and want the stupid model? You can still buy it for 3x the price.
Do the new models even have non-"smart" fittings? I thought all the electronic chip plants closed during covid.
-
So you think they're all at full load at all times? Does that seem reasonable to you?
Given that cloud providers are desperately trying to get more compute resources, but are limited by chip production - yes, of course? Why do you think they're trying to expand their resources while their existing resources aren't already limited?
