Do any of you have a buttload of RAM sitting around?
-
True, but there are also some legitimate applications for 100s of gigabytes of RAM. I've been working on a thing for processing historical OpenStreetMap data and it is quite a few orders of magnitude faster to fill the database by loading the 300GiB or so of point data into memory, sorting it in memory, and then partitioning and compressing it into pre-sorted table files which RocksDB can ingest directly without additional processing. I had to get 24x16GiB of RAM in order to do that, though.
Yea, that makes sense. You could sort it in chunks, but it would probably be slower. If you are regularly doing that and can afford the ram go for it. Otherwise maybe extract the bits that need to be sorted and zip them back up later.
-
Hi,
I have a friend who is looking to run a few simulations he has implemented in python and needs around 256GB of ram. He is estimating it will take a couple of hours, but he is studying economics so take that with a grain of salt
For this instance, I recommended GCP, but I felt a bit dirty doing that. So, I was wondering if any of you have a buttload of memory he can burrow? Generally, would you lend your RAM for a short amount of time to a stranger over the internet? (assuming internet acccess is limited to a signle ssh port, other necessary safeguards are in place)
wrote on last edited by [email protected]Not anymore... Had a box of old/junk parts lying around before I moved but I didn't bring it with me because it was essentially just garbage I never got around to taking to the electronic recycling thing because it was just a once a year thing that cost me money.
I'd have no problem giving away stuff like that. I wouldn't take parts out of an active machine to let someone "borrow" tho.
-
put your butt on a scale, convert the result to RAM, duh
wrote on last edited by [email protected]Yup, that’s some random ass memory.
-
Borrow it from NewEgg, then return it
Newegg isn't so bad. Do a shit corporate like best buy.
-
Hi,
I have a friend who is looking to run a few simulations he has implemented in python and needs around 256GB of ram. He is estimating it will take a couple of hours, but he is studying economics so take that with a grain of salt
For this instance, I recommended GCP, but I felt a bit dirty doing that. So, I was wondering if any of you have a buttload of memory he can burrow? Generally, would you lend your RAM for a short amount of time to a stranger over the internet? (assuming internet acccess is limited to a signle ssh port, other necessary safeguards are in place)
As a hardware guy there is so little info here
DDR2, 3, 4, or 5? Clock speed? ECC? Registered?
Yeah I have boxes of older memory. But there needs to be a lot more specifics. Most of my home lab machines have at least 384gb (VMs need a lot of memory).
-
Hi,
I have a friend who is looking to run a few simulations he has implemented in python and needs around 256GB of ram. He is estimating it will take a couple of hours, but he is studying economics so take that with a grain of salt
For this instance, I recommended GCP, but I felt a bit dirty doing that. So, I was wondering if any of you have a buttload of memory he can burrow? Generally, would you lend your RAM for a short amount of time to a stranger over the internet? (assuming internet acccess is limited to a signle ssh port, other necessary safeguards are in place)
First of all, he should drop Python for anything resource intensive as such a simulation. And then think about how to optimize the algorithm.
-
Hi,
I have a friend who is looking to run a few simulations he has implemented in python and needs around 256GB of ram. He is estimating it will take a couple of hours, but he is studying economics so take that with a grain of salt
For this instance, I recommended GCP, but I felt a bit dirty doing that. So, I was wondering if any of you have a buttload of memory he can burrow? Generally, would you lend your RAM for a short amount of time to a stranger over the internet? (assuming internet acccess is limited to a signle ssh port, other necessary safeguards are in place)
wrote on last edited by [email protected]I do not have any RAM to share, sorry.
Economics simulation + Python needing 200+GB of RAM sounds preventable.
In your friend's shoes, I might start asking for pointers over on the programming.dev Lemmy.
As others have said, a rewrite in a faster language like C or goLang could help - but my guess is there's also ways to cut that memory need way down, while still using Python.
-
As a hardware guy there is so little info here
DDR2, 3, 4, or 5? Clock speed? ECC? Registered?
Yeah I have boxes of older memory. But there needs to be a lot more specifics. Most of my home lab machines have at least 384gb (VMs need a lot of memory).
I don't think OP wants you to lend them physical RAM modules but asks about letting his friend run random code on your high-RAM machine.
-
I don't think OP wants you to lend them physical RAM modules but asks about letting his friend run random code on your high-RAM machine.
Maybe? After rereading it I’m really not sure …
-
Hi,
I have a friend who is looking to run a few simulations he has implemented in python and needs around 256GB of ram. He is estimating it will take a couple of hours, but he is studying economics so take that with a grain of salt
For this instance, I recommended GCP, but I felt a bit dirty doing that. So, I was wondering if any of you have a buttload of memory he can burrow? Generally, would you lend your RAM for a short amount of time to a stranger over the internet? (assuming internet acccess is limited to a signle ssh port, other necessary safeguards are in place)
First, define what you are asking for.
Do you want someone to send you a cardboard box full of RAM? Then forget it. Nobody would be stupid enough to lend that much expensive hardware to someone on the internet.
Or are you asking for someone to let you run random code on their PC for a few hours? Then forget it. Nobody would be stupid enough to open "a single SSH port" to someone on the internet to run potential malware on their PC.
That's exactly what cloud platforms are there for, and if you don't like google, get any other cloud provider.
-
Maybe? After rereading it I’m really not sure …
That's at least what I got from the comment with the SSH port.
-
That's at least what I got from the comment with the SSH port.
Yeah I can definitely see your point.
-
Hi,
I have a friend who is looking to run a few simulations he has implemented in python and needs around 256GB of ram. He is estimating it will take a couple of hours, but he is studying economics so take that with a grain of salt
For this instance, I recommended GCP, but I felt a bit dirty doing that. So, I was wondering if any of you have a buttload of memory he can burrow? Generally, would you lend your RAM for a short amount of time to a stranger over the internet? (assuming internet acccess is limited to a signle ssh port, other necessary safeguards are in place)
All my extra RAM was super old and I let it get offed when I hired a junk hauling company clear out my last place when I moved (I'd been there for like 15 years, so there was a lot of worn out furniture and stuff).
-
Hi,
I have a friend who is looking to run a few simulations he has implemented in python and needs around 256GB of ram. He is estimating it will take a couple of hours, but he is studying economics so take that with a grain of salt
For this instance, I recommended GCP, but I felt a bit dirty doing that. So, I was wondering if any of you have a buttload of memory he can burrow? Generally, would you lend your RAM for a short amount of time to a stranger over the internet? (assuming internet acccess is limited to a signle ssh port, other necessary safeguards are in place)
i have 8 gigs thats been living on my desk for the last 4 years
-
First, define what you are asking for.
Do you want someone to send you a cardboard box full of RAM? Then forget it. Nobody would be stupid enough to lend that much expensive hardware to someone on the internet.
Or are you asking for someone to let you run random code on their PC for a few hours? Then forget it. Nobody would be stupid enough to open "a single SSH port" to someone on the internet to run potential malware on their PC.
That's exactly what cloud platforms are there for, and if you don't like google, get any other cloud provider.
Seconded. If they can't optimize their code (which, I have never seen applications require 256 gigs of ram even in FAANG so I find that doubtful), then they need to rent a machine. The cloud is where you rent it. If not Google, then AWS, Azure, Digital Ocean, any number of places let you rent compute
-
Needing that much RAM is usually a red flag that the algo is not optimized.
*looking at the 14TB cluster I had running for 18 hours
Yep, nobody would ever need that much memory
-
Hi,
I have a friend who is looking to run a few simulations he has implemented in python and needs around 256GB of ram. He is estimating it will take a couple of hours, but he is studying economics so take that with a grain of salt
For this instance, I recommended GCP, but I felt a bit dirty doing that. So, I was wondering if any of you have a buttload of memory he can burrow? Generally, would you lend your RAM for a short amount of time to a stranger over the internet? (assuming internet acccess is limited to a signle ssh port, other necessary safeguards are in place)
Tell your friend to open source the algorithm.. Somebody will surely point at a easy optimization.
100 others will just shit on your friend -
Seconded. If they can't optimize their code (which, I have never seen applications require 256 gigs of ram even in FAANG so I find that doubtful), then they need to rent a machine. The cloud is where you rent it. If not Google, then AWS, Azure, Digital Ocean, any number of places let you rent compute
Yeah, it's an economics student running something on python. I can guarantee that it's horribly unoptimized.
-
*looking at the 14TB cluster I had running for 18 hours
Yep, nobody would ever need that much memory
Wow, yea I think you win that contest lol.
-
Wow, yea I think you win that contest lol.
To be honest, it was a very paralell process. I could do a fraction of the compute, needing a fraction of the RAM, but taking a shit ton more time.
Also, theres no perfect machine for this use. I can have 3.5 times more RAM than needed, or start swapping and waste time.
