What is your favourite way to transfer files in your homelab?
-
Ye old samba share.
But I do like using Nextcloud. I use it for syncing my video projects so I can pick up where I left off on another computer.
Samba Bamba!!
-
If you want to use the terminal though, there is
scpwhich is supported on both windows and Linux.Its just
scp [file to copy] [username]@[server IP]:[remote location]That's essentially the same as rsync
-
Hello selfhosted!
Sometimes I have to transfer big files or a large amounts of small files in my homelab. I used rsync but specifying the IP address and the folders and everything is bit fiddly. I thought about writing a bash script but before I do that I wanted to ask you about your favourite way to achieve this. Maybe I am missing out on an awesome tool I wasn't even thinking about.I work from home, however my two systems (home and work) are on the same LAN, they don't see each other for file sharing. I get paid via direct deposit like everyone else which means my pay stubs are all electronic. I print those out and then use WinSCP to copy those over to my desktop. No other files are ever sent.
At home, depending on the amount of files, I either use SFTP via Filezilla, or if the mood strikes me and for a single file, I will just use SCP if I'm already on the cli which is most of the time it seems anymore doing work on my personal servers. I've found that SFTP is faster at transferring than doing a copy/paste to the NFS share to the same drive.
-
S [email protected] shared this topic on
-
Hello selfhosted!
Sometimes I have to transfer big files or a large amounts of small files in my homelab. I used rsync but specifying the IP address and the folders and everything is bit fiddly. I thought about writing a bash script but before I do that I wanted to ask you about your favourite way to achieve this. Maybe I am missing out on an awesome tool I wasn't even thinking about.smb share if its desktop to desktop. If its from phone to PC, I throw it on nextcloud on the phone, then grab it from the web ui on pc.
Smb is the way to go if you have identity set up, since your PC auth will carry over for the connection to the smb share. Nextcloud will be less typing if not since you can just have persistent auth on the app / web.
-
I just type
sftp://[ip, domain or SSH alias]into my file manager and browse it as a regular folder -
Sounds very straight forward. Do you have a samba docker container running on your server or how do you do that?
Set up smb on my file share VM.
My dedicated docker host accesses it through an NFS mount. -
That's essentially the same as rsync
Just slower if you already have some of the files there.
-
Hello selfhosted!
Sometimes I have to transfer big files or a large amounts of small files in my homelab. I used rsync but specifying the IP address and the folders and everything is bit fiddly. I thought about writing a bash script but before I do that I wanted to ask you about your favourite way to achieve this. Maybe I am missing out on an awesome tool I wasn't even thinking about.What's wrong with rsync? If you don't like IP addresses, use a domain name. If you use certificate authentication, you can tab complete the folders. It's a really nice UX IMO.
If you'll do this a lot, just mount the target directory with sshfs or NFS. Then use rsync or a GUI file manager.
-
What's wrong with rsync? If you don't like IP addresses, use a domain name. If you use certificate authentication, you can tab complete the folders. It's a really nice UX IMO.
If you'll do this a lot, just mount the target directory with sshfs or NFS. Then use rsync or a GUI file manager.
I never even set up DNS for things that aren't public facing. I just keep /etc/hosts updated everywhere and ssh/scp/rsync things around using their non-fqdn hostnames.
-
I never even set up DNS for things that aren't public facing. I just keep /etc/hosts updated everywhere and ssh/scp/rsync things around using their non-fqdn hostnames.
You could also use mDNS to the same effect.
-
Linux is truly extensible and it is the part I both love and struggle to explain the most.
I can sit at my desktop, developing code that physically resides on my server and interact with it from my laptop. This does not require any strange janky setup, it's just SSH. It's extensible. -
Linux is truly extensible and it is the part I both love and struggle to explain the most.
I can sit at my desktop, developing code that physically resides on my server and interact with it from my laptop. This does not require any strange janky setup, it's just SSH. It's extensible.I love this so much. When I first switched to Linux, being able to just list a bunch of server aliases along with the private key references in my .ssh/config made my life SO much easier then the redundantly maintained and hard to manage putty and winscp configurations.
-
Sounds very straight forward. Do you have a samba docker container running on your server or how do you do that?
Do you really need a container for Samba?
I see the benefits of containers, but a use would be overkill.
-
Not gonna lie, I just map a network share and copy and paste through the gui.
Yeah, I mean I do still use rsync for the stuff that would take a long time, but for one-off file movement I just use a mounted network drive in the normal file browser, including on Windows and MacOS machines.
-
Do you really need a container for Samba?
I see the benefits of containers, but a use would be overkill.
Yeah, if OP has command line access through rsync then the server is already configured to allow remote access over NFS or SMB or SSH or FTP or whatever. Setting up a mounted folder through whatever file browser (including the default Windows Explorer in Windows or Finder in MacOS) over the same protocol should be trivial, and not require any additional server side configuration.
-
People have already covered most of the tools I typically use, but one I haven't seen listed yet that is sometimes convenient is
python3 -m http.serverwhich runs a small web server that shares whatever is in the directory you launched it from. I've used that to download files onto my phone before when I didn't have the right USB cables/adapters handy as well as for getting data out of VMs when I didn't want to bother setting up something more complex.Honestly, this is an easy way to share files with non-technical people in the outside world, too. Just open up a port for that very specific purpose, send the link to your friend, watch the one file get downloaded, and then close the port and turn off the http server.
It's technically not very secure, so it's a bad idea to leave that unattended, but you can always encrypt a zip file to send it and let that file level encryption kinda make up for lack of network level encryption. And as a one-off thing, you should close up your firewall/port forwarding when you're done.
-
Hello selfhosted!
Sometimes I have to transfer big files or a large amounts of small files in my homelab. I used rsync but specifying the IP address and the folders and everything is bit fiddly. I thought about writing a bash script but before I do that I wanted to ask you about your favourite way to achieve this. Maybe I am missing out on an awesome tool I wasn't even thinking about.sftp
All my machines have my keys, nothing to set up, nothing to tear down.
-
smb share if its desktop to desktop. If its from phone to PC, I throw it on nextcloud on the phone, then grab it from the web ui on pc.
Smb is the way to go if you have identity set up, since your PC auth will carry over for the connection to the smb share. Nextcloud will be less typing if not since you can just have persistent auth on the app / web.
Solid explorer on android is pretty useful too, it can access the SMB share. I use nextcloud for photo backup, but usually solid explorer for one off file transfers.
-
Hello selfhosted!
Sometimes I have to transfer big files or a large amounts of small files in my homelab. I used rsync but specifying the IP address and the folders and everything is bit fiddly. I thought about writing a bash script but before I do that I wanted to ask you about your favourite way to achieve this. Maybe I am missing out on an awesome tool I wasn't even thinking about.What do you mean by specifying IP address?
-
What's wrong with rsync? If you don't like IP addresses, use a domain name. If you use certificate authentication, you can tab complete the folders. It's a really nice UX IMO.
If you'll do this a lot, just mount the target directory with sshfs or NFS. Then use rsync or a GUI file manager.
Just don't run rsync as a daemon as that's a security nightmare
