introducing copyparty, the FOSS file server
-
I made a video about copyparty, the selfhosted fileserver I've been making for the past 5 years.
The main focus of the video is the features, but it also touches upon configuration. Was hoping it would be easier to follow than the readme on github... not sure how well that went, but hey
This video is also available to watch on the copyparty demo server, as a high-quality AV1 file and a lower-quality h264.
Everyone loves CP. Tell all your friends about CP.
-
I made a video about copyparty, the selfhosted fileserver I've been making for the past 5 years.
The main focus of the video is the features, but it also touches upon configuration. Was hoping it would be easier to follow than the readme on github... not sure how well that went, but hey
This video is also available to watch on the copyparty demo server, as a high-quality AV1 file and a lower-quality h264.
Elderly raspberry pi B [✓]
Large portable drive gathering dust [✓]
Guess I'm setting up a locally hosted file server in the near future.
-
I made a video about copyparty, the selfhosted fileserver I've been making for the past 5 years.
The main focus of the video is the features, but it also touches upon configuration. Was hoping it would be easier to follow than the readme on github... not sure how well that went, but hey
This video is also available to watch on the copyparty demo server, as a high-quality AV1 file and a lower-quality h264.
Now it would be interesting to setup a raspberry pi with harddrives plugged in the USB 3 ports
-
Can you point me to the WebDAV code? I’m interested to see your implementation. There are some parts of the spec that are ambiguous, and I like to see how those are implemented in different servers.
sure! my implementation is really basic, just the stuff that's needed to make the clients i've tested happy, so there's probably still clients that won't be able to connect (And i'll fix those as soon as I hear about them!)
httpcli.py is the http methods handler, and the webdav-specific handlers are all next to eachother, propfind // proppatch // lock // unlock // mkcol // and there's also put for the uploads, but that's not entirely webdav-specific, just webdav-aware.
-
What do you use to bundle into one file?
copyparty-sfx.pyis a custom packer (see this reply) created by make-sfx.sh, andcopyparty.pyzis a standard zipapp, created by make-pyz.sh. The zipapp has more disadvantages than thesfx.py, so that's the default/recommended build. -
I have a question, and I want to emphasise thar this is not criticism but a request for dive into technicalities.
In the video you mentioned copyparty has an one-way sync tool. Is there a good reason why it's not two-way, or is this just something you weren't motivated to do?
No worries, good question
The problem with bidirectional filesync is that it's an absolutely massive can of worms, very easy to mess up, and the consequences of messing up are usually the worst kind (loss of data). There's an insane amount of edgecases to keep in mind, and you need to get every edgecase right every single time, otherwise you might wipe someone's vacation photos, or suddenly downgrade someone's keepass database to an older version... And stuff like syncing multiple devices to the same server makes it balloon further.
I've started becoming more confident in copyparty's filesystem-index database, but it's still just a hint/guideline, with the filesystem being the only source of truth -- it's still not something I'd trust with tracking sync-state against one or more clients.
The bigger guys who offer bidirectional sync (nextcloud, syncthing, etc.) have spent years perfecting their logic, so I'd like to leave this in their capable hands.
-
Everyone loves CP. Tell all your friends about CP.
-
No worries, good question
The problem with bidirectional filesync is that it's an absolutely massive can of worms, very easy to mess up, and the consequences of messing up are usually the worst kind (loss of data). There's an insane amount of edgecases to keep in mind, and you need to get every edgecase right every single time, otherwise you might wipe someone's vacation photos, or suddenly downgrade someone's keepass database to an older version... And stuff like syncing multiple devices to the same server makes it balloon further.
I've started becoming more confident in copyparty's filesystem-index database, but it's still just a hint/guideline, with the filesystem being the only source of truth -- it's still not something I'd trust with tracking sync-state against one or more clients.
The bigger guys who offer bidirectional sync (nextcloud, syncthing, etc.) have spent years perfecting their logic, so I'd like to leave this in their capable hands.
Thank you for your answer! Do you think copyparty would work together with Syncthing on the same backing directory, or would they compete for changes etc? Copyparty in this scenario would be for sharing content with friends and occasional remote upload
-
Thank you for your answer! Do you think copyparty would work together with Syncthing on the same backing directory, or would they compete for changes etc? Copyparty in this scenario would be for sharing content with friends and occasional remote upload
that should be totally fine, I think a lot of people are doing that
-
sooo this is one of the things that started with someone saying "wouldn't it be funny if..."
if you open copyparty-sfx.py in a text editor, you'll see how -- but please make sure to use an editor which is able to handle about 600 KiB of comments which contain invalid utf8 / binary garbage
I ended up rolling my own packer since I wanted optimal encoding efficiency, and everything I could find would do stuff like base85 or ucs2 tricks, but it turns out python is perfectly happy with binary garbage in comments if you declare that the file is
latin-1so it realizes all hope is lostthe only drawback of the sfx.py is that it needs to extract to $TEMP before running, so that's the slight advantage of the zipapp (the .pyz alternative), but that suffers from some performance reduction in return, and is more hermetic (doesn't let you swap out the bundled dependencies with fresh versions as easily if necessary)
Ah, reminds me of the old self-extracting gzip executable trick. I used that once a very long time ago to make a 4k linux intro, before I realized to be competitive I should switch to windows to be able to use Crinkler, which is superior even though the decompressor is part of the executable.
-
I made a video about copyparty, the selfhosted fileserver I've been making for the past 5 years.
The main focus of the video is the features, but it also touches upon configuration. Was hoping it would be easier to follow than the readme on github... not sure how well that went, but hey
This video is also available to watch on the copyparty demo server, as a high-quality AV1 file and a lower-quality h264.
Don't believe it. This is easily too good to be true.
-
I replaced nextCloud’s file sharing with Syncthing just recently, as I found myself only synchronising between two computers (one desktop, one smartphone) anyway.
That won’t work for me since I have multiple people using this Nextcloud instance, and I also use it to publicly share files. I also have a big network share (currently 5 TB of data) that I would like to better integrate somehow (right now it’s available from Nextcloud read-only in essentially “anonymous mode” with no access to private user files).
-
I made a video about copyparty, the selfhosted fileserver I've been making for the past 5 years.
The main focus of the video is the features, but it also touches upon configuration. Was hoping it would be easier to follow than the readme on github... not sure how well that went, but hey
This video is also available to watch on the copyparty demo server, as a high-quality AV1 file and a lower-quality h264.
This is cool. Idk if it’s just me but the h.264 version of your video has no sound!
-
No worries, good question
The problem with bidirectional filesync is that it's an absolutely massive can of worms, very easy to mess up, and the consequences of messing up are usually the worst kind (loss of data). There's an insane amount of edgecases to keep in mind, and you need to get every edgecase right every single time, otherwise you might wipe someone's vacation photos, or suddenly downgrade someone's keepass database to an older version... And stuff like syncing multiple devices to the same server makes it balloon further.
I've started becoming more confident in copyparty's filesystem-index database, but it's still just a hint/guideline, with the filesystem being the only source of truth -- it's still not something I'd trust with tracking sync-state against one or more clients.
The bigger guys who offer bidirectional sync (nextcloud, syncthing, etc.) have spent years perfecting their logic, so I'd like to leave this in their capable hands.
Sync is one of those things that seems like it should be trivial but is actually super complicated.
-
I made a video about copyparty, the selfhosted fileserver I've been making for the past 5 years.
The main focus of the video is the features, but it also touches upon configuration. Was hoping it would be easier to follow than the readme on github... not sure how well that went, but hey
This video is also available to watch on the copyparty demo server, as a high-quality AV1 file and a lower-quality h264.
The video are amazing. It's entertaining and explains everything so it's easily understood.
-
This is cool. Idk if it’s just me but the h.264 version of your video has no sound!
man... that's really unexpected, I went with h264+mp3 which should be the most conservative / broadly-supported combination you could possibly use, yet still (ﾉ ﾟヮﾟ)ﾉ ~┻━┻
what webbrowser / device / player are you using?
-
Everyone loves CP. Tell all your friends about CP.
Are China Parties like Tupperware Parties, where friends get together and one shills a pyramid scheme? That's what CP is, right?
-
sure! my implementation is really basic, just the stuff that's needed to make the clients i've tested happy, so there's probably still clients that won't be able to connect (And i'll fix those as soon as I hear about them!)
httpcli.py is the http methods handler, and the webdav-specific handlers are all next to eachother, propfind // proppatch // lock // unlock // mkcol // and there's also put for the uploads, but that's not entirely webdav-specific, just webdav-aware.
Thanks! So you put in the
displaynameprop even if it’s not set by the client. For the life of me, I can’t figure out what that prop is supposed to be in the spec. It calls it a live prop, but doesn’t give an explanation or an equivalent HTTP header.I love how you named the error for 400 statuses, “Pebkac”! xD
-
I made a video about copyparty, the selfhosted fileserver I've been making for the past 5 years.
The main focus of the video is the features, but it also touches upon configuration. Was hoping it would be easier to follow than the readme on github... not sure how well that went, but hey
This video is also available to watch on the copyparty demo server, as a high-quality AV1 file and a lower-quality h264.
Have been playing around with it for a bit and it's AMAZING.
-
I agree but it's still in an early development state. Not really usable for everyday work let alone most people never heard about it
But yeah still cool to mention it under "modern" browsers. I wish them good luck with the first alpha next year. I hope it'll be successful.wrote last edited by [email protected]Can claim "It even works with browsers from the future!"
