I made a video about copyparty, the selfhosted fileserver I’ve been making for the past 5 years.
The main focus of the video is the features, but it also touches upon configuration. Was hoping it would be easier to follow than the readme on github… not sure how well that went, but hey :D
This video is also available to watch on the copyparty demo server, as a high-quality AV1 file and a lower-quality h264.
You made this on your phone on the bus ride to and from work.
I cleaned the cat box yesterday and considered that an accomplishment.
Fuck.
Congrats on the cat box cleaning!
Clearly a labour of love 👍
Maybe support for some music streaming apps (subsonic?) would be cool?
That’s a neat idea – I’ve heard that a lot of stuff uses the subsonic API under the hood, so I’ll see what it would take to become compatible with that. At first glance it looks like I’d have to mine and index way more information about audio files, but could still be doable :>
Your readme looks super in depth, thanks for that! I haven’t watched the video yet but will later.
I didn’t see it mentioned from a quick glance, but is either sftp or ftps supported?
SFTP is not currently on the roadmap, but it’s not entirely implausible.
FTPS is supported, but it requires an optional dependency to be installed (pyopenssl), so it’s not available in the Windows EXE. And I just realized that the dependency is currently not present inside the docker images either, so I’ll get that fixed right away.
Screenshots of the ui at the top of the readme would be nice
The fact you mention security features, without ever saying it’s ‘super secure’ tells me you know a lot about what you’re doing. I’m so sick of apps like this that start with “most secure app on the net” but you know they’re delusional. Thank you, going to check this out.
I have a question, and I want to emphasise thar this is not criticism but a request for dive into technicalities.
In the video you mentioned copyparty has an one-way sync tool. Is there a good reason why it’s not two-way, or is this just something you weren’t motivated to do?
No worries, good question :>
The problem with bidirectional filesync is that it’s an absolutely massive can of worms, very easy to mess up, and the consequences of messing up are usually the worst kind (loss of data). There’s an insane amount of edgecases to keep in mind, and you need to get every edgecase right every single time, otherwise you might wipe someone’s vacation photos, or suddenly downgrade someone’s keepass database to an older version… And stuff like syncing multiple devices to the same server makes it balloon further.
I’ve started becoming more confident in copyparty’s filesystem-index database, but it’s still just a hint/guideline, with the filesystem being the only source of truth – it’s still not something I’d trust with tracking sync-state against one or more clients.
The bigger guys who offer bidirectional sync (nextcloud, syncthing, etc.) have spent years perfecting their logic, so I’d like to leave this in their capable hands.
Sync is one of those things that seems like it should be trivial but is actually super complicated.
Thank you for your answer! Do you think copyparty would work together with Syncthing on the same backing directory, or would they compete for changes etc? Copyparty in this scenario would be for sharing content with friends and occasional remote upload
that should be totally fine, I think a lot of people are doing that :>
Look cool! I think you should consider putting a screenshot of the UI somewhere near the top of the README
I would probably remove python 2 support, it was end of life when the project was started.
As long as it’s not causing any issues or drawbacks for modern python versions (and it isn’t), I don’t see any reason to do that – on the contrary, I know people are running copyparty on retro equipment, so I’d very much prefer to keep it for as long as possible :>
I don’t think there is a Python 2 version for retro equipment (DOS?). ;-)
Hey fellow scener, cool project!
Just a few thoughts/questions:
- BTRFS and ZFS support real deduplication via copy on write, and would eliminate all current disadvantages of symlink and hardlink deduplication. It just works.
- Why have it be one huge python source file? This is a serious code smell imo, and something you really should avoid doing as this can be a major maintenance burden.
BTRFS and ZFS support real deduplication via copy on write, and would eliminate all current disadvantages of symlink and hardlink deduplication. It just works.
yeah that’s a good point, I’ll add an option to take advantage of this if you know you’re running on a filesystem where that works as intended.
Why have it be one huge python source file?
oh don’t worry, it’s all separate files during development – there’s a build-stage which bundles everything up into a single file for distribution. But thanks for the concern :D
Ah, so you have compiled it into one file? Didn’t know that was possible for python, what tool do you use for this?
sooo this is one of the things that started with someone saying “wouldn’t it be funny if…”
if you open copyparty-sfx.py in a text editor, you’ll see how – but please make sure to use an editor which is able to handle about 600 KiB of comments which contain invalid utf8 / binary garbage 😁
I ended up rolling my own packer since I wanted optimal encoding efficiency, and everything I could find would do stuff like base85 or ucs2 tricks, but it turns out python is perfectly happy with binary garbage in comments if you declare that the file is
latin-1so it realizes all hope is lost :Dthe only drawback of the sfx.py is that it needs to extract to $TEMP before running, so that’s the slight advantage of the zipapp (the .pyz alternative), but that suffers from some performance reduction in return, and is more hermetic (doesn’t let you swap out the bundled dependencies with fresh versions as easily if necessary)
Ah, reminds me of the old self-extracting gzip executable trick. I used that once a very long time ago to make a 4k linux intro, before I realized to be competitive I should switch to windows to be able to use Crinkler, which is superior even though the decompressor is part of the executable.
Why not use zipapp?
What do you use to bundle into one file?
copyparty-sfx.pyis a custom packer (see this reply) created by make-sfx.sh, andcopyparty.pyzis a standard zipapp, created by make-pyz.sh. The zipapp has more disadvantages than thesfx.py, so that’s the default/recommended build.
Just a remark from someone who runs ZFS since the beginning. Many people don’t like the deduplication feature because of its memory footprint.
It’s also nice to have this feature without relying on a certain filesystem.
Put out some in-depth docker instructions and this will be common use in a month. Good work.
Yup, I tried to run the docker image with the suggested docker command and it errored out for lack of a config file (though it did offer a fix in the logs for mounting the current directory as read/write)
Everyone loves CP. Tell all your friends about CP.
I bumped into copyparty the other month when looking for a software that could let me transfer files to a friend with the ability to pause/resume. Didn’t bother with it, tried another software instead. Never really got it to work so I gave up on it.
Bumped into the YouTube video today, decided to give copyparty a shot, damn sir you’ve written a fine piece of software. It’s so easy to get up and configure. The UI is a bit janky, but charming at the same time. Thanks for all the hard work!
Oh my gawd what a README!! I’m on my phone and I was trying to scroll back to the top of it from the bottom and I just kept on scrolling… Holy shit I’m going to put this on my kanban board give it proper attention
At this rate might be faster to read code than a read me. Or convert to a wiki style if this much details are really needed.
Don’t believe it. This is easily too good to be true.
Ho… Ly… Shit… This is great! The UI is a bit confusing at first but doesn’t take long to get what’s going on. I might even be disappointed with a UI revamp 😁 I can’t believe how much functionality this has. It’s already replacing some processes I have for mounting drives and backing up files. Maybe I missed something, but my only complaint would be the lack of an automatic one-way folder sync in the Party UP! app.
I’m blown away, great job!
That “functional but charming” UI reminds me of audiobookshelf with the soundleaf app - same vibe where it dosn’t look fancy but gets the job done better than most polished alternatives!
Oh my god, this seems really good and closer to what I want than anything yet. Been looking for something to replace Nextcloud and found nothing good so I might take a look at this.
I replaced nextCloud’s file sharing with Syncthing just recently, as I found myself only synchronising between two computers (one desktop, one smartphone) anyway. :-)
That won’t work for me since I have multiple people using this Nextcloud instance, and I also use it to publicly share files. I also have a big network share (currently 5 TB of data) that I would like to better integrate somehow (right now it’s available from Nextcloud read-only in essentially “anonymous mode” with no access to private user files).
Just curious, why did you replace nextcloud? I’m looking into transitioning from my current file server, and I’ve mostly heard only good things and not NextCloud.
Nextcloud is like Windows 95, it works great when you install it then it just keeps getting slower as you fill it with content
Also, it’s like Wordpress, growing new slow features you’ll never need with each new release.
