Goal:
- 16TB mirrored on 2 drives (raid 1)
- Hardware raid?
- Immich, Jellyfin and Nextcloud. (All docker)
- N100, 8+ GB RAM
- 500gb boot drive ssd
- 4 HDD bays, start with using 2
Questions:
- Which os?
- My though was to use hardware raid, and just set that up for the 2 hdds, then boot off an ssd with Debian (very familiar, and use it for current server which has 30+ docker containers. Basically I like and am good at docker so would like to stick to Debian+docker. But if hardware raid isn’t the best option for HDDs now a days, I’ll learn the better thing)
- Which drives? Renewed or refurb are half the cost, so should I buy extra used ones, and just be ready to swap when the fail?
- Which motherboard?
- Which case?
ProxMox.
But also, in case if your only data backup plan for them is raid 1 - in such cases I prefer to have only one HDD in the machine & use the other one as a backup on a separate machine, preferably in another location. I find it that the missing 12h (or whatever) of the latest data overshadows the (lower) probability of losing all the data (fire, flood, burglary, weirdly specific accidents, etc).
And ofc you can select what to backup/rsync or not.
Eg Immich, after return to operation the apps will just resync any missing pics from the last backup.
Also with two systems you don’t have to care that much about drive quality. Im now buying Exos 22+TB bcs why not. But when I needed quiet drives I bought Red Plus (not the regular Red ones, nor the Pro ones), they are even quieter than Exos, but smol.
I’d recommend BTRFS in RAID1 over hardware or mdadm raid. You get FS snapshotting as a feature, which would be nice before running a system update.
For disk drives, I’d recommend new if you can afford them. You should look into shucking: It’s where you buy an external drive and then remove (shuck) the HDD from inside. You can get enterprise grade disks for cheaper than buying that same disk on its own. The website https://shucks.top tracks the price of various disk drives, letting you know when there are good deals.
How does the replacing of a HDD work on btrfs? Like if one failed and I’m using Debian, how do I rebuild the raid 1?
Or should I use an actual raid os?
Assuming that the disk is of identical (or greater) capacity to the one being replaced, you can run
btrfs replace.https://wiki.tnonline.net/w/Btrfs/Replacing_a_disk#Replacing_with_equal_sized_or_a_larger_disk
Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I’ve seen in this thread:
Fewer Letters More Letters DNS Domain Name Service/System LVM (Linux) Logical Volume Manager for filesystem mapping LXC Linux Containers NAS Network-Attached Storage NUC Next Unit of Computing brand of Intel small computers PCIe Peripheral Component Interconnect Express PSU Power Supply Unit Plex Brand of media server package RAID Redundant Array of Independent Disks for mass storage SATA Serial AT Attachment interface for mass storage SSD Solid State Drive mass storage VPN Virtual Private Network VPS Virtual Private Server (opposed to shared hosting) ZFS Solaris/Linux filesystem focusing on data integrity
14 acronyms in this thread; the most compressed thread commented on today has 19 acronyms.
[Thread #872 for this sub, first seen 15th Jul 2024, 16:35] [FAQ] [Full list] [Contact] [Source code]
If you can do at least three nodes with high availability. It is more expensive and trickier to setup but in the long run it is worth it when hosting for others. You can literally unplug a system and it will fail over
It is overkill but you can use Proxmox with a docker swarm.
Again way overkill but future proof and reliable
I think this is the way and not an overkill at all!
Its super easy to swarm ProxMox, and you make your inevitable admin job easier. Not to mention backups, first testing & setting up a VM on your server before copying it to their, etc.
You need at minimum three ceph nodes but actually four if you want it to work better. But ceph isn’t ideally designed in mind with clusters that small. 7 nodes would be more reasonable.
While clustering proxmox using ceph is cool as fuck it’s not easy or cheap to accomplish at home.
No, I didn’t mean with ceph. No quorum needed either. Just add to the group.
Ah, so you mean completely independent hosts.
Yes, it’s still easier to manage, transfer stuff, Proxmox Backup server integrates nicely, etc.
If you want to build it yourself, you have to decide on size.
Are you trying to keep it as small as possible?
Do you want a dedicated GPU for multiple jellyfin streams? (Definitely get the Intel A380, cheap and an encoding beast)
If you don’t want to start a rack and don’t want to go with a prebuilt NUC, there are 2 PC cases I would recommend.
Node 304 and Node 804.
Node 304 is mini-ITX (1 PCIe slot, 1 M.2 slot for boot OS, 4 HDDs, SFX-L PSU, and great cooling)
Node 804 is micro-ATX (2 PCIe slots, 2 M.2 slots, 8-10 HDDs, ATX PSU, and 2 chambers for the HDDs to stay cool)
Why do you want a N100? Is electricity very expensive where you are that idle power is a big factor? Because desktop CPUs are more powerful and the CPUs can idle down to 10W or so without a GPU and they can have way more RAM.
Tldr; go with prebuilt NUC or go with a desktop CPU for a custom build.
No hardware RAID. Use zfs, if you can. Mirror the boot SSD. I would use a stripe over mirror and 4 HDDs. Two drives are not enough redundancy. Use enterprise or nearline drives, if you can. Debian is great, you can install Proxmox on top of it, but from the sound of it plain Debian would work for you.
Don’t use hardware RAID, use a nice software RAID like zfs. 2 HDDS and an OS SSD would be a great use case for zfs.
Then just go with debian+docker. As raid software i would recommend ZFS, its a filesystem that does both and also integrity on file level. (and lots more)
I personally would only buy new ones. No matter the brand just the best TB/€ you can get.
For MB basically every Chipset gives you 4 SATA ports. You could consider picking one that Supports unbuffered ECC memory but that is not a must. If you want to Hardware Transcode in Jellyfin, then Intel is probably your best since the dGPU with Quicksync is pretty good and well supported, otherwise i would go AMD.
For 4 drives you can use most ATX cases have no recommendations here.
Just make sure the drives are CMR, not SMR.
You don’t want hardware raid. Some options you can research:
- Mdadm - Linux software raid
- ZFS - Combo raid and filesystem
- Btrfs - A filesystem that can also do raid things
Some OS options to consider:
- Debian - good if you want to learn to do everything yourself
- Truenas Scale - Comercial NAS OS. I bit of work to get started, but very stable once going.
- Unraid - Enthusiast focused NAS OS. Not as stable as Truenas, but easier to get started and a lot of community support.
There are probably other software/OS’s to consider, but those are the ones I have any experience with. I personally use ZFS on Truenas with a lot of help from this YouTube channel. https://youtube.com/@lawrencesystems?si=O1Z4BuEjogjdsslF
Ditto on hardware raid. Adding a hardware controller just inserts a potentially catastrophic point of failure. With software raid and raid-likes, you can probably recover/rebuild, and it’s not like the overhead is the big burden it was back in the 90s.
Just want to mention that TrueNAS is FOSS and unRAID is not. And I wouldn’t necessarily say that unRAID is much easier.
I would add LVM to the list of software raids, and remove btrfs as poorly engineered.
