Having gradually built up my media collection to near the capacity of my 16TB external HDD, I’ve reached the point where I’ll probably need to build a RAID array to keep the collection in one place. Assuming the RAID array will be at least 32TB, I have a few questions:
-
From what I’ve read RAID arrays can help mitigate the risk of individual drives failing if extra space is allotted on the hard drives. Assuming a total capacity of 32TB, how much of that space would be reserved by the RAID array for data loss prevention?
-
Is there a certain type of hard drive I would have to use? Aside from my 16TB drive, I also have two 2 8TB drives that I’d ideally like to be able to re-use in the RAID array, but have left them in their enclosures for the time being.
-
If the hard drives in the array have different transfer speeds, does the array as a whole default to the slowest one?
-
Whether the hard drives I already have are compatible or not, what RAID enclosure and hard drives would you recommend?
- MalReynolds ( @MalReynolds@slrpnk.net ) English25•9 months ago
The general datahoarder hive mind seems to be moving away from shucking and towards manufacturer refurbished drives, see https://serverpartdeals.com/ for example, especially in RAID where you can lose a drive with impunity and warranty is uncomplicated by shucking.
In RAID and similar strategies the redundancy comes from a parity drive, which protects against the loss of one (or more with some schemes) drive, so if you only have two drives of the same size it’s just a mirror (50% of total pool) but you can have a drive die without data loss. With four drives you get to use three of the drives (75%), five you can use four (80%) etc. Classical RAID uses identical sized disks but there are other approaches that allow different sizes e.g. mergerfs + SnapRAID or Unraid, here you lose your largest disk to parity.
RAID is generally faster than the individual drives, e.g. mirroring is nearly twice as fast.
Perhaps go with another 2*8Tb which will get you 24Tb usable and use the 16Tb for offline, preferably offsite backup (remember RAID is not a backup, it protects you from disk failure, but not user error for example accidentally deleting things)
- quirzle ( @quirzle@kbin.social ) 4•9 months ago
- https://www.synology.com/en-au/support/RAID_calculator or similar is good to easily do these calculations
- No, but more RAID configurations than not are limited by the smallest size drive. It’s a factor to consider, assuming you can’t afford to just buy a bunch of disks. I wound up maintaining two separate NAS devices, one of which gets my old, smaller disks.
- Generally yes, though you’d be surprised how little difference disk speed makes once you get enough of them in an array.
- I use Synology with various shucked WD externals. I have a bunch of other stuff in my homelab though, so I need the storage to not be it’s own project, else I likely would have built something less expensive. I’m sure there will be better suggestions in this thread than mine.
- Nach [Ohio] ( @Nach@midwest.social ) English3•9 months ago
Start googling. Hardware or Software based? How techy are you? RAID should not be considered for backup. Think of it as an individual drive. Still needs a backup. Personally I use a ZFS pool in my Linux homelab that’s been rock solid. However the pucker factor would be high if a drive fails.
-
Depends on how you set them up. There are calculators online that can estimate this for you.
-
I used NAS drives bc they’re expected to be spinning a lot.
-
No. You will see better performance on average compared to your slowest drive.
-
I have no relevant experience with specific hardware. For a general base of knowledge figure out what RAID type you want to use. Look at what ZFS or btrfs can do in this space. Same for unraid.
-
- Chahk ( @chahk@beehaw.org ) English3•9 months ago
Are you planning to use this only for media storage, or other tasks as well?
Many dedicated NAS boxes from the likes of Synology and QNAP come with low-powered CPUs that in addition to handling media storage can do many other useful things. For example - running various self-hosted apps like password managers, photo organizers, even office suites. Those do have many downsides as well. They use proprietary hardware and software, so you are at the mercy of the manufacturer if they suddenly decide that your NAS is too old for them to support. The CPUs in them are not as powerful as consumer-grade ones found in most desktops, and they don’t usually come with a lot of RAM. They are also quite expensive for what they offer - for example, a very basic 4-bay enclosure with 2GB RAM and Realtek 4-core CPU costs $370, and that’s without the cost of the disks. For that kind of money you can build a desktop from used server parts that will be ten times as powerful, and more versatile since it will be able to run any OS that you want. Granted, it will not be as small or power efficient.
If you just need a turn-key off-the-shelf solution, a prebuilt NAS may be the way to go. If you want to tinker with hardware and experiment with software however, self-built system with used parts may be better suited for you. Most homelabbers don’t start off with a rack full of equipment, they grow their hardware organically. You have outgrown your external enclosure, so now you have several choices. For prebuilt NAS appliances Synology and QNAP are not the only choices; there are many other lesser known in this category - Asustor, Terramaster, Buffalo, to name a few. For building a server on your own I would recommend doing some research before going on a shopping spree - https://serverbuilds.net is an excellent resource.
That said, I’d echo others’ answers. RAID is not backup. It offers some resiliency from disk failure, but that shouldn’t replace a proper backup strategy. Synology offers various RAID levels, including their own take on it that allows for different sized drives. For self-built option there’s UnRaid which handles resiliency differently, but requires an entire separate drive for parity.
In terms of hard drive choice, it all depends on how comfortable you are with possible data loss. If you back up your critical data (e.g. family photos, important documents) and don’t care about downloaded media (you can always re-download it) then go ahead and use whatever - shucked, used, etc. If you absolutely cannot handle any data loss, then new drives made specifically for NAS appliances are the way to go. Enterprise level HDDs are an option also, but those mostly use SAS interface instead of SATA, and require specialized hardware (HBA adapters).
- Altima NEO ( @altima_neo@lemmy.zip ) English2•9 months ago
Gonna go with MalReynolds and mention buying refurbished drives over shucking. You are able to get a drive tailored for server use, rather than the crappy drives theyve been sticking in external enclosures lately. In a some cases, youre even getting a unused drive, even though its refurbished.
- BlueAure ( @BlueAure@infosec.pub ) English1•9 months ago
I use mergerfs to pool a bunch of varying sized drives and then run snapraid on the whole pool to protect it. Then, I’ve got Kopia backups running and backup up to a remote repository. This solution has been flexible and I’ve already been able to recover from 2 drive failures very quickly and easily.
- LarmyOfLone ( @LarmyOfLone@lemm.ee ) English1•9 months ago
There is btrfs - a linux filesystem. It keeps hashes for blocks to prevent data rot. And you can configure it into a raid that duplicates data across drives but doesn’t require the exact same drives. You can just group any old drives together to create one large volume where everything is duplicated. At least that is how I understand this, I’m new to this too.