I’m installing 3x2TB HDDs into my desktop pc. The drives are like-new.
Basically they will replace an ancient 2tb drive that is failing. The primary purpose will basically be data storage, media, torrents, and some games installed. Losing the drives to failure would not be catastrophic, just annoying.
So now I’m faced with how to set up these drives. I think I’d like to do a RAID to present the drives as one big volume. Here are my thoughts, and hopefully someone can help me make the right choice:
- RAID0: Would have been fine with the risk with 2 drives, but 3 drives seems like it’s tempting fate. But it might be fine, anyhow.
- RAID1: Lose half the capacity, but pretty braindead setup. Left wondering why pick this over RAID10?
- RAID10: Lose half the capacity… left wondering why pick this over RAID1?
- RAID5: Write hole problem in event of sudden shutoff, but I’m not running a data center that needs high reliability. I should probably buy a UPS to mitigate power outages, anyway. Would the parity calculation and all that stuff make this option slow?
I’ve also rejected considering things like ZFS or mdadm, because I don’t want to complicate my setup. Straight btrfs is straightforward.
I found this page where the person basically analyzed the performance of different RAID levels, but not with BTRFS. https://larryjordan.com/articles/real-world-speed-tests-for-different-hdd-raid-levels/ (PDF link with harder numbers in the post). So I’m not even sure if his analysis is at all helpful to me.
If anyone has thoughts on what RAID level is appropriate given my use-case, I’d love to hear it! Particularly if anyone knows about RAID1 vs RAID10 on btrfs.
Setup a raid5 with mdadm, format that with btrfs.
Using btrfs for raid is not recomended by btrfs itself.
But setting up a raid5 mdadm array is very simple and let you use btrfs.
I agree that zfs is too much.
Edit: with three drives, raid1 doesn’t make sense. Raid10 i never seen it in over 20y being used by anybody.
with three drives, raid1 doesn’t make sense
In raid1c2 mode btrfs will give 3TB of usable storage with 3x2TB. It always stores two copies on two drives. Not three.
Cool! I didnt know that… You still waste 50% but you can use 3 disks, good to know
with three drives, raid1 doesn’t make sense
It’s perfectly usable with a btrfs setup. If one drive fails, you can mount in a degraded state.
If you mean using three disks as two by wasting 66% of space to keep a spare drive… I think it’s a waste on a home pc
If you mean to use three disks to get 6tb/2=3tb of available space, then it’s something I only recently got aware of (see previous comment to my comment) so yes.
three disks to get 6tb/2=3tb of available space
Exactly! A happy outcome when using this FS.
guidance on BTRFS RAID
Don’t.
:\
If you just want to combine their capacities, and don’t need redundancy, just use single mode?
No need to use a raid mode for multi-device btrfs.
Edit: You could also do two volumes.
Split each drive in half. Use the first half of each drive for a raid1c2 volume to get 1.5TB of redundant storage for important data.
Use the second half of each drives for a raid0 volume to get 3TB of faster storage for games.
I was unfamiliar with single mode. What advantage does it give me over RAID0 in terms of combining their capacities?
It combines capacity without any fancy striping. It can still provide some performance benefit as different blocks of the same file can be stored on different drives, but it doesn’t stripe data across the drives for performance.
It also allows you to just add more drives later. The drives don’t need to be the same size or type. You can also remove drives, provided there is enough free space to move the data on a drive to the ones that will remain.
It really just pools the storage capacity into one big volume.
If a drive fails, it still takes the whole volume with it tho. But as long as you monitor smart, it is fairly simple to try ejecting a failing drive from the device group before it takes the whole thing with it.
With 3 drives I would go with BTRFS RAID1. You get half the capacity, can lose any one of the drives, and it is really simple.
If you were to add drives later on to increase the array size, I might consider RAID10, but if you plan on sticking with 3 drives, I would go with RAID1 instead.
RAID0 offers no redundancy, so I would skip it.
RAID5/6 are not recommended for use in production.
I run an 8 disk 90tb BTRFS RAID0 and its been going strong for over a year. I hesitate to actually encourage anyone to live this dangerously but its mainly a media server and if i needed to i could restore any lost data from usenet or torrents using my arr stack which is on a different drive. If you can handle the risk its a nice speed boost.
Damn. And I thought 3 disks was risky…
Briefly addressing the RAID types you mentioned:
- RAID-0: OH NO OH GOD PLEASE NO AHHHHHHH
- RAID-1: There’s nothing wrong with it, but it feels very weird to me that BTRFS can do RAID-1 over 3 disks. It’s still technically 2 copies of every block, meaning you can in theory lose any single drive and still recover the data, but idk, it just feels wrong.
- RAID-10: Again, weird that BTRFS allows this with an odd amount of disks. From what I can find, this has no advantage over RAID-1 with 3 disks and can in fact perform worse.
- RAID-5: It’s probably fine if you have backups, and you get the most usable space out of your disks, but performance will take a hit, and scrubs are terribly slow.
Now, you mentioned not wanting ZFS due to complexity, but really, it is no more complex to manage than BTRFS. It’s fairly easy to get it working on any modern Linux distro (Ubuntu has support out-of-the-box, Debian has it packaged as a DKMS module, Arch has it in AUR, and so on).
With ZFS, you could create a RAID-Z1 (equivalent to a RAID-5) without any of the performance penalties or risks that BTRFS RAID-5 has. Both have pretty much them same features, with the main difference that ZFS can’t be “re-balanced” to a different disk layout like BTRFS can, but it will also generally not corrupt your data if you look at it wrong. Everything else maps pretty much 1:1 between them. Both support:
- Transparent compression
- Transparent checksums
- Scrubbing
- Snapshots
- Block-level transfers (btrfs-send / zfs-send)
- Fairly intuitive CLI
- Filesystem (BTRFS) == Pool (ZFS)
- Subvolume (BTRFS) == Dataset (ZFS)
My concern with ZFS is I use Fedora, so the kernel updates really frequently. I know that it kicks ass, but I just like having it straightforward in my kernel that I already have installed so that I never have to deal with a
If kernel module can not be loaded, your kernel version might be not yet supported by OpenZFS. An option is to an LTS kernel from COPR, provided by a third-party. Use it at your own risk:
situation. (https://openzfs.github.io/openzfs-docs/Getting Started/Fedora/index.html)
Zfs has built-in encryption that can be enabled selectively per dataset, which is handy for some private data.
Also, as long as you avoid raidz, you can now add/remove disks to a pool, and attach/detach mirrors too. That’s much more flexible than it used to be.
Too bad raid1 from btrfs has not a zfs equivalent. You can to some extent replicate that via partitioning in zfs, but the simplicity of btrfs raid1 is great for uneven disks.
Has btrfs got its act together with large amounts of snapshots? It used to be a pain point.
I built my own NAS from Arch (btw) and wanted to just install standard packages and use standard tools (ie gparted, clonezilla, etc).
I’ve got btrfs RAID5 running fine.
Had loads of power outages (we had water meeting electrics) and have no problems at all.
Of course, RAID is not a backup, so all my important stuff is on Hetzner too.
I don’t really follow the ZFS news, but I understand that resizing / adding new drives is / was a problem, plus, back when I built this, I would’ve needed kernel patching… there’s a higher chance of those causing a problem - for my use case - than btrfs, so if you’re like me, just crack on and enjoy the easy life.
It kinda seems like you don’t need to be using btrfs. If the possibility of your kernel updates breaking scared you off zfs, a system that has been in very widespread use for just going off the dome twice as long as btrfs has been around, why do you think using btrfs is somehow okay?
I’m not trying to suggest you use zfs either.
E: I went and looked and zfs has been in widespread use for around twice as long as btrfs has been marked as stable. It’s worth mentioning too that since being marked as stable btrfs has suffered from a silent data corruption bug.
I’m not sure what you are suggesting as the alternative. Nor do I know what silent btrfs corruption bug you are referring to, either. Btrfs has been widely deployed in enterprise and personal environments for years, and I cannot find evidence of data loss due to the file system itself.
here’s the Debian mailing list thread about it. It made a splash a little while back. Might be due to a combination of kernels and btrfs but nonetheless that’s exactly the situation you describe yourself trying to avoid.
Not a slag against btrfs or a lauding of zfs, just trying to point out that you might be barking up the wrong tree if you’re looking for stability and simplicity.
I use ext4 volumes in mergerfs with nightly snapraid parity snapshots for my own data that doesn’t matter. Migrated to that system from zfs looking for simplicity, stability and straightforward recovery and gained increased drive life and lowered power consumption as well. Your mileage may vary.
