10

One of the cool things about btrfs on linux is that it can correct bit rot if it has redundant data because of its per-block checksumming. I can get redundant data by setting up a raid1 with two disks. However, can I also get redundant data to prevent bit rot on a single disk?

I see that btrfs has a DUP option for metadata (-m dup) that stores two copies of the metadata on each drive. However, the documentation says that dup is not an option for data (i.e. -d dup is not an option). Is there a good way around this? Partition a single disk into two equal parts and raid1 them together?

Alternatively, is there another simple way to get file system level error detection and correction on linux (something like an automatic parchive for file systems)?

(I'm not interested in answers suggesting that I use two drives.)

EDIT: I did find this, which is a FUSE filesystem that mounts files with error correction as normal files. That said, it's a little hack/proof of concept the someone put together in 2009 and hasn't really touched since.

3 Answers 3

1

Btrfs supports duplicated data blocks if you enable mixed block groups:

mkfs.btrfs --mixed --metadata dup --data dup /dev/<device>

EDIT: Notice that there is a patch so that we can do this without using mixed mode. Following that thread of Nov. 2015, it appears it is being added to the mainline btrfs code.

1
  • --mixed is not recommended above 1GB filesystems. Commented Sep 17, 2019 at 16:04
1

What about partitioning the drive with, eg., LVM into, say, 10 partitions and raid-5-ifying them on the block level? Wouldn't this give around 10% of redundancy for error correction?

4
  • 2
    It would also give you bad performance. Commented Aug 29, 2019 at 21:47
  • For storing files that aren't read/written often, that's a really creative idea! Commented Sep 19, 2022 at 18:51
  • RAID5 is able to: 1. Recreate lost data from one data member. 2. Detect corruption on one member (but not be able to tell you which one is corrupted). ... as such you could argue that it doesn't offer any meaningful error correction in the context that the author is talking about. Commented Mar 27, 2023 at 9:09
  • A big issue with RAID5 these days is that recovering a failed disk actually destroys the other "healthy" disks very often. It's especially prevalent with spinning disks over 2TB. A restore is basically guaranteed to destroy at least one other disk if each disk is 10TB spinning. I've actually seen 3 disks fail in sequence on a RAID5 restore, causing nearly a week of downtime. Source: my own experience (though I have seen some write-ups about it). Unsure how SSD is affected. Commented Apr 25, 2023 at 3:52
0

Similar to Btrfs's dup --data, ZFS allows you to store multiple data block copies with the zfs set copies command

zfs set copies=2 users/home

See Storing Multiple Copies of ZFS User Data

When a block is accessed, regardless of whether it is data or meta-data, its checksum is calculated and compared with the stored checksum value of what it "should" be. If the checksums match, the data are passed up the programming stack to the process that asked for it; if the values do not match, then ZFS can heal the data if the storage pool provides data redundancy (such as with internal mirroring), assuming that the copy of data is undamaged and with matching checksums. It is optionally possible to provide additional in-pool redundancy by specifying copies=2 (or copies=3 or more), which means that data will be stored twice (or three times) on the disk, effectively halving (or, for copies=3, reducing to one third) the storage capacity of the disk. Additionally some kinds of data used by ZFS to manage the pool are stored multiple times by default for safety, even with the default copies=1 setting.

https://en.wikipedia.org/wiki/ZFS#Data_integrity

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.