dd backup fails with "File too large" error despite ample free space on FAT32 partition [duplicate]

Question

I am attempting to back up a drive, but alas I am met with the following issue:

trisquel@trisquel:~$ sudo dd if=/dev/sda of=/media/trisquel/Backup/ImageBackupOfSeptember202024/full-backup.image status=progress
4261118464 bytes (4.3 GB, 4.0 GiB) copied, 77 s, 55.3 MB/s
dd: writing to '/media/trisquel/Backup/ImageBackupOfSeptember202024/full-backup.image': File too large
8388608+0 records in
8388607+1 records out
4294967295 bytes (4.3 GB, 4.0 GiB) copied, 122.389 s, 35.1 MB/s
trisquel@trisquel:~$ cd /media/trisquel/Backup/ImageBackupOfSeptember202024/
trisquel@trisquel:/media/trisquel/Backup/ImageBackupOfSeptember202024$  df -Ph . | tail -1 | awk '{print $4}'
834G
trisquel@trisquel:/media/trisquel/Backup/ImageBackupOfSeptember202024$ 

As stated, the drive has 834G of free space (far above the 500 GB maximum of the drive to be backed up), and is formatted as FAT32

Is there anything I can do to help remedy this issue? Thank you so much for any help you can provide.

---

OS: Trisquel Live Installation Media (an Ubuntu-based live usb system)

Device: x230 ThinkPad

Figure 1: Screenshot from Gnome Disk Utility

Answer 1

dd fails because your FAT filesystem does not support large files.

The obvious solution is to use a different filesystem.

The alternative is to write multiple files.

For this, you can use split:

sudo dd bs=1M if=/dev/sdx | split -b 1G - split-backup.

And it will create files like split-backup.aa, split-backup.ab, etc. with 1GiB per file.

To restore, you can use:

cat split-backup.* | sudo dd bs=1M of=/dev/sdx

cat concatenates the split data back together.

---

There is a risk of restoring data in the wrong order, if the files were named incorrectly. This is mainly a concern when splitting manually or renaming files later, since split normally takes care of this for you. Once it reaches .yz it will continue with .zaaa so it stays alphabetical.

You can also use numeric suffixes -d and set a fixed suffix length -a, so for example split -a 3 -d would use 000, 001, 002 etc. as the suffix.

However in this configuration, split will abort after 999 (split: output file suffixes exhausted). So only use -a if you're 100% sure how many splits there are going to be, or use larger -a values just in case.

Answer 2

FAT32¹ only uses 32-bits for file sizes in directory entries, which means that any one file can be at most 2^32-1 bytes long (one byte short of 4 GiB). There is no way to get around this limitation in FAT32 if you want to use a single file (unless that 500 GiB disk is mostly zero bytes and you use a good compression tool, but even then probably not).

The ‘simple’ workaround would be to use the split command to split the data into multiple files as suggested in the answer by frostschutz.

The less simple but far more robust approach is to use a proper filesystem such as Ext4 or XFS, which will not have this issue, and will also be generally more resilient against all kinds of other possible failures.

However, using dd for this type of thing is undesirable for multiple reasons². It’s almost always better to use a tool designed for disk imaging use cases, such as fsarchiver, or if you truly do need the whole disk (for example, the system isn’t using UEFI to boot) or are dealing with a known bad drive ddrescue set up to write a sparse file (though that would again require using a proper filesystem and not FAT32).

---

1: This is all you could be using here. You wouldn’t be seeing this issue if it was ExFAT, and the volume couldn’t be that big if it were FAT16 or FAT12.

2: To list a few major ones, you can’t resume an interrupted copy, it will choke on bad sectors, and it almost certainly is storing far more data than you actually need to.