I have a LUKS (Linux Unified Key Setup) partition that contains the rootfs. I wish to replace it with an unencrypted ext4 partition containing the same rootfs. Having searched far and wide I've seen no tools to aid in this process. Did I miss something?
If not what is the solution? Use parted to delete the LUKS partition, create an ext4 partition and copy the rootfs into it?
At the most simplified level, there is a utility called cryptsetup-reencrypt which allows for this operation. It explicitly calls out in it's man page:
WARNING: The cryptsetup-reencrypt program is not resistant to
hardware or kernel failures during reencryption (you can lose your
data in this case).
ALWAYS BE SURE YOU HAVE RELIABLE BACKUP BEFORE USING THIS TOOL.
The reencryption can be temporarily suspended (by TERM signal or by
using ctrl+c) but you need to retain temporary files named
LUKS-<uuid>.[log|org|new]. LUKS device is unavailable until
reencryption is finished though.
As per the "Examples" section of the man page:
Remove LUKS encryption completely
cryptsetup-reencrypt /dev/sdb1 --decrypt
This will likely need to be done from a "Live" environment (meaning a fully running ephemeral Linux enviornment like the one provided by Fedora Live or Ubuntu Live.
As an alternative to this workflow, you can also take the more explicit route of backing up the data, removing the partition, and recreating it. Arch Linux has excellent documentation on this workflow as well as additional troubleshooting steps:
https://wiki.archlinux.org/index.php/Removing_System_Encryption
Again, it should be stressed that this is not a trivial operation and backups of all data are paramount in the event of catastrophic failure.
As @BrianRedbeard says, there's a tool for this (cryptsetup-reencrypt). Read the related documentation carefully. This is a dangerous operation. (In-place data conversion always is. One wrong step and it goes up in smoke.)
Just for kicks, the manual process:
Actually it's quite simple (and doesn't work this way in the other direction!). You can actually just copy it over like this:
pv < /dev/mapper/cryptsdx1 > /dev/sdx1
But since we're reading and writing to the same physical device, it might be faster to do with dd and a large blocksize (provided you have plenty of RAM). On a HDD, this is probably faster since it does fewer seeks, and seeks are slow.
dd status=progress bs=1G iflag=fullblock if=/dev/mapper/cryptsdx1 of=/dev/sdx1
But first, you should backup your LUKS header, since the LUKS header is the first thing you'll overwrite in this process, and you couldn't resume even if you wanted to.
cryptsetup luksHeaderBackup /dev/sdx1 --header-backup-file myluksheader.backup
And make sure to keep that backup safe, not just in RAM of a LiveCD environment.
The LUKS device basically is like this:
| LUKS HEADER | ENCRYPTED DATA |
and after the pv / dd, it should look like this:
| DECRYPTED DATA | FREE SPACE |
So the LUKS header is at the start of the device, and it's about 2MiB in size (used to be 1MiB plus a few sectors). And since we're getting rid of it, there'll be free space at the end of the device, and you can grow the filesystem by 2MiB. Wheee!
The data itself is accordingly to be found at an offset (of usually 2MiB, check with cryptsetup luksDump). This offset is important here - it means the data not only has to be decrypted, but also shifted - the start of the decrypted will appear where originally the LUKS header was stored.
It also means that throughout the process, there will be an overlapping region that exists in both decrypted, and encrypted forms. This is useful in case the machine crashes and you have to find the resume-point.
So, in what to do in case of crash?
First you have to re-create the crypt device using the backup:
cryptsetup --header myluksheader.backup luksOpen /dev/sdx1 cryptsdx1
Then find the overlap point:
skip=0 # or skip=X if you're sure at least X bytes were copied
step=$((1024*1024)) # 1MiB, use 512KiB for old 1MiB headers (offset / 2)
while ! cmp --bytes=$step /dev/sdx1 /dev/mapper/cryptsdx1 $skip $skip
do
skip=$(($skip+$step))
done
cmp --bytes=$step /dev/sdx1 /dev/mapper/cryptsdx1 $skip $skip && echo $skip || echo fail
Finally resume:
dd status=progress bs=1G iflag=fullblock,skip_bytes oflag=seek_bytes \
skip=$skip seek=$skip if=/dev/mapper/cryptsdx1 of=/dev/sdx1
Before using either method on your real filesystem, it's worthwhile to do a practice run on an unrelated partition (or even just a loop device) just to make sure it still works.
For example, I haven't taken the new LUKS2 format into account here, which in its basic form works the same way (4MiB header) but it supports strange things like multiple data segments and whatnot. So if you are using such features, it's a bit more complicated (and I don't think cryptsetup-reencrypt covers that yet either).
