10

I am looking for a way to compress swap on disk. I am not looking for wider discussion of alternative solutions. See discussion at the end.

I have tried...

Using compressed zfs zvol for swap is NOT WORKING. The setup of it works, swapon works, swapping does happen somewhat, so i guess one could argue it's technically a working solution, but exactly how "working" is your working solution, if it's slower than floppy disk access, and causes your system to completely freeze forever 10 times out of 10? Tried it several times -- soon as system enters memory pressure conditions, everything just freezes. I tried to use it indirectly as well, using losetup, and even tried using zfs zvol as a backing device for zram. No difference, always same results -- incredibly slow write/read rates, and system inevitably dies under pressure.

BTRFS. Only supports uncompressed swapfiles. Apparently, only supports uncompressed loop images as well, because i tried dd-ing an empty file, formatting it with regular ext2, compressing it, mounting as a loop device, and creating a swapfile inside of it. Didn't work, even when i mounted btrfs with forced compression enabled -- compsize showed the ext2 image compression ratio of exactly 1.00 .

Zswap -- it's just a buffer between ram and regular disk swap. The regular disk swap keeps on being the regular disk swap, zswap uncompresses pages before writing them on there.

Zram -- has a backing device option since it's inception as compcache, and one would think, is a perfect candidate to have had compressed disk swap for years. No such luck. While you can do writeback of compressed in-ram pages to disk at will, the pages get decompressed before they're written. Unlike zswap, doesn't write same- and zero-filled pages though, which both saves i\o, slightly improves throughput, and warrants the use of loop-mounted sparse files as backing_dev. So far, this is the best option I found for swap optimization on low-end devices, despite it still lacking disk compression.

Any ideas what else I can try? Maybe there's some compressed block device layer, that I don't know of, that can compress anything written to it, no filesystem required? Maybe there's some compressed overlay I could make use of? Not done in FUSE though, as FUSE itself is a subject to swapping, unless you know a way to prevent it from being swapped out.

Since i don't see this being explored much, you're welcome to suggest any madness you like. Please, let's throw stuff at the wall and see what sticks.

For experts -- if any of you have read, or even written, any part of linux sourse code that relates to this problem, please describe in as much detail as possible, why do you think this hasn't been implemented yet, and how do you think it could be implemented, if you have any idea. And obviously, please do implement that if you can, that'll be awesome.

Discussion

Before you mark it as a duplicate -- I'm aware there have been a few questions like that around stackexchange, but none i saw had a working answer, and few had any further feedback. So I'll attempt to describe details, sort of aggregate everything, here, in hopes that someone smarter than me can figure this out. I'm not a programmer, just a user and a script kiddie, so that should be a pretty low bar to jump over.

just buy more ram, it's cheap

get an ssd

swap is bad

compression is slow anyway, why bother

If all you have to say, are any of the above quotes -- go away. Because the argument is optimization. However cheap RAM is these days, it's not free. Swap is always needed, the fact that it's good for the system to have it, has been established for years now. And compression is nothing, even "heavy" algorithms perform stupidly fast on any processors made in the last decade. And lastly, sure, compression might actually become a bottleneck if you're using an ssd, but not everyone prioritizes speed over disk space usage, and hdd drives, which DO benefit grossly from disk compression, are still too popular and plentiful to dismiss.

Improve this question
9
  • 5
    One significant hurdle is that the kernel skips the file system when dealing with swap, so you’d need block-level compression, not file-level. Commented Mar 20, 2023 at 8:51
  • 2
    I was about to read that with great interest, but after the first two paragraphs, I got the feeling that I'm being antagonized though not having said anything, and if I then wrote an answer that contradicts the author's perception in any way, it would fall on deaf ears. Deciding to just leave this question alone altogether :( Commented Mar 20, 2023 at 9:43
  • I recognise your pain asking about swap which is somehow a religious question in the Linux world. FYI swap is not required and does not always make your system better. But it does in many cases. Commented Mar 20, 2023 at 19:44
  • 2
    Thank you for editing, yes I've seen too many times that a question gets no useful reply whatsoever, so I guess I went a bit too hard on specifying what I need. Also I agree, swap isn't absolutely necessary, I just meant that in a majority of cases it's a good idea to have one, especially for home desktop use, which is my use case Commented Mar 23, 2023 at 2:13
  • 1
    @MarcusMüller No. That is a different world. Knowing well Germany, honestly I understand you, but please, please, believe me, he has this time right. He is not antagonizing, he is trying to evade the unthinkable flood of bad answers... Commented Apr 11 at 15:00

4 Answers 4

Reset to default
9

I don't have a concrete answer for you: something you might explore is LVM.

LVM is primarily an alternative form partitioning. However technically LVM's physical volumes can be any block device. It provides logical block devices ultimately backed by physical ones.

Since LVM logical volumes are block devices they can usually be used for swap.

LVM has a feature called VDO which provides compression:

The Virtual Data Optimizer (VDO) feature provides inline block-level deduplication, compression, and thin provisioning for storage. You can manage VDO as a type of Logical Volume Manager (LVM) Logical Volumes (LVs), similar to LVM thin-provisioned volumes.

More information on VDO has now been added to a new answer

Trouble you might run into

Any form of compression requires some memory. My main concern with any solution that was not designed to work with swap is that it may dynamically request memory from the kernel to compress pages. Since it would be compressing because the kernel needed to free up RAM, requesting RAM would be prone to cause failure.

It is of course possible that drivers are written to be aware of this and primitively request all the RAM they might need. The point is this might be a problem if the drivers were not written with swap in mind.

Improve this answer
4
  • Okay now we're moving somewhere. This worked better than zfs zvol. I did indeed run into a system deadlock immediately after running swapon directly on it, but this thing works surprisingly well, if I plug it into zram backing_dev. The writing speed is about 10 times slower though, but it's at least somewhat usable. Curiously, abstracting it with losetup and THEN plugging as a backing_dev seems to improve writing speed almost twice Commented Mar 23, 2023 at 4:07
  • Weird, I tried the reverse, making a vdo volume using zram, plugging regular swap as backing_dev and the writes to vdo volume were faster, but still incredibly slow, at least 10 times slower than regular zram. I don't get it, it's ram-to-ram, should be blazing fast despite double compression. The writes to backing_dev ( aka reads from zram ) were also noticeably slower, despite the data staying compressed due to vdo. Is it just the vdo overhead? In any case, this is a slow, but genuinely working solution, I'll wait in case anyone has a better idea, and will accept this if not. Thank you =) Commented Mar 23, 2023 at 16:00
  • While I can see why you would not want to call a compression process that takes up more ram in the memory pressure situation. However, this does not explain why zswap calls upon a process to decompress the page before writing it to disk. Why not just write the compressed page to disk? Commented Jun 12, 2024 at 14:23
  • @thoth if the arch wiki is to be believed, zswap is not doing any disk IO and I immagine that adding such functionality would make it's code considerably more complex. Instead, it's acting as a transparent cache. Transparant caches are generally trivial to write and easy to test. They often have few hooks in the code. If it tried to write compressed data instead of uncompressed data, this would force it to take ownership of decompressing as pages are reloaded from swap and that may be complex. Commented Jun 12, 2024 at 22:46
5

VDO has been mentioned here, but with no specifics. So I went ahead and fully tested the case and here I explain the experience and how to set it up.

VDO is an LVM feature providing compression and deduplication. It is supported since Linux kernel 6.9 (for older versions there was a KVDO out-of-tree module). Depending on your distro you may also need to install vdo utils (if it's necessary you'll get an error from lvcreate).

Case: I've recently been travelling with my old laptop which is both low on RAM and has very slow HDD ST1000LM024 with a passmark rating of 658. Running Firefox and Qutebrowser simultaneously and opening an HD video in one of them quickly results in /proc/pressure/io peaking to ≈90% most of the time and freezes. Launching browsers with ionice -c3 kind of helped, but constant swapping was another source of lags.

Solution: after having set up swap over VDO compressed device the system responsiveness improved considerably. Unfortunately Idk offhand how to measure it more scientifically, but based on my experience freezes upon switching virtual desktops were almost gone.

  1. pvcreate /dev/sdXn: mark device for LVM to use

  2. vgcreate vg /dev/sdXn: create an LVM pool named vg

  3. lvcreate --type vdo -L 5G -V 3G --compression y --deduplication n vg/lv_vdo: create a VDO volume of size 5G that will be seen as a 3G block device. Considerable size here will be taken for VDO internal purposes, so wasn't it for -V, I was getting 2G-sized block device on these sizes. You might want a different size or even drop -V completely. I just kind of presumed that I will get compression of at least 33%, so replacing 2G to 3G should be more-or-less safe, but I may be wrong.

    We also disable deduplication here, as it has no use for our case (moreover, deduplication typically takes additional memory which we just don't have).

    Resulting block device should appear at /dev/vg/lvol0.

  4. mkswap /dev/vg/lvol0 && swapon /dev/vg/lvol0: create swap partition over the compressed device and use it

  5. (optional) get swap UUID with lsblk -fQ 'FSTYPE=="swap"', and then write a line in /etc/fstab so swap is getting used on boot:

    UUID=<INSERT_SWAP_UUID>  none  swap  defaults  0  0

    On Archlinux at least I didn't need any other settings. In particular, since swap is loaded after / is mounted, nothing has to be added to initramfs.

Misc

  • Current usage may be monitored with command vdostats --human-readable. If by any chance the Use% column would get to 100%, writing to the device would result in ENOSPC. It should be noted though that depending on the pool and volume sizes, the Size column may get increased once Use% is ≈85-87%, so the number may be misleading.

    If you use large enough partitions you shouldn't need to monitor that though.

  • Upon creation, noticeable size is taken by VDO for internal use, so make sure to create SWAP partition of a larger size than what you'd want to use for swap.

Improve this answer
1

simplistic version:

  • put a VM file on a compressed (ZFS-ish) volume, recent advances in compression algos should help out a bit, virtualisation will isolate the abstractions like cpu, memory and disk format

incredible cumbersome hard way:

  • use zswap and put the swapfile inside a (v)ram backed bcache NVME raid array

when you do this, like reported in previous posts please run your workload inside a memory & swap constrained container because otherwise your kernel will deadlock because the compression algo for swap needs (more) memory to (de)compress and gets an OOM ( yes, that is an inception exception with a 0% recovery rate )

simplistic recent example seems https://loskoderos.com/2023/05/25/hybrid-storage-with-zfs-bcache-luks/

my own highly experimental and tailormade examples for my system are https://github.com/twobombs/thereminq/blob/master/systemscripts/mdadmswap.sh

do not run those scripts without knowing what they do because they will kill data or just not work ( and then kill your data )

I've spent hours, day, weeks on this but because of CPU compression constrainsts I decided to wait for this to be really usefull. Compression algos are getting better so some day this will work to be practical

Improve this answer
0

You can have a loop block device on anything, then you can swap on that loop device.

In details, this very simple solution:

  1. I create the swap file on a btrfs filesystem. Btrfs has transparent compressed file support.
  2. There is no way to just swapon it; because btrfs has also CoW (copy-on-write) support enabled by default, which is required for transparent compression. But, to have swapfile in btrfs, turned off CoW is needed (Cow can be turned on or off by the chattr -C / +C command).
  3. However, our schweizer knife helps this time: the loopback block devices. So losetup /dev/loop0 /swap and we have what we wanted.
  4. After that, I could freely swapon /dev/loop0.

At the moment I do not know, maybe it might cause deadlocks or similar, but I think it is very unlikely. It might have some speed degradation, because swapped out pages have to walk twice in the block cache. Time will say that.

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.