I have a strange problem on my Debian 10 desktop system when I am doing large IO operations like copy large files from/to external drivers or also on the internal disc.

I could pin this down to the following simple case:

sudo dd if=/dev/sdb of=/dev/null

When i issue this comand to just read all content of the drive /dev/sdb (e.g. a plugged USB drive, but could also /dev/sda the internal drive) then I can see in e.g. htop that the system aggressively swaps out large amounts of memory. And therefore the system gets completely unresponsive until the operation is completed and (after a while again) the system swaps my desktop applications back in.

To pin this down I used swapoff / swapon to clear the whole swap, resulting in ~5GB used memory:

During the operation the amount of io buffers (blue) increases continuously  .

...

After the command above the system looks like this:

I've tested several system parameters to reduce the swapiness and the writeback size, e.g. setting

/proc/sys/vm/swapiness = 1
/proc/sys/vm/dirty_ratio = 10

but this does not really change anything on this problem.

Interestingly if I just completely disable the swap then I could not see any performance problem or memory pressure. The kernel just uses the available memory and allows applications to allocate even more (by reducing the buffer sizes).

Has anyone similar problems and know a solution? I have this problem on both of my computers and no idea how to solve this.

I have a strange problem on my Debian 10 desktop system when I am doing large IO operations like copy large files from/to external drivers or also on the internal disc.

I could pin this down to the following simple case:

sudo dd if=/dev/sdb of=/dev/null

When I issue this command to just read all content of the drive /dev/sdb (e.g. a plugged USB drive, but could also /dev/sda the internal drive) then I can see in e.g. htop that the system aggressively swaps out large amounts of memory. And therefore the system gets completely unresponsive until the operation is completed and (after a while again) the system swaps my desktop applications back in.

To pin this down I used swapoff / swapon to clear the whole swap, resulting in ~5GB used memory:

During the operation the amount of io buffers (blue) increases continuously.

...

After the command above the system looks like this:

I've tested several system parameters to reduce the swappiness and the writeback size, e.g. setting

/proc/sys/vm/swapiness = 1
/proc/sys/vm/dirty_ratio = 10

but this does not really change anything on this problem.

Interestingly if I just completely disable the swap, then I could not see any performance problem or memory pressure. The kernel just uses the available memory and allows applications to allocate even more (by reducing the buffer sizes).

Has anyone similar problems and know a solution? I have this problem on both of my computers and no idea how to solve this.

(This question is a copy of https://stackoverflow.com/questions/58893593/linux-aggresivly-swaps-when-doing-large-io-operations )

I have a strange problem on my Debian 10 desktop system when I am doing large IO operations like copy large files from/to external drivers or also on the internal disc.

I could pin this down to the following simple case:

sudo dd if=/dev/sdb of=/dev/null

When i issue this comand to just read all content of the drive /dev/sdb (e.g. a plugged USB drive, but could also /dev/sda the internal drive) then I can see in e.g. htop that the system aggressively swaps out large amounts of memory. And therefore the system gets completely unresponsive until the operation is completed and (after a while again) the system swaps my desktop applications back in.

To pin this down I used swapoff / swapon to clear the whole swap, resulting in ~5GB used memory:

During the operation the amount of io buffers (blue) increases continuously .

...

After the command above the system looks like this:

I've tested several system parameters to reduce the swapiness and the writeback size, e.g. setting

/proc/sys/vm/swapiness = 1
/proc/sys/vm/dirty_ratio = 10

but this does not really change anything on this problem.

Interestingly if I just completely disable the swap then I could not see any performance problem or memory pressure. The kernel just uses the available memory and allows applications to allocate even more (by reducing the buffer sizes).

Has anyone similar problems and know a solution? I have this problem on both of my computers and no idea how to solve this.

I have a strange problem on my Debian 10 desktop system when I am doing large IO operations like copy large files from/to external drivers or also on the internal disc.

I could pin this down to the following simple case:

sudo dd if=/dev/sdb of=/dev/null

When i issue this comand to just read all content of the drive /dev/sdb (e.g. a plugged USB drive, but could also /dev/sda the internal drive) then I can see in e.g. htop that the system aggressively swaps out large amounts of memory. And therefore the system gets completely unresponsive until the operation is completed and (after a while again) the system swaps my desktop applications back in.

To pin this down I used swapoff / swapon to clear the whole swap, resulting in ~5GB used memory:

During the operation the amount of io buffers (blue) increases continuously .

...

After the command above the system looks like this:

I've tested several system parameters to reduce the swapiness and the writeback size, e.g. setting

/proc/sys/vm/swapiness = 1
/proc/sys/vm/dirty_ratio = 10

but this does not really change anything on this problem.

Interestingly if I just completely disable the swap then I could not see any performance problem or memory pressure. The kernel just uses the available memory and allows applications to allocate even more (by reducing the buffer sizes).

Has anyone similar problems and know a solution? I have this problem on both of my computers and no idea how to solve this.