Timeline for Why Linux does not clean up disk caches and swap automatically?
Current License: CC BY-SA 4.0
17 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Aug 4, 2019 at 15:28 | comment | added | sourcejedi |
@spectras if you take the example in this question, we may fill ram by reading at disk speeds. E.g. with a 100MB/s disk, 1GB of ram, and a difference between the high watermark and min of 1%, expressing that in terms of a time could give you 0.1 seconds. So I assume "ahead of time" refers to more than that timescale. Instead you're talking about the strategy of balancing reclaim of swappable memory v.s. file-backed memory. As in the question, the lore says "opportunistic swapping" happens "when the system is idle". I assume you at least disagree with that. kswapd runs at default priority.
|
|
| Aug 4, 2019 at 14:32 | comment | added | spectras | @sourcejedi I read it, and as someone put it correctly, you answer no then proceed to show that yes, it does. You just disagree on the definition of the word "opportunistic". I am not interested in spending time on debating whether the preliminary thresholds strategy fulfills your definition of the word “opportunistic”. | |
| Aug 4, 2019 at 13:40 | comment | added | sourcejedi | @spectras the answer does that as well, it just seemed better to start it with the published documentation because that's easier for people to read (and introduces necessary concepts). | |
| Aug 4, 2019 at 13:27 | comment | added | spectras | @sourcejedi sorry, but I have better things to do. Go read the source for accurate info. | |
| Aug 3, 2019 at 22:11 | comment | added | sourcejedi | @spectras fite me :-p | |
| Aug 2, 2019 at 23:12 | comment | added | spectras |
@sourcejedi look up the documentation of vm.swappiness setting. It's nothing complex actually, it's a threshold that triggers preventive caching before it's actually needed. For instance if it's 60, linux will start looking for pages to swap out when memory use goes above 40%. As for the kernel, just read the source yourself, relevant code is in mm/vmscan.c.
|
|
| Aug 2, 2019 at 13:51 | comment | added | sourcejedi |
@spectras if someone has any fragment of information about Linux copying or moving pages to swap "ahead of time", I would appreciate their citation. So far I've seen no reference for how this hypothetical behaviour is calculated, or configured, nor any references to kernel code or statements by kernel developers. I have a paragraph on it in this question and would welcome comments: atop shows swout (swapping) when I have gigabytes of free memory. Why?
|
|
| Aug 27, 2018 at 13:53 | comment | added | chingis | @spectras I don't want to disable swappiness completely, I need it. My servers get a CPU spikes because of swapping out because of low "free" cache | |
| Aug 27, 2018 at 13:41 | comment | added | spectras |
It does not. It optimizes memory use, usually ahead of time. A frequently-used cache is much more useful than a page that was really allocated, but it was 20 days ago and the process never used it since. The aggressiveness of that optimization is controlled by the vm.swappiness setting, as dolapevich said in his answer. You can disable it entirely by setting it to 0.
|
|
| Aug 27, 2018 at 11:48 | comment | added | chingis | @spectras I know that very well, it does not change the fact that Linux tries to swap out memory because of low "free" memorey | |
| Aug 26, 2018 at 3:53 | comment | added | spectras | You are misunderstanding memory reports from the system. Pages used as cache are free memory. It's only used as cache until some process wants it. Basically that's the OS using spare memory to cache stuff. Dropping cached content does no good (that memory was already available). Worse, if any of the dropped data is needed, the system will have to fetch it again. | |
| Jun 25, 2018 at 10:24 | vote | accept | chingis | ||
| Jun 22, 2018 at 12:18 | comment | added | sourcejedi |
Nice hack with docker! If you don't specifically want to run inside docker, you could also use a cgroup manually. systemd-run should be able to apply systemd.resource-control properties to any command (these are also implemented using cgroups). As a non-root user, you can use systemd-run --user
|
|
| Jun 22, 2018 at 12:12 | answer | added | sourcejedi | timeline score: 7 | |
| Jun 22, 2018 at 11:56 | history | edited | chingis | CC BY-SA 4.0 |
added 767 characters in body
|
| Jun 22, 2018 at 11:37 | answer | added | Dolapevich | timeline score: 2 | |
| Jun 22, 2018 at 10:17 | history | asked | chingis | CC BY-SA 4.0 |