File backed, locked shared memory and disk interaction

Question

Varnish, a HTTP accelerator, uses a ~80MB file backed SHM log that is mlock()ed into memory. The Varnish docs recommend to store the file on tmpfs to avoid unnecessary disk access. However if the entire file is locked into memory, does the Linux kernel still write to the backing file?

I tried to monitor this using inotify and fatrace, however since this interaction presumably happens all inside the kernel, no file activity was visible to these tools. There is clearly some kind of update happening either to the file or the filesystem, as monitoring the backing file with ls showed the file time changing, and sha1sum showed the contents were changing, but does this actually involve disk access or is it all happening in memory?

Basically I'm trying to avoid having to do the tmpfs workaround, as using SHM to back SHM seems like an ugly workaround for a problem that might not even exist.

Answer 1

Varnish appears to use a plain memory-mapped file for its shared memory (instead of, e.g., POSIX shm_open). From the source:

loghead = mmap(NULL, heritage.vsl_size,
    PROT_READ|PROT_WRITE,
    MAP_HASSEMAPHORE | MAP_NOSYNC | MAP_SHARED,
    heritage.vsl_fd, 0);

On BSD, MAP_NOSYNC requests that the kernel not write the shared data to disk unless forced (e.g., to free up memory). When it's mlocked as well, that should almost never happen. Unfortunately, Linux does not support MAP_NOSYNC.

So Linux will wind up routinely writing dirtied (changed) pages from the cache to disk. Putting the cache on a tmpfs will avoid that. So too would Varnish using POSIX or SysV shared memory (actually, POSIX shared memory is implemented on Linux with a tmpfs mounted at /dev/shm, so using the tmpfs should be fine).