0

I have noticed that when I mount my external HDD NTFS partition with the sync option, the write performance is terrible (1MB/s max). But when I use async the system finishes the operation very quickly (yes I know it is filled in the write cache), but according to htop I can see over 100MB/s writes to the external HDD. Why is this happening? Isn't sync immediately applying writes instead of caching? Why would it perform so poorly (x100 less than the max speed of the drive)

I am Running Debian unstable (equivalent to 13 - trixie)

1 Answer 1

2

Write caching has multiple purposes:

  • It allows the application to continue once the kernel has accepted the write but before the disk has accepted it. This can greatly reduce application latency and make it faster.
  • It allows the kernel to reorder and coalesce writes to consecutive blocks that were not issued in order, greatly increasing actual speed of writing to the disk.
  • If your application is doing writes of smaller than a block size, forcing a sync rather than caching the write might force the kernel to write to the same block multiple times rather than waiting for the block to fill up and writing it once. (Note: by default STDIO does block buffering application side without the kernel knowing.)
  • Write caching allows the kernel to delay disk writes in favor of disk reads, as an application absolutely can not continue if it doesn't have data to work on, but it generally doesn't have to know that the data hasn't been written yet.
  • It allows the kernel to delay disk writes and do batches of disk writes all at once, rather than have to mix small writes with reads (which are likely not in the same disk location) which can cause thrashing in a mechanical disk.
  • Some filesystems don't allocate the location of written blocks until it writes them to disk. For these filesystems, delaying disk writes with cache allows merging what might have been small writes of multiple blocks to scattered disk regions into one big write to consecutive blocks or even merging separate small writes to multiple blocks into one big write to one block. (This is one big advantage of log based filesystems -- most writes are batched consecutive writes to the log.)
1
  • I think I will reduce the write cache size instead of forcing sync everywhere. Nothing annoys me more than transferring a large file to a usb drive only to find unmounting take minutes with no progress bar or anything. Thank you for the info Commented Jun 20 at 5:52

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.