3

I'm using ARM boxes running Linux 2.6 (I'm stuck with this kernel version unfortunately). The boxes boot off of an SD card, and use a secondary SD card for storage. The boxes have approximately 256MB of memory.

Both SD cards are encrypted with dm-crypt, but I don't believe that this has a major impact on the read/write performance, because the performance was pretty abysmal before I implemented disk encryption. I don't have exact numbers to back up this claim, but just as an example, writing 1GB to an SD card would take upwards of an hour, without dm-crypt in use.

Under normal usage, the read/write speeds are good enough. However, certain tasks such as building software from source tend to lock up on I/O operations, locking up for minutes at a time. Specifically, I believe that the write operations are the ones that block for a long time. I finally found out about the tool iostat, which provided some useful information.

While a program (example: configure script) is blocking, I can run watch iostat. Watching the Blk_wrtn field seems to confirm that the bottleneck causing lock-ups is the write operations. Specifically, during a blocking write operation, the dm-1 device (second SD card's unlocked partition) has N blocks written to it, while the mmcblk1 (second SD card) has M blocks written to it, where M < N. M increases as time passes, until it is eventually equal to N, at which point the write operation is complete and the program continues executing. As of writing this, the Blks_wrtn/s is approximately 10 for mmcblk1 and dm-1.

I conclude from this observation that blocks are written to the dm-1 pseudo-device immediately, and are then flushed to the physical disk; something about this flushing operation is taking longer than it should.

These are the options I have tried / considered to try to improve the SD card write performance:

  1. Upgrade to a newer kernel with better drivers / performance: I would love to have a newer kernel, but we are stuck with what the manufacturer provides. I am still trying to figure out if it would be possible to use a newer kernel.
  2. Use a different IO scheduler for the SD card: I tried switching between deadline, CFQ, and noop, with no noticeable difference in performance.
  3. Use higher-class SD cards: I'm actually not sure what class the SD cards provided by the manufacturer are, but I have tried switching to class 4 SD cards, with the same performance. I don't think the SD card itself is the bottleneck; it's possible that the readers themselves are the bottleneck, but I don't know how to check this.
  4. Use a different filesystem on the SD cards: I'm currently using Ext4. Would a different filesystem make a major difference here?

Is there anything I can do to improve the speed when writing to SD cards?

Improve this question
8
  • 2
    This isn't a direct answer to improving the write speed, but maybe you can cross compile on a machine with more horsepower and just copy the binaries over rather than building software on anemic little ARM boxen? Commented Aug 15, 2017 at 18:40
  • 1
    You're right that cross-compiling is much faster, and I use that when I can, but the poor write speeds affect things other than just compiling software. For example, I would like to use the boxes to read and write disk images to SD cards, but like I said writing a 1GB image to an SD card takes more than an hour, while it takes under a minute on an x86 PC. Commented Aug 15, 2017 at 18:42
  • Typically those kernels are a specific version "hacked" for the board. Forget about upgrades... Commented Aug 15, 2017 at 20:28
  • I was lucky enough to get the source for the kernel from the manufacturer, so I was thinking that it could be possible to essentially diff the manufacturer's version with the original Linux source, and 'apply the diff' (it would be much more complicated than that of course) to a newer kernel source. That would probably be quite a bit of work, though, so I think I am stuck with the kernel I have. Commented Aug 15, 2017 at 20:44
  • 1
    SD cards are slow, but minutes has to be a bug. Have you checked if there was a bug fix to the SD card driver after that kernel version was forked? Which version is it exactly, which manufacturer? Commented Aug 15, 2017 at 22:26

1 Answer 1

Reset to default
1

If you have enough RAM free, you can run /tmp from RAM.

It will give you a performance boost while compiling or with daemons that use temporary sockets in /tmp.

Another strategy is instructing syslog to not log at all (not the best of the ideas), or better yet, send logs to a remote server. Ditto for possible disk-based databases.

Be also aware of log files, depending on the syslog daemon and configurations, a line write in a couple of syslog files may trigger/force a fsync of that file.

See also if there are unnecessary daemons, or with that low RAM, it may pay of for instance, stopping a web server when it is not necessary.

About the cards, the prices have fallen down enough for using a Class 10 card.

Improve this answer
1
  • 1
    That's a good point about using RAM to prevent writes to the disk, but I don't think that it would resolve the fundamental issue, which is slow SD card write speeds. Consider the case of just writing an image to an SD card (dd if=someFile.img of=/dev/mmcblk1); nothing else is using the SD card, but the process still takes way too long. I should try a class 10 card, but I don't think that the card is the bottleneck, so I don't expect to see a major difference. Commented Aug 15, 2017 at 20:47

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.