Why does df show different used values for identical data on two different partitions?

Question

I've just copied across some data from one raid partition to another. When I look at the output of df:

Filesystem      1K-blocks       Used  Available Use% Mounted on
/dev/md127     1921673480 1652707240  171280736  91% /home/chris/mount/raid18t
/dev/md3       2881491232 1652719488 1082329968  61% /home/chris/mount/raid3t

The Used values are not identical (1652707240 vs 1652719488). However, if I do a du -s * in each of the directories though - I get the same values/output...

Why is there a difference in the Used values?

Both filesystems are ext4.

Answer 1

The reason is, df reports the size on disk whereas du reports on actual file size. Even in the case of files being exactly the same, does not imply that the underlying file system blocks are guaranteed to be identical.

Consider a block size of 8k on Disk-1 and a block size of 4k on Disk-2.
File foo has an actual size of 11k.

Disk-1/foo will take 2 blocks, which uses 28k=16k.
Disk-2/foo will take 3 blocks, which uses 34k=12k.

EDIT
Just for fun, pick a directory with lots of small files in it. run ls -l and total the file sizes. This is the actual size of the file. On my flavor of Linux, the file size is in column 5. So I'd do something like this:
ls -l | awk '{ttl=ttl+$5} END {print "Total file size " ttl}'
This will total the file sizes in the directory.
Next, list the number of sectors on the disk, with ls -ls and total the sectors:
ls -ls | awk '{ttl=ttl+$1} END {print "Total sector count " ttl}'

On my system, I checked my boot directory:
ls -lsR /boot | awk '{if (($1+0) >0) tt_sec=tt_sec+$1; tt_byt=tt_byt+$6} END {print "Sectors " tt_sec " Bytes " tt_byt}'
Sectors 151,352 Bytes 153,894,246
du -xs /boot
151360
df /boot
1-K blocks: 499656
Used: 151756

Notice how all 3 numbers are different? ls shows the size in bytes, by default.
df unless otherwise configured, reports 1K blocks
My disk is configured with 512 byte sectors (I got that from fdisk)
If I add hidden files & Directories into my ls command:
ls -lRsa (…) Sectors 151,504 Bytes 154,045,696
Closer, but still off.
Let me show you, now, what is happening:
-copied for convenience-
ls -lsRa /boot | awk '{if (($1+0) >0) tt_sec=tt_sec+$1; tt_byt=tt_byt+$6} END {print "Sectors " tt_sec " Bytes " tt_byt}'
Sectors 151,504 Bytes 154,045,696
df /boot
1-K blocks: 499656
Used: 151756
du -xs /boot
151360

echo "Hi" > junk ls -ls junk
4 -rw-r--r-- ScottieH user 3 junk
Notice that this 3-byte file is using 4 sectors!

-recompute sizes- ls -lsRa /boot | awk '{if (($1+0) >0) tt_sec=tt_sec+$1; tt_byt=tt_byt+$6} END {print "Sectors " tt_sec " Bytes " tt_byt}'
Sectors 151,508 Bytes 154,045,972 <-How much did this go up?
du -xs /boot
151364
df /boot
1-K blocks: 499656
Used: 151760

** Do you see how these numbers will NEVER add up?**