53
votes
Why does piping `tar` into `dd` not stop until the disk is full?
It's because you're doing it wrong.
You're using bs=1M but reading from stdin, pipe, will have smaller reads. In fact, according to dd, you didn't get a single full read.
And then you have conv=sync ...
- 52.1k
48
votes
Accepted
Why does my dd Full Disk Copy Keep failing at 8 GB?
The error is, dd: error reading '/dev/sda': Input/output error, which tells you that the problem is reading the source disk and not writing to the destination. You can replace the destination disk as ...
- 128k
47
votes
Benchmark ssd on linux: How to measure the same things as crystaldiskmark does in windows
I'd say fio would have no trouble producing those workloads. Note that despite its name CrystalDiskMark is actually a benchmark of a filesysystem on a particular disk - it can't do I/O raw to the disk ...
- 3,884
41
votes
Why can't I copy my DVD with dd?
People mention that opening the DVD with VLC (which displays the DVD menu) magically makes the data accessible to dd, but nobody has yet explained why that is and how VLC accomplishes this feat.
I ...
- 515
40
votes
Accepted
Splitting a small file into 512 byte segments changes it, but splitting it in 1k segments doesn't
The order find processes the files is not deterministic. It may be just the same order as the underlying system call gives, which probably depends on the underlying filesystem structure and can be ...
- 148k
32
votes
Force dd not to cache or not to read from cache
Based on @sendmoreinfo's answer:
dd if=/dev/device iflag=direct bs=1M
It does not affect read-cache.
- 37.5k
32
votes
Accepted
How do I find the first non-zero byte on a block device, with an optional offset?
You can do this using cmp, comparing to /dev/zero:
cmp /path/to/block-device /dev/zero
cmp will give you the offset of the first non-zero byte.
If you want to skip bytes, you can use GNU cmp’s -i ...
- 481k
31
votes
Accepted
compressing and decompressing dd image - zstd instead of gzip
Using dd like that (without any options) will make your life miserable. Just cut it out entirely. Or at the very least increase its block size and tell it not to object to short reads.
Without dd, ...
- 128k
31
votes
Got "No space left on device" when cloning 1TB disk to 1.2TB disk using dd
Compare this answer (note bs=1K is used there):
dd is an old, cranky program initially intended to operate on tape devices. When you tell it to read one block of 1kB, it attempts to read one block. ...
- 24.4k
31
votes
Accepted
Is "dd" a reliable tool to make an SSD image?
TLDR: Use ddrescue
It supports resume/continue capabilities, has automatic logs, and tons of other options. More at the ddrescue home page.
Example syntax:
ddrescue /dev/sde yourimagename.image sde....
- 2,502
30
votes
Accepted
Is dd if=/dev/urandom of=/dev/mem safe?
Don't try this at home! It can crash your system, and if you're really unlucky it could damage a peripheral or make your computer unbootable.
Actually, on most platforms, it just fails with an error, ...
30
votes
Accepted
How can I verify that my hard disk has been zeroed / wiped?
Has the disk been zeroed?
Yes. The output of your dd command shows that it has written 5000981077504 bytes. Your cmp command says that it's reached EOF (end of file) after 5000981077504 bytes, which ...
- 3,175
29
votes
Benchmark ssd on linux: How to measure the same things as crystaldiskmark does in windows
Edit 2025: There is now an open source software that imitates crystaldiskmark called kdiskmark which can be used. It functions similarly to my script but has a GUI. Do note that since fio is not ...
- 2,562
29
votes
Accepted
dd command oflag direct and sync flags
We can probably rule out conv=sync to start with. It does something rather different, which I expect you do not want :-).
pad every input block with NULs to ibs-size; when used with
block or ...
- 53.5k
27
votes
Why does `dd` use `=` in its parameters?
POSIX mentions the following in its rationale section for dd:
Certainly, many of the operands could have been designed to use the Utility Syntax Guidelines, which would have resulted in the classic ...
- 481k
26
votes
Benchmark ssd on linux: How to measure the same things as crystaldiskmark does in windows
Try KDiskMark, very similar to CrystalDiskmark.
The application is written in C++ with Qt and doesn't have any KDE dependencies.
Usage
To start, after select a folder (and optionally other params), ...
- 1,646
25
votes
Can I safely utilize `dd` to backup my drive to a singular image file?
it is indeed a bit redundant, because this has been said in answers to other questions. However, I couldn't find one that stated this explicitly.
Yes, you can.
It's not a good method of making a whole-...
- 51.2k
24
votes
Accepted
What is the maximum value for the bs argument of dd?
The POSIX specifications for dd don’t specify a maximum explicitly, but there are some limits:
the datatype used to store the value given can be expected to be size_t, since that’s the type of the ...
- 481k
23
votes
Accepted
Will dd if=/dev/zero of=/dev/sda wipe out a pre-existing partition table?
Will dd if=/dev/zero of=/dev/sda wipe out a pre-existing partition table?
Yes, the partition table is in the first part of the drive, so writing over it will destroy it. That dd will write over the ...
- 148k
23
votes
Got "No space left on device" when cloning 1TB disk to 1.2TB disk using dd
As others have already identified, the problem is that you're using dd. Rule #1 of using dd is you do not use dd. Rule #2 of using dd is you do not use dd. Rule #3 of using dd (for experts only) is ...
21
votes
How can I copy arbitrary part of binary file with reasonable speed?
GNU dd supports count_bytes, seek_bytes, skip_bytes flags. This allows you to use a performant blocksize choice with arbitrary offsets and sizes.
GNU dd since version 9.1 does this by default if you ...
- 52.1k
20
votes
Accepted
Why does writing random data using dd result in disk partitions?
Several possibilities:
Linux supports a lot of different partition table types, some of which use very few magic bytes, and then it's easy to mis-identify random data (*) [so it's possible to randomly ...
- 52.1k
20
votes
Accepted
Why does `dd` have `swab` functionality
Because that was its original purpose. In the words of Douglas McIlroy, who was the head of the team that created Unix, in A Research UNIX Reader: Annotated Excerpts from the Programmer’s Manual, 1971-...
19
votes
Accepted
Different size of file from /dev/zero and /dev/urandom
With larger blocksize, there is a risk of getting incomplete reads. This also happens a lot when reading from a pipe, rather than a block device.
If you expect to receive a certain size (count*bs) ...
- 52.1k
19
votes
Accepted
`dd` does not create bootable Ubuntu iso
You've got a file as /dev/sda not a device, so when you write to /dev/sda you're overwriting the file. With your NVMe disk this explains why writing speed is so high.
Remove the file /dev/sda, unplug ...
- 128k
19
votes
Accepted
Accidentally used dd command
dd with no arguments reads from its standard input and writes to its standard output; you haven’t done anything to your computer.
- 481k
18
votes
Why does writing random data using dd result in disk partitions?
As seen here, the MBR (Master Boot Record) is relatively simple; https://en.wikipedia.org/wiki/Master_boot_record.
When you use /dev/urandom you can always create something that looks like a ...
- 280
17
votes
How to pad a file to a desired size?
Simple answer, courtesy of @frostschutz, posted as an answer
truncate -s 16M thefile
- 271
17
votes
Accepted
How do I interpret the output of dd when the option status=progress is chosen?
The first number (1207841280) is the number of bytes copied so far. The second number is this interpreted as the SI unit Gigabytes (1 GB is 10003 bytes). The third number is this interpreted as the ...
- 356k
17
votes
Accepted
How do I create binary file with certain value?
Probably easiest to use tr to convert the zeroes from /dev/zero to whatever you want, and then cut to length using dd or head or such.
This would write 18520 bytes with all bits ones, so value 0xff or ...
- 148k
Only top scored, non community-wiki answers of a minimum length are eligible