What does xxd -r -p do?

Question

So I am trying to calculate a SHA1 digest (and base64 encode it) for an attachment I received so it matches their (base64 encoded) digest. Using ChatGPT, I was able to use the command below to succeed. But I don't understand the command xxd -r -p or why it's necessary.

sha1sum foo | awk '{print $1}' | xxd -r -p | base64

There's two things I don't understand about it:

1. Why is the option -p needed? I understand xxd -r makes a binary dump of a hexdump, but if I leave off the -p it prints nothing while the man page says it should print to stdout and that -p is just for a particular output style.
2. What is the purpose of even passing the SHA1 digest through xxd? My understanding is xxd essentially converts data between hex and binary. But when I run the above command without xxd -r -p, it gives a different, and unmatched, output. I do know what hex style/representation is (that's the format of a SHA1 digest), but perhaps I don't know what "binary" means in this context?

Answer 1

First, look at the input to xxd -r p to find out what we're dealing with. Here's an example:

echo boo | sha1sum | awk '{print $1}'
6132b58967cf1ebc05062492c17145e5ee9f82a8

This line generates the SHA1 hash of the four characters, boo and its trailing newline.

Now let's investigate the xxd command a little. The man page for xxd (see man xxd) explains that the xxd -r command reverses a normal hexdump generated by xxd. So we can construct a pipeline like this:

echo boo | xxd | xxd -r
boo

Now contrast these two outputs:

echo boo | xxd
00000000: 626f 6f0a                                boo.

echo boo | xxd -p
626f6f0a

To reverse the second output into binary we need not only -r but also -p:

echo boo | xxd -p | xxd -r -p
boo

This structure (hex bytes without spaces or other formatting, generated by xxd -p) happens to match the output from sha1sum, which is why that can be converted to binary using this approach. The resulting binary is then (finally!) passed through base64, converting it to Base 64:

echo boo | sha1sum | awk '{print $1}' | xxd -r -p | base64
YTK1iWfPHrwFBiSSwXFF5e6fgqg=