How to create sha1 checksums of files inside a tar archive without using much disk space

Question

I would like to get the sha1 checksums of all files inside a simple tar archive as a list, or a new file

Without using the disk space to unpack the big tar file. Something with piping and calculating the sha1 on the fly, directing the output to /dev/null

I searched google a lot and did some experiments with pipes but could not get very far.

Would really save me a lot of time checking backups.

Answer 1

Too easy :

tar xvJf myArchive.tar.xz --to-command=sha1sum

The result is like this :

z/
z/DOCUMENTATION
3c4d9df9bcbd1fb756b1aaba8dd6a2db788a8659 *-
z/getnameenv.sh
1b7f1ef4bbb229e4dc5d280c3c9835d9d061726a *-

Or create "tarsha1.sh" with :

#!/bin/bash

sha1=`sha1sum`
echo -n $sha1 | sed 's/ .*$//'
echo " $TAR_FILENAME"

Then use it this way :

tar xJf myArchive.tar.xz --to-command=./tarsha1.sh

The result is like this :

3c4d9df9bcbd1fb756b1aaba8dd6a2db788a8659 z/DOCUMENTATION
1b7f1ef4bbb229e4dc5d280c3c9835d9d061726a z/getnameenv.sh

Answer 2

This is an incremental improvement on Vouze's answer to make the output of the script look exactly like the normal output of GNU's *sum.

The following version of tarhash.sh handles the situation where the filename contains a newline or backslash in the same way that the current version of md5sum/sha1sum/... does, in case that turns out to be necessary. It also preserves the two spaces between the sum and the filename (or the single space and *, if you change the command from sha1sum to sha1sum --binary).

As long as the filenames are sorted in the same way (or if you sort both sides first), the output of this script can be compared to an original sha1sum output using your favorite diff tool.

tarhash.sh:

#!/bin/bash
(echo -n "`sha1sum`" | sed 's/-$//' && echo -n "$TAR_FILENAME") |
sed -z 's/\\/\\\\/g; s/\n/\\n/g; /\\/ s/^/\\/; s/$/\n/;'

Usage:

# Note: In this tar invocation, make sure not to specify -v
# (The output comes out wrong if it is included).
tar xJf myArchive.tar.xz --to-command=./tarhash.sh > sha1sums.txt