105

At my company, we download a local development database snapshot as a db.dump.tar.gz file. The compression makes sense, but the tarball only contains a single file (db.dump).

Is there any point to archiving a single file, or is .tar.gz just such a common idiom? Why not just .gz?

Improve this question
5
  • 5
    It is just a matter of convention in my opinion. When people see a file with gz extension, their default thinking is, using tar -zxvf. But for those who look at the file name and see it doesn't have .tgz as extension, it is perfectly fine to gzip the db dump file. Since I don't know the compression algorithms in detail, I am not sure if tar makes any compression on sparse files like db dump, but for plain text files, direct gzip of the file has a very tiny size advantage over taring first and gzip ing the file Commented Apr 20, 2016 at 12:59
  • 3
    All tarring a single file will do is add a few metadata blocks to the start and end of the file. The actual file data passes through tar to the compressor untouched. So for a large file the size difference between plain compression and taring will be negligable. Commented Apr 20, 2016 at 14:29
  • In the past when trying various compression methods I found .tar.gz to be superior to most other common methods. I recall it was superior to just .tar but cannot remember if it was better than just .gz. Ironically Window's .cab format was the best of the methods I tried, which was very unexpected. Commented Apr 25, 2016 at 3:14
  • @Pharap tar is not a compression algorithm, it's an archiving format Commented Apr 25, 2016 at 16:37
  • 1
    @gardenhead Well that would explain why it didn't work very well. Commented Apr 26, 2016 at 4:18

8 Answers 8

Reset to default
166

Advantages of using .tar.gz instead of .gz are that

  • tar stores more meta-data (UNIX permissions etc.) than gzip.
  • the setup can more easily be expanded to store multiple files
  • .tar.gz files are very common, only-gzipped files may puzzle some users. (cf. MelBurslans comment)

The overhead of using tar is also very small.

If not really needed, I still do not recommend to tar a single file. There are many useful tools which can access compressed single files directly (such as zcat, zgrep etc. - also existing for bzip2 and xz).

Improve this answer
5
  • 38
    I didn't consider the meta-data aspect. Very good point Commented Apr 20, 2016 at 13:38
  • 5
    If I see a .gz, my first instinct is to tar -zxf foo.gz. Remembering that gzip is even a command takes a few more seconds. Commented Apr 20, 2016 at 14:26
  • 2
    @bgStack15 FWIW you don't need the z (or the - for that matter), most modern tars will automatically detect the file needs to be decompressed. Commented Apr 20, 2016 at 18:00
  • 2
    By default gzip will store the original file name and time stamp. You can use the -N option when decompressing to restore them. Commented Apr 22, 2016 at 4:56
  • @RossRidge thanks, I removed again the text about the original file name. Commented Apr 22, 2016 at 8:50
65

You are actually asking only half of the question. The other question being, "Why would I compress a tar file with gzip?". And the answer is not just that gzip makes the file smaller (in most cases):

tar:

  • stores filename and other metadata: mode, owner ID, group ID, filesize, modification time
  • stores a checksum (for the header only)

gzip:

  • can store the original filename, but that is optional
  • has a CRC-32 checksum over the original data
  • it compresses the file

With only tar you cannot be sure your data was not corrupted. With only gzip you cannot restore user/group ID, modification time and possible not the original filename.

The combination is more powerful than the individual commmands/formats provides, because they complement each other's features.

Improve this answer
3
  • Thanks for clarifying that! When I was reading the tar wikipedia page, I misunderstood the description to mean that the checksum was for the whole file. Commented Apr 21, 2016 at 14:23
  • This feels to me like the correct answer. I'd also add a few more reasons, which you might wanto to edit in if you agree. 1) there's no additional cost to the admin for .tgz over .tar or .gz alone: they're all just one command 2) Admins back up, copy, relocate, move a LOT of files, for a lot of different reasons; DB backups are just one of these. They can use the same workflow, tools and commands whether backing up one or multiple files; so why specialcase using the syntax of the gzip command, for the case where there is one file? Commented Apr 22, 2016 at 23:12
  • Dewi: I think it's relatively obvious why one would compress a single file, but tarring a single file is less obvious. I came to this question bc I wanted to know why one would tar a single file regardless of compression, and jofel's answer is just wanted I wanted to know. Commented Mar 29 at 13:55
30

There is a quite big advantage to using only-gzipped text files - the contents can be directly accessed with command-line tools like less, zgrep, zcat.

Improve this answer
3
  • interesting point, but the question is about a database snapshot, unlikely to be a text file, and not only-gzipped. Commented Apr 20, 2016 at 20:39
  • 9
    @underscore_d all of my database dumps (mostly mysql and pgsql) are text dumps, partly because they're more salvageable if something happens to partially-corrupt the dump, and partly because i can pre-process any restore with the usual tools (sed, awk, perl, etc) if I need to. i.e. more reliable and more useful than binary dumps. The trade-off is that text-dumps tend to be larger (who cares? disk space is cheap and we have good compression) and restores are significantly slower (but less so if you wrap the restore in a transaction). Commented Apr 21, 2016 at 0:32
  • 1
    What is the advantage of these tools over simply piping the output of a decompressor into the plain tools? Commented Apr 23, 2016 at 20:16
21

I would say it's likely that the people just don't realise they can use gzip/bzip2/xz without tar. Possibly because they come from a DOS/Windows background where it is normal for compression and archiving to be integrated in a single format (ZIP, RAR, etc).

While there may be slight advantages to using tar in some situations due to the storage of metadata or the ability to add extra files, there are also disadvantages. With a plain gzip/bzip2/xz file you can decompress it and pipe the decompressed data straight to another tool (such as your database) without ever having to store the decompressed data as a file on disk. With a tarball this is harder.

Improve this answer
2
  • 2
    With GNU tar,it takes just -O switch to output to stdout, so I wouldn't say it is much harder! Commented Apr 20, 2016 at 20:28
  • 5
    The first paragraph seems plausible enough for files using the tgz extension. However, the OP's case uses tar.gz - and if these hypothetical ex-Win/DOS users are anything like I was, the first thing they say when looking at such a file is: 'Why does it have 2 extensions?'. Then they google it and quickly get the answer, which specifically explains that tar and compression are distinct. ;-) Commented Apr 20, 2016 at 20:42
18

There is an important difference that could make using tar important under some circumstances: Besides the "metadata" that @jofel mentioned in his answer, tar records the filename in the archive. When you extract it, you get the original filename regardless of what the archive is called.

In your case the tar archive and the file it contains have the related names db.dump.tar.gz and db.tar, but suppose you rename the tar file to 20-Apr-16.dump.tgz, or whatever. Untar this with tar xvfz, and you get db.dump. For comparison, unzip 20-Apr-16.dump.gz and you've got 20-Apr-16.dump. (Edit: as pointed out in the comments, gzip also makes a record of the filename; but it's not normally used when unzipping). A tar archive can also contain a relative pathname that puts the extracted file in a subdirectory.

Your use case will dictate whether this kind of filename persistence is needed, or even wanted, or is actually undesirable. But certainly, regardless of compression, a tar archive travels differently from a regular file.

Improve this answer
3
  • 6
    gzip also records the original filename. Commented Apr 21, 2016 at 0:59
  • 8
    Yup. The name is optional in the gzip header—obviously there won't be one if you compressed the streaming output of a command—and most tools won't restore it by default (for instance, you have to use gzip --name explicitly when decompressing), but you don't have to use tar to get filename persistence. Commented Apr 21, 2016 at 3:12
  • Thanks for pointing this out, I hadn't known that. Still, since that's not the default behavior, the point stands: Distributing a file in tar format preserves the original filename (and possibly the relative path), without intervention of the recipient. Distributing a (g)zipped file doesn't. Commented Apr 21, 2016 at 14:33
8

In addition to all the other answers, I've recently struck a scripting situation where only one file was expected, but a previous employee wrote the scripts with the possibility of more than one file being generated. So files were tarred and bzipped, then transferred, and expanded.

When the process grew to the point it made a 4.3 GB file, it rolled over and made a .dump.001 file in addition to a .dump file. All the scripts just kept working.

That is proactive sysadmin laziness defined!

Improve this answer
2

I would tar a single file, to copy it preserving the timestamp (which is easily overlooked in downloads). File permissions and ownership are less important: download is a term that applies to systems which are not well integrated.

Whether tar'd or not, it is standard practice to compress the file to make downloads faster — and avoid running out of diskspace.

Improve this answer
-1

Tar is especially useful for multiple files not written to a formal file system, it always has been. If for some reason there is on occasion, only 1 file to be written it is of no real consequence. I can dd my .tar.gz directly to /dev/sdx without regard to partition or file system. It may as well be tape.

It is generally done because the script or process has been copied from heritage code. Of course there is no need to tar if there is only one file, but it leaves room for enhancement to multiple files......

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.