Skip to main content
Unix & Linux

Return to Answer

added 218 characters in body
Source Link
mikeserv
mikeserv
  • 59.4k
  • 10
  • 122
  • 242

A squashfs archive differs from most other kinds in that it is a filesystemfile-system. If you've ever booted a linux live disc then you've very likely already seen how this worksin action - very nearly all of these work by mounting a squashfs archive as their root filesystemfile-system. squashfs is supported in vanilla linux kernels since version 2.6.34. And so it is fairly universal to any modern linux system.

...to get at it. The contents of the archive will then be provided you by the kernel's native vfs as a read-only mount, and you can read all files within in the same way you would any other kind of file - and without having to decompress the archive (which is handled on the fly into disk-cache on an as-needed basis by the linux kernel itself). In fact, with the latest kernels' lz4 support, you can likely do this at native speed (and perhaps even faster in some cases) as well - though there will be increased cpu-usage for squashed file access.

To create or decompress a squashfs archive you will need the squashfs-tools package installed. It is not generally installed by distributions as default by any distribution of which I am aware, but I'm notneither am I aware of any distribution which does not provide the package via package-manager. Once installed you can create an archive like:

As you can see - it clearly respects file permissions - and even preserves and respects extended filesystem file-system file attributes (xattrs) in most cases. And the compression ratio you see noted there is in addition to my regular file-system's default lzo compression - (my root fs is btrfs and all files are compressed w/ lzo already) - which is not to mention that much of ~/Downloads is occupied by downloaded compressed archives in the first place.

It is immediately usablemountable as a filesystem itself allproper file-system in its own right:

And last, you don't need any elevated privileges to uncompressdecompress a squashfs archive, only the unsquashfs tool (which is also provided in the squashfs-tools package):

Parallel unsquashfs: Using 6 processors
933 inodes (1025 blocks) to write
 
[=================================================|] 1025/1025 100%

created 927 files
created 132 directories
created 6 symlinks
created 0 devices
created 0 fifos

this is my new file

A squashfs archive differs from most other kinds in that it is a filesystem. If you've ever booted a linux live disc then you've very likely seen how this works - very nearly all of these work by mounting a squashfs archive as their root filesystem. squashfs is supported in vanilla linux kernels since version 2.6.34. And so it is fairly universal to any modern linux system.

...to get at it. The contents of the archive will then be provided you by the kernel's native vfs as a read-only mount, and you can read all files within in the same you would any other kind of file - and without having to decompress the archive (which is handled on the fly into disk-cache on an as-needed basis by the linux kernel itself). In fact, with the latest kernels' lz4 support, you can likely do this at native speed as well - though there will be increased cpu-usage for squashed file access.

To create or decompress a squashfs archive you will need the squashfs-tools package installed. It is not generally installed by distributions as default, but I'm not aware of any distribution which does not provide the package via package-manager. Once installed you can create an archive like:

As you can see - it clearly respects file permissions - and even preserves and respects extended filesystem file attributes in most cases. And the compression ratio you see noted there is in addition to my regular file-system's default lzo compression - (my root fs is btrfs and all files are compressed w/ lzo already).

It is immediately usable as a filesystem itself all its own:

And last, you don't need any elevated privileges to uncompress a squashfs archive, only the unsquashfs tool (which is also provided in the squashfs-tools package):

Parallel unsquashfs: Using 6 processors
933 inodes (1025 blocks) to write
 
[=================================================|] 1025/1025 100%

created 927 files
created 132 directories
created 6 symlinks
created 0 devices
created 0 fifos

this is my new file

A squashfs archive differs from most other kinds in that it is a file-system. If you've ever booted a linux live disc then you've very likely already seen this in action - very nearly all of these work by mounting a squashfs archive as their root file-system. squashfs is supported in vanilla linux kernels since version 2.6.34. And so it is fairly universal to any modern linux system.

...to get at it. The contents of the archive will then be provided you by the kernel's native vfs as a read-only mount, and you can read all files within in the same way you would any other kind of file - and without having to decompress the archive (which is handled on the fly into disk-cache on an as-needed basis by the linux kernel itself). In fact, with the latest kernels' lz4 support, you can likely do this at native speed (and perhaps even faster in some cases) as well - though there will be increased cpu-usage for squashed file access.

To create or decompress a squashfs archive you will need the squashfs-tools package installed. It is not generally installed by default by any distribution of which I am aware, but neither am I aware of any distribution which does not provide the package via package-manager. Once installed you can create an archive like:

As you can see - it clearly respects file permissions - and even preserves and respects extended file-system file attributes (xattrs) in most cases. And the compression ratio you see noted there is in addition to my regular file-system's default lzo compression - (my root fs is btrfs and all files are compressed w/ lzo already) - which is not to mention that much of ~/Downloads is occupied by downloaded compressed archives in the first place.

It is immediately mountable as a proper file-system in its own right:

And last, you don't need any elevated privileges to decompress a squashfs archive, only the unsquashfs tool (which is also provided in the squashfs-tools package):

Parallel unsquashfs: Using 6 processors
933 inodes (1025 blocks) to write
[=================================================|] 1025/1025 100%

created 927 files
created 132 directories
created 6 symlinks
created 0 devices
created 0 fifos

this is my new file
Source Link
mikeserv
mikeserv
  • 59.4k
  • 10
  • 122
  • 242

If you're talking about linux systems then another option for you is squashfs. It can often achieve very high compression ratios - and the compression process itself is multi-threaded - which means that you can apply all processor cores to the compression task.

A squashfs archive differs from most other kinds in that it is a filesystem. If you've ever booted a linux live disc then you've very likely seen how this works - very nearly all of these work by mounting a squashfs archive as their root filesystem. squashfs is supported in vanilla linux kernels since version 2.6.34. And so it is fairly universal to any modern linux system.

Squash supports any of gzip, lzma, lzo, xz, or (since kernel 3.19) lz4 compression methods. To mount and access the contents of a squash archive you shouldn't need any tools at all and can just do:

mount ./img.sfs /mnt; cd /mnt

...to get at it. The contents of the archive will then be provided you by the kernel's native vfs as a read-only mount, and you can read all files within in the same you would any other kind of file - and without having to decompress the archive (which is handled on the fly into disk-cache on an as-needed basis by the linux kernel itself). In fact, with the latest kernels' lz4 support, you can likely do this at native speed as well - though there will be increased cpu-usage for squashed file access.

To create or decompress a squashfs archive you will need the squashfs-tools package installed. It is not generally installed by distributions as default, but I'm not aware of any distribution which does not provide the package via package-manager. Once installed you can create an archive like:

echo 'this is my new file' >~/Downloads/newfile.txt
mksquashfs ~/Downloads ./mysqsh.sfs -comp xz

Parallel mksquashfs: Using 6 processors
Creating 4.0 filesystem on ./mysqsh.sfs, block size 131072.
[===================================================-] 1018/1018 100%

Exportable Squashfs 4.0 filesystem, xz compressed, data block size 131072
    compressed data, compressed metadata, compressed fragments, compressed xattrs
    duplicates are removed
Filesystem size 12592.01 Kbytes (12.30 Mbytes)
    57.19% of uncompressed filesystem size (22019.04 Kbytes)
Inode table size 8482 bytes (8.28 Kbytes)
    23.91% of uncompressed inode table size (35477 bytes)
Directory table size 10210 bytes (9.97 Kbytes)
    42.90% of uncompressed directory table size (23802 bytes)
Xattr table size 3976 bytes (3.88 Kbytes)
    48.67% of uncompressed xattr table size (8170 bytes)
Number of duplicate files found 61
Number of inodes 1064
Number of files 926
Number of fragments 68
Number of symbolic links  6
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 132
Number of ids (unique uids + gids) 1
Number of uids 1
    mikeserv (1000)
Number of gids 1
    mikeserv (1000)
mksquashfs ... 7.08s user 0.35s system 462% cpu 1.607 total

As you can see - it clearly respects file permissions - and even preserves and respects extended filesystem file attributes in most cases. And the compression ratio you see noted there is in addition to my regular file-system's default lzo compression - (my root fs is btrfs and all files are compressed w/ lzo already).

It is immediately usable as a filesystem itself all its own:

 sudo mount ./mysqsh.sfs /mnt;   \
 cd /mnt; cat newfile.txt; cd -; \
 sudo umount /mnt

this is my new file

As always, root permissions are generally required for arbitrary mounts, but a squashfs mount can be named like any other in /etc/fstab if you desire it.

And last, you don't need any elevated privileges to uncompress a squashfs archive, only the unsquashfs tool (which is also provided in the squashfs-tools package):

unsquashfs -d /tmp/mysqsh ./mysqsh.sfs; \
cd /tmp/mysqsh; cat newfile.txt

Parallel unsquashfs: Using 6 processors
933 inodes (1025 blocks) to write

[=================================================|] 1025/1025 100%

created 927 files
created 132 directories
created 6 symlinks
created 0 devices
created 0 fifos

this is my new file