0

I'm an admirer of this page, i hope I find my answer here.

So, i wanted to check disk usage in particular file system for all directories excluding few directories(which i have in a file stored, gets created dynamically with some requirements).

But what i observed was when i hardcoded the values directly in exclude command, it was way fast and results were accurate. But when i was passing though commands inside exclude command. It was way more time consuming and yielded wrong results. I dont want to hardcode always, please suggest

Harcoded version (a, b are subdirectories) -- gave correct results

du -sh --exclude={a,b} filesystem/*

Command version

Assuming i have a file(directory_list.txt) with a and b directory names in it.

du -sh --exclude={`cat directory_list.txt`} /filesystem/*

Another thing which comes to my mind is as exclude is not yielding me accurate and timely results, i will select directories directly which i want

as below

ls filesystem/ | grep -v $(cat directory_list.txt) > selected.txt  --

excluding directories and selecting only needed

du -sh $(cat selected.txt)

But please suggest better way and how can i make my command in exclude command work Thanks Mohammed

3
  • 1
    Please show the exact contents of directory_list.txt. The syntax is --exclude=PATTERN, so you must make sure that {$(cat directory_list)} results in a valid pattern. Commented Jan 10, 2020 at 17:20
  • 1
    Does your du version offer the -X, --exclude-from=FILE exclude files that match any pattern in FILE option? Commented Jan 10, 2020 at 17:20
  • Assuming that directory_list.txt contains one directory per line, does du -sh $(ls filesystem|fgrep -v -f directory_list.txt) do what you want? Note that command substitution $( ... ) has some pitfalls. It will not work if there are directory names that contain spaces or other special characters. Commented Jan 10, 2020 at 17:24

2 Answers 2

1

The du command on Linux, which seems to be what you are using, offers a --exclude-from option:

du -h -s --exclude-from=directory_list.txt /filesystem/*

or

du -h -s -X directory_list.txt /filesystem/*

This assumes that the file directory_list.txt contains shell globbing patterns (not regular expressions), one per line, that should be excluded. These patterns may be pathnames, or just names of files or directories, and may include shell globbing characters.


In your command that uses --exclude={`cat directory_list.txt`}, you are trying to get the shell to do a brace expansion, using the contents of directory_list.txt. This does not work as brace expansions are carried out before command substitutions are even considered.

What you could have done, if your du didn't have --exclude-from, is to read the things to exclude from your file into an array, and while doing so, add --exclude= to every element, and then use that array in a call to du:

exclude=()
while IFS= read -r item; do
    exclude+=( --exclude="$item" )
done <directory_list.txt

du -h -s "${exclude[@]}" /filesystem/*

This assumes that, just like with --exclude-from, your file contains one pattern per line.

If you are using a shell that does not support arrays, you have the benefit of getting to type less:

set --
while IFS= read -r item; do
    set -- "$@" --exclude="$item"
done <directory_list.txt

du -h -s "$@" /filesystem/*

This uses the list of positional parameters instead of a named array.

Both of these loops would also preserve whitespace in pathnames, and would not allow the shell to erroneously expand filename globbing patterns, which is a couple of the issues with your grep -v $(cat directory_list.txt) command (another is that you'll pass most of the pathnames from the files to grep as files to search rather than as patterns to exclude).

1
  • Thank you Kusalananda for such great explanation, i was able to work fix the problem with -X command as you advised Commented Jan 14, 2020 at 12:36
-1

This doesn't answer your question exactly, but sometimes people do convulsions with du -sh in efforts to find the disk hog.

If this is the case, you should consider the tool ncdu. It is curses based and fast and will sort everything by total disk space consumed as you walk through the tree with arrow keys.

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.