515

I want a bash command that I can pipe into that will sum a column of numbers. I just want a quick one liner that will do something essentially like this:

cat FileWithColumnOfNumbers.txt | sum
2

9 Answers 9

1129

Using existing file:

paste -sd+ infile | bc

Using stdin:

<cmd> | paste -sd+ | bc

Edit: With some paste implementations you need to be more explicit when reading from stdin:

<cmd> | paste -sd+ - | bc

Options used:

-s (serial) - merges all the lines into a single line

-d - use a non-default delimiter (the character + in this case)

Sign up to request clarification or add additional context in comments.

16 Comments

Just as an FYI, the -s option is in GNU paste; it is not supported by Mac OS X 10.7.4 paste. However, since the POSIX 2008 specification of paste supports -s, this is a deficiency in the Mac OS X version.
Just another FYI for OS X, I had to add a - at the end of the paste command in order for this to work on OS X 10.6.8.
With bc -l you can even add up float numbers. With cut you can select columns from the input: cat input.txt | cut -d ' ' -f 8 | paste -sd+ | bc -l will add all float numbers in column 8 of the input (space being the field separator).
Well, that would be easier with awk: awk 'END { print s } { s += $8 }' infile :)
Sorry, all you paste lovers. -1 from me for being too specific. This works ONLY if you have nothing but numbers in your file. I almost have never had that situation. Google "bash add column numbers" and you get here. Invariably, I am trying to add a column interspersed with other data (du output or the like- we are talking bash, the sysadmin's language, right?). Ghostdog's awk answer works with any data that includes a column of numbers AND other columns. It's much less fragile, almost as concise, and faster to boot.
|
260

I like the chosen answer. However, it tends to be slower than awk since 2 tools are needed to do the job.

$ wc -l file
49999998 file

$ time paste -sd+ file | bc
1448700364

real    1m36.960s
user    1m24.515s
sys     0m1.772s

$ time awk '{s+=$1}END{print s}' file
1448700364

real    0m45.476s
user    0m40.756s
sys     0m0.287s

6 Comments

Good point! On SunOS 5.8 bc even core dumps with such a big input file (see my post below).
awk is the correct tool for the job! The bc solution is OK but what happens when you need to sum two columns or perhaps filter out negative numbers. With awk you can easily and sensibly add in extra logic, with the bc solution you end up with piping through yet another command (cut or grep)
I would use awk also because it allows to initialize s to 0, by doing awk 'BEGIN{s=0}{s+=$1}END{print s}'. In this case, if an empty column is piped into paste | bc, the latter pipe will return 0 instead of null. (There may be cases where and empty column is a legitimate input).
This solution seems to round down though to a single decimal? I have a list of numbers which happen to be to 2 decimal places (dollar/cents). With the paste/bc I get 11027.24, but with the awk solution I get 11027.2. That's a pretty critical bug if you end up using this one-liner with monetary values!
@romemmy Never fear, awk can do precision: awk '{s+=$1}END{printf "%.2f\n",s}' /tmp/py.venv.strace . I'll update the answer.
|
109

The following command will add all the lines (first field of the awk output)

awk '{s+=$1} END {print s}' filename

5 Comments

This ones works with mac OS 10.15
command | paste -s -d'+' - | bc that one works on mac OS too
On my system (Ubuntu 20.04) the command with nawk is about twice faster than with gawk (default awk), and about x3 faster than the winner paste|bc command.. Tested with the 1..49999998 file.
command | paste -s -d'+' - | bc works also on AIX. :)
Upvoting, though the expression in @gbgnv's comment under ghostdog74's answer is better solution for empty stream. I needed to return 0 when stream is empty.
76

Does two lines count?

awk '{ sum += $1; }
     END { print sum; }' "$@"

You can then use it without the superfluous 'cat':

sum < FileWithColumnOfNumbers.txt
sum   FileWithColumnOfNumbers.txt

FWIW: on MacOS X, you can do it with a one-liner:

awk '{ sum += $1; } END { print sum; }' "$@"

5 Comments

@jskaggz - see my answer for a bit shorter/simpler Perl version :)
the one-liner awk works on any awk that I've tried, not just OS X
@unhammer: I don't have access to 7th Edition UNIX and its version of awk any more. I don't know whether the one-line would have worked there, but I didn't try it back then when I learned awk. You're probably right; all current versions of awk are likely to support the one-liner.
what does the $@ do? It seems to have no effect for my usage of pbpaste | awk ...
@DavidMann: "$@" is used inside a script to represent all the arguments to the script, or nothing/none if there were no arguments. If you're using the awk in pbpaste | awk …, you simply omit the "$@" (though it would usually do no damage; most interactive shells at a terminal have no 'positional parameters' so "$@" is nothing). But if you have a shell script sumcol1.sh, adding the "$@" would be sensible — you'd then use pbpaste | sumcol1.sh or similar and it would work correctly, and so would sumcol1.sh file1 file2.
21

[a followup to ghostdog74s comments]

bash-2.03$ uname -sr
SunOS 5.8

bash-2.03$ perl -le 'print for 1..49999998' > infile

bash-2.03$ wc -l infile
 49999998 infile

bash-2.03$  time paste -sd+ infile | bc
bundling space exceeded on line 1, teletype
Broken Pipe

real    0m0.062s
user    0m0.010s
sys     0m0.010s

bash-2.03$ time nawk '{s+=$1}END{print s}' infile
1249999925000001

real    2m0.042s
user    1m59.220s
sys     0m0.590s
bash-2.03$ time /usr/xpg4/bin/awk '{s+=$1}END{print s}' infile
1249999925000001

real    2m27.260s
user    2m26.230s
sys     0m0.660s

bash-2.03$ time perl -nle'
  $s += $_; END { print $s }
   ' infile
1.249999925e+15

real    1m34.663s
user    1m33.710s
sys     0m0.650s

4 Comments

+1 for pointing out the coredump in the bc version of this. I'm hesitant to change the "answer" of this question because the bc version works fine for me (i'm totalling up 30 numbers).
+1 for the perl version - fastest on my ubuntu 12.04
+1 for Perl version working on floating-point numbers. Also, for exactly what I was looking for, here's summing a particular column (column 2 indexed from zero): perl -nle '$s += (split)[2]; END { print $s }' foo.txt or using pipes: cat foo.txt | perl -nle '$s += (split)[2]; END { print $s }'.
Also: paste -sd+ -|perl -nle 'print eval'
16

You can use bc (calculator). Assuming your file with #s is called "n":

$ cat n
1
2
3
$ (cat n | tr "\012" "+" ; echo "0") | bc 
6

The tr changes all newlines to "+"; then we append 0 after the last plus, then we pipe the expression (1+2+3+0) to the calculator

Or, if you are OK with using awk or perl, here's a Perl one-liner:

$perl -nle '$sum += $_ } END { print $sum' n
6

1 Comment

For the perl, this works too: perl -nle '$s+=$_}{print $s' It is a bit simpler. :)
13
while read -r num; do ((sum += num)); done < inputfile; echo $sum

4 Comments

If you've a lot of numbers to add, this ain't so good: using @radulov's file, this took 10m28s (Ubuntu 12.04). The perl solution took 11s, awk 13s and |bc 37s.
@drevicko On the other hand, if there is not a lot of numbers to add (jskaggz tell his need is for 30 numbers), this way is the quickest, because there is no fork!
@techno True enough, though there's only one fork. On the other hand, for 30 numbers, speed is a bit irrelevant - best to go with the tool you're most familiar with, such as bash (:
He-he, this is in fact the first answer to question asked ("how to sum column with bash").
5

Use a for loop to iterate over your file …

sum=0; for x in `cat <your-file>`; do let sum+=x; done; echo $sum

2 Comments

without cat :: for s in $(< infile); do let sum+=$s ; done && echo $sum
This will not work for float numbers
4

If you have ruby installed

cat FileWithColumnOfNumbers.txt | xargs ruby -e "puts ARGV.map(&:to_i).inject(&:+)"

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.