Why variable values are lost after terminating the loop in bash? [duplicate]

Question

I have a very simple bash script to compute the sum of numbers appear at each line of a file (I know there are better ways for doing this but I actually need this sum as an auxiliary information and the script is supposed to something more later on). The script is as follows:

TOTAL=0;
cat $DATAFILE | while read LINE; 
do
      COUNT=`echo $LINE| awk '{print $2;}'`;
      TOTAL=$((TOTAL+COUNT));
done
echo "Total = $TOTAL";

However, I always get the output "Total = 0". Surprisingly enough if I move the last line inside the while loop, I get the correct result. For example if the input file contains

A 5
B 3
C 6

I get the output

Total = 5
Total = 8
Total = 14

But the current version always outputs 0. It seems the value assigned to the variable TOTAL is somehow lost.

Can anyone help me out with the issue?

Thanks in advance

Answer 1

This is BashFAQ #24. Fortunately, you're only hitting it here because of an unnecessary use of cat -- you have no good reason to have a pipeline in use at all.

---

The right-hand side of a pipeline (like the rest of its contents, being made up of transient subshells) exits when the pipe is done unless you have shopt -s lastpipe active.

Instead, either add the line:

shopt -s lastpipe

or restructure your loop:

while read LINE; do
      COUNT=`echo $LINE| awk '{print $2;}'`;
      TOTAL=$((TOTAL+COUNT));
done <"$DATAFILE"

...or, in a case where you really need to pipe from another process, use process substitution:

while read LINE; do
      COUNT=`echo $LINE| awk '{print $2;}'`;
      TOTAL=$((TOTAL+COUNT));
done < <(cat "$DATAFILE")

By the way, if your shebang is #!/bin/bash, not #!/bin/sh, this would be better written as follows:

total=0
while read -r _ count; do
  (( total += count ))
done <"$DATAFILE"

read can split lines on characters in IFS itself -- you don't need to use awk for that.

By convention, variable names should be all-lowercase unless they represent environment variables or shell builtins.