Bash - iterate over last number of elements of associative array

Question

I'm trying to iterate over the last several entries of an associative array in a bash script - in this case to sum up their value. I've tried the following:

SUM="0"
for last_element in $(printf "%s\n" "${!ELEMENTS[@]}" | tail -100); do
  ELEMENT_SUM="$(echo "scale=5; $SUM +  ${ELEMENTS[$last_elements]}" | bc)"
done

What happens in practice is the the $last_element prints out the entire list (of 100 elements in this case), instead of iterating over each - and thus the ${ELEMENTS[$last_elements]} fails.

I've tried multiple approaches (including negative indexing - ${ELEMENTS[-100]} for example) with no luck. I'm not sure what it is I'm doing wrong...

Any pointers and ideas would be greatly appreciated.

Thanks!

Answer 1

Associative arrays in bash are inherently un-ordered, so there is no concept of a last element or the last 100 elements.

However, if want to sum the values of 100 elements extracted from the array in pseudo-random order, you could do something like this:

e100=$(printf "%s\n" "${elements[@]}"  | tail -n 100)
sum=$(echo ${e100//$'\n'/+} | bc)
echo $sum

The first line uses printf to store 100 elements from associative array $elements into variable $e100, separated by newline characters.

The second line uses echo and bash parameter pattern replacement to change all the newline characters to + symbols, which is then piped into bc to perform the calculation. The result is stored in variable $sum.

BTW, this uses the values of the associative array directly, there is no need to iterate over the indexes of the array.

---

There are far better ways of doing this and, as the old joke goes "well, if you wanted to go there, I wouldn't start from here", bash is not a good starting point for any kind of data processing. Use awk or perl or python or almost anything that isn't shell.

Presumably you're working with floating point numbers and have already realised that bash can only do integer arithmetic, so you're using bc to work around that. You'd be better off using a language that doesn't require you to work around the limitations to do such basic things.

Answer 2

bash associative arrays (for which you need a recent version) don't support arbitrary keys. It also has no operator for sorting lists (which you'd need for that last to make sense), and can't do floating point arithmetics.

Here, I'd use a less limited shell like zsh or a proper programming language like perl.

#! /usr/bin/perl
use List::Util "sum";

%elements = (
  "foo"  => 1.213,
  "\0\n" => 0x12,
  ''     => -3,
  "bar"  => 1e-2
  ...
);
$sum = sum(map {$elements{$_}} (sort keys %elements)[-100 .. -1]);

#! /bin/zsh -

typeset -A elements
elements=(
  foo     1.213
  $'\0\n' 0x12
  ''      -3
  bar     1e-2
  ...
)
set -o extendedglob
sum=$(( ${(@j[ + ])${(@ko)elements}[-100,-1]/(#m)*/$elements[$MATCH]} ))

Those give you the sum of the elements for the last 100 keys sorted lexicographically (memcmp()-like comparison in perl, strcoll()-like in zsh).

For zsh, values may be any arithmetic expression recognised by zsh, but as we're eventually constructing a value1 + value2 + ... expression for zsh to evaluate, beware of precedence.

For instance, if value1 is var = 3 and value2 is 4 || hash[$(some-cmd)], that ends up evaluating var = 3 + 4 || hash[$(some-cmd)] (and some-cmd will be executed btw), so you may want to make sure values are surrounded with (...) if they're not just numbers.