You are getting that error because you using an older version of bash (3.2.25).
Since Bash4, there's
&>>TARGET, which is equivalent to>> TARGET 2>&1.
Source: Appending redirected output
So, you should take in consideration an upgrade. I use bash version 4.2.45 and echo "hello" &>> file1 works like a charm for me.