I have a short script that uses scp to copy files to a number of remote hosts (yes, I know about rdist and rsync; they both fail to work for a few of the hosts - that's not the point here; I'm only copying a few non-critical files anyway).
The meat of the script looks something like this:
for h in $HOSTS; do
echo $h
echo '----------------------------------------'
scp -r $FILES ${h}:
echo ''
done
Here is partial output from running this script:
protector
----------------------------------------
.bash_profile 100% 555 0.5KB/s 00:00
.bashrc 100% 2124 2.1KB/s 00:00
.zshenv 100% 561 0.6KB/s 00:00
.zshrc 100% 2354 2.3KB/s 00:00
.shrc 100% 1887 1.8KB/s 00:00
.bash_logout 100% 17 0.0KB/s 00:00
.logout 100% 64 0.1KB/s 00:00
.zlogout 100% 17 0.0KB/s 00:00
.vimrc 100% 717 0.7KB/s 00:00
pup
----------------------------------------
.bash_profile 100% 555 0.5KB/s 00:00
.bashrc 100% 2124 2.1KB/s 00:00
.zshenv 100% 561 0.6KB/s 00:00
.zshrc 100% 2354 2.3KB/s 00:00
.shrc 100% 1887 1.8KB/s 00:00
.bash_logout 100% 17 0.0KB/s 00:00
.logout 100% 64 0.1KB/s 00:00
.zlogout 100% 17 0.0KB/s 00:00
.vimrc 100% 717 0.7KB/s 00:00
Since this script copies to hundreds of hosts, it takes a little while, so I decided to try to use GNU Parallel to speed it up. Here is the revised version of the script utilizing parallel (please don't comment on the echo hack at the beginning, it's not relevant to the problem):
(for host in $(echo $HOSTS); do echo $host; done) | parallel "echo {}; echo '----------------------------------------' ; scp -r $FILES {}: ; echo ''"
The problem is that the output from using scp in this way looks like this:
pup
----------------------------------------
soils
----------------------------------------
Because the commands are running in parallel, the ordering is different, and parallel seems to complete much faster than a for loop of scps, but as you can see, there is no output to stdout from scp. I can't figure out why - output to stdout from echo is not being suppressed, for instance. When I try scp -v inside parallel, I get partial verbose output, but I don't get any of the usual size/time information.
Is there anybody who knows why scp output is getting suppressed?
