1

I'm trying to get GNU parallel to read in a file and load BASH variables from it and then perform some action on them.

For example:

ls -1 build*/*.properties | parallel source {}; echo "var set to: $foo";

Where each of the *.properties files is of the form:

foo=bar
var=blah

However, when I try either using "source" or the "." to load the variables in the file as Bash variables, setting the variables from the file doesn't appear to work.

$ ls -1 build*/*.properties | parallel source {} ; echo "variable set to $foo";
variable set to

Can anyone tell me what I'm doing wrong here? I know there must be a simple way to do this.

Improve this question
2
  • What do you want/need to do with the variables? Commented Oct 10, 2016 at 20:51
  • The variables specify which "service" to deploy. So, it should go through all subdirectories that have a *.properties file, load their vars, including "service_name" and then for each one do "deploy $service_name". Commented Oct 10, 2016 at 22:13

2 Answers 2

Reset to default
2

The first mistake is that this:

ls -1 build*/*.properties | parallel source {}; echo "var set to: $foo";

is interpreted by Bash as:

ls -1 build*/*.properties | parallel source {}
echo "var set to: $foo";

which is clearly not what you want.

An improvement is to quote the ;

ls -1 build*/*.properties | parallel "source {}; echo var set to: $foo";

But it still does not do the right thing because variables are interpolated in double quotes. So GNU Parallel does not see the string $foo. Bash replaces $foo with the content of $foo before GNU Parallel starts. So we need to force Bash to not replace $foo with the value of $foo:

ls -1 build*/*.properties | parallel 'source {}; echo "var set to: $foo"';

Or just as good:

ls -1 build*/*.properties | parallel source {}\; echo \"var set to: \$foo\"

If you find quoting is a hazzle, then use a Bash function:

mybuild() {
    source "$1"
    echo "var set to: $foo"
}
export -f mybuild
ls -1 build*/*.properties | parallel mybuild {}
# or shorter:
parallel mybuild ::: build*/*.properties
Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

Thank you! This works for me. Can't believe just by putting single quotes around the command it makes the "source" command work.
Do you understand why?
Hmm.... no idea to be honest. Is it something to do with single vs double quotes and how they're treated by Bash?
1

parallel is a program, not a built-in command, so it will source your variables OK, but only in its context.

Variables don't propagate to parent process so you just cannot do that.

You're stuck with the non-parallel version which is

for i in build*/*.properties
do
   source $i
done

(note: same problem if you do source $i &: another process is created and you lose the result)

Improve this answer

1 Comment

Thank you for this! Would it work if I changed the logic to just be a bash script that accepts an argument (path to properties file) and the bash script then loads the variables and runs the command? This should mean it's run in the same context, no?

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.