6

GNU Parallel quotes replacement strings by default so that they are not expanded by the shell. But in certain cases you really want the replacement string to be interpreted by the shell.

E.g.

$ cat variables.txt
--var1 0.1 --var2 0.2
--var1 0.11 --var3 0.03

Here I want GNU Parallel to run:

myprogram --var1 0.1 --var2 0.2
myprogram --var1 0.11 --var3 0.03

How is that done?

How is it done, if only some of the replacement strings should be interpreted:

E.g.

$ ls
My file1.txt
My file2.txt

And I want this run:

myprogram --var1 0.1 --var2 0.2 'My file1.txt'
myprogram --var1 0.11 --var3 0.03 'My file1.txt'
myprogram --var1 0.1 --var2 0.2 'My file2.txt'
myprogram --var1 0.11 --var3 0.03 'My file2.txt'

1 Answer 1

6

From version 20190722 you can use uq() in a perl replacement string to make that replacement unquoted:

parallel myprogram '{=1 uq(); =}' {2} :::: variables.txt ::: My*.txt

This can not be done in earlier versions. You can, however, unquote the full command with eval. This solves the first problem, but not the second.

parallel eval myprogram {} :::: variables.txt

If you prefer all replacement strings to be unquoted, you can do that by redefining them:

parallel --rpl '{} uq()' echo {} ::: '*'

You can put the --rpl in ~/.parallel/config to make them active by default (these are simply the definitions in the source code with uq() added):

--rpl '{} uq()'
--rpl '{#} 1 $_=$job->seq(); uq()'
--rpl '{%} 1 $_=$job->slot(); uq()'
--rpl '{/} s:.*/::; uq()'
--rpl '{//} $Global::use{"File::Basename"} ||= eval "use File::Basename; 1;"; $_ = dirname($_); uq()'
--rpl '{/.} s:.*/::; s:\.[^/.]*$::; uq; uq()'
--rpl '{/.} s:.*/::; s:\.[^/.]*$::; uq()'
--rpl '{.} s:\.[^/.]*$::; uq()'
6
  • is it possible to use uq(); with a replacement string defined with --rpl? For example, in this case, is it possible to do something like this: parallel --rpl "{var} s/var/Var/; uq();" myprogram {1 var} {2} :::: variables.txt ::: My*.txt Commented Apr 23, 2021 at 12:16
  • 1
    @IsoBar Almost. ... {1var} ... (no space between 1 and var) Commented Apr 23, 2021 at 17:07
  • @OleTange - might you consider an option where replacement strings are not quoted at all. IMO quoting is gratuitous and makes assumptions that are not always correct about the environment in which the string is to appear. I would argue that the default should be not to quote them, but it is too late, and you will at least want to be backwards compatible if you were to take my suggestion. Commented Sep 2, 2021 at 17:39
  • @malcook The non-quoting by default was one of the most annoying thing I found about xargs: I was bitten by this more than once. With uq I have yet to see examples where this is not sufficient. Commented Sep 2, 2021 at 20:57
  • 1
    @OleTange - I guess each to their own.... I find that parallel 's quoting presents an issue especially since it only quotes a parameter if it "thinks" quoting will be needed using its own idiosyncratic rules. Personally, I'd much rather quote or escape as appropriate to the context (for a shell, for a url) when required rather than strip them using the {= crufty perl escape syntax =} (which you introduced at my suggestion, thank you very much). So, my Q stands: "might you consider an option where replacement strings are not quoted at all". Commented Sep 3, 2021 at 22:10

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.