I have a command which I want to run on all free cores to speed up the execution time. Specifically I am running the Pitman-Yor Adaptor-Grammar Sampler software I downloaded from here
./py-cfg/py-cfg-mp -r 0 -d 10 -x 10 -D -E -e 1 -f 1 -g 10 -h 0.1 -w 1 -T 1 -m 0 -n 500 -G x.tgt y.tgt < z.tgt
I tried adding parallel -j "$(nproc)" before the command as specified in this answer
but it is generating the following error:
Error in ./py-cfg/py-cfg-mp, argc = 29, optind = 27
The answer is: It depends.
You can run the same program with different input in parallel with GNU Parallel:
parallel gzip ::: *.txt
this will run gzip on every .txt-file in the current dir.
gzip is single threaded, so each instance only uses a single CPU core.
This is not that complex.
If, however, you wanted to parallelize gzip so it used multiple cores to compress a single (big) .txt-file, the task is much more complex. In practice you would use a different program (such as pigz) that is build for that.
In other words: You cannot automatically make gzip multithreaded.
I do not know Pitman-Yor Adaptor-Grammar Sampler. If it like gzip is single-threaded, you can run multiple instances on different input (and GNU Parallel can help you do that). But you cannot automatically convert the Sampler to run on multiple cores given a single input.
If this is not clear, try making a https://stackoverflow.com/help/minimal-reproducible-example
