Timeline for How to run groups of commands in parallel?
Current License: CC BY-SA 3.0
8 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Sep 10, 2020 at 14:25 | comment | added | peterh | @simbo1905 Thank you very much! It is very nice to hear. | |
| Sep 10, 2020 at 14:15 | comment | added | simbo1905 |
based on this answer we used while read remote_file; do echo "$remote_file $target_folder"; done < files.txt | xargs -n 2 -P 10 hdfs dfs -get which worked great at downloading hundreds of files in half the time.
|
|
| Aug 18, 2017 at 8:48 | history | edited | peterh | CC BY-SA 3.0 |
added 11 characters in body
|
| Aug 18, 2017 at 6:01 | comment | added | Ole Tange |
GNU Parallel is built to be switch compatible with xargs.
|
|
| Aug 17, 2017 at 17:38 | comment | added | peterh |
Checking the parallel, I think it nearly so many as xargs, although also its syntax is nearly so complex, if you use for complex tasks. For easy tasks, it is much simpler.
|
|
| Aug 17, 2017 at 14:18 | history | edited | peterh | CC BY-SA 3.0 |
added 25 characters in body
|
| Aug 17, 2017 at 14:13 | history | edited | peterh | CC BY-SA 3.0 |
added 1637 characters in body
|
| Aug 17, 2017 at 14:00 | history | answered | peterh | CC BY-SA 3.0 |