Skip to main content

You are not logged in. Your edit will be placed in a queue until it is peer reviewed.

We welcome edits that make the post easier to understand and more valuable for readers. Because community members review edits, please try to make the post substantially better than how you found it, for example, by fixing grammar or adding additional resources and hyperlinks.

Required fields*

6
  • I was not clear. I need to use a template script so that each of the process are run on a different computer. Commented Jul 23, 2020 at 3:04
  • OK, is the issue that you have n (10 in this case) cpus and an unknown number of files, which may be more or less than n? I am sorry but I am struggling to understand what the problem is. Commented Jul 23, 2020 at 4:01
  • I have (for example) exactly 10 files that match the pattern (2019 - *) and a large number of CPUs on physically different computers (for example, 1000 computers with 2 CPUs each). I want to spread the processes across the computers. I need to do this with Slurm Workload Manager. Commented Jul 23, 2020 at 4:19
  • Did my updated answer provide you with a solution? If this script is run on 10 different hosts with access to the same filesystem but each host has a different SLURM_ARRAY_TASK (in the range 1 to 10) then it provides a different filename to your program. Each file will be used once. Commented Jul 23, 2020 at 6:30
  • This works for me, but what if I have more complicated pattern than 2019 - * that can't be made in glob expansion? (or I don't know how convert find regex pattern to glob) Commented Jul 23, 2020 at 15:50