Running multiple Python scripts simultaneously and then sequentially

Question

I can run multiple Python scripts simultaneously from a bash script like this;

#!/bin/bash
python pr1.py & 
python pr2.py &
python aop.py &
python loader.py &

But what if I want a batch to fire simultaneously and after they've run, start some more sequentially. Will this work?:

#!/bin/bash
python pr1.py & 
python pr2.py &
python ap.py &
python loader.py
python cain.py
python able.py

Answer 1

Once you put & at the end, it runs as a background process. Hence all the scripts ending with & run in parallel.

To run the other 3 scripts in sequential order you can try both:

&& runs the next script only if the preceding script has run successfully

python loader.py && python cain.py && python able.py 

|| runs scripts sequentially irrespective of the result of preceding script

python loader.py || python cain.py || python able.py

Answer 2

On your bash script you can simply add the wait command like this:

#!/bin/bash
python pr1.py & 
python pr2.py &
python ap.py &
wait
python loader.py
python cain.py
python able.py

wait will, obviously, wait for all the jobs (the background proccess you fired) to be finished for it to continue.

Answer 3

With the & command you are running the scripts in the background. you could add a check in a loop to run the command jobs and see if it continues to return a list of jobs. when it stops you can continue with your next batch of python calls.

Answer 4

Why not try it out ?

#1.py
import time
time.sleep(3)
print("First script")

#2.py
import time
time.sleep(3)
print("Second script")

If you put the processes into background, you will see the output from both the python scripts at the same time.

#!/bin/bash
python 1.py &
python 2.py &

If you execute it without the &, then you will see the output from the second script after 6 seconds.

#!/bin/bash
python 1.py
python 2.py

PS: Be careful to take care of dependencies and concurrent access issues while running it in parallel