Process synchronization in Python

Question

I try to synchronize multiple processes by using semaphores. I thought instead of creating a function for each process, it might be possible in a more generic way, with a single function and some arguments for the dependencies:

import multiprocessing
from multiprocessing import *

class Worker:
    def __init__(self, size):
        self._semaphores = [Semaphore(0)]*(size + 1)

    def run(self, name, acquire, release):
        for i in acquire:
            self._semaphores[i].acquire()

            print('Running', name)

        for i in release:
            self._semaphores[i].release()

In this case I've got five processes. The first starts first, the second and third after the first, the fourth after the second and third, and the fifth after the fourth.

if __name__ == '__main__':
    worker = Worker(5)
    Process(target=worker.run, args=('5', [5], [])).start()
    Process(target=worker.run, args=('4', [4,4], [5])).start()
    Process(target=worker.run, args=('3', [3], [4])).start()
    Process(target=worker.run, args=('2', [2], [4])).start()
    Process(target=worker.run, args=('1', [], [2,3])).start()

The expected output would be:

Running 1
Running 2
Running 3
Running 4
Running 5

But the synchronization doesn't work as expected. The execution is random including deadlocks. Why is that?

Blender · Accepted Answer · 2013-05-21 21:54:32Z

1

[Semaphore(0)]*(size + 1) creates a list with size + 1 references to a single Semaphore object. You need to make different instances of that class:

self._semaphores = [Semaphore(0) for _ in range(size + 1)]

answered May 21, 2013 at 21:54

Blender

300k55 gold badges462 silver badges511 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

witrin Over a year ago

Thank you very much, this is absolutely correct! But there is one little thing I don't get. The fixed version doesn't force the expected order. So the output is still random but without deadlocks on a Windows system. Any idea?

Blender Over a year ago

@witrin: I'm not entirely sure why you need the semaphores. Why don't you create a process pool and use pool.map_sync to get the results back in order?

witrin Over a year ago

Because it is part of an exercise that requires the use of semaphores.

witrin Over a year ago

It seems, that the fixed code doesn't work on Windows, on Linux there are no problems anymore. Thanks!

Wayne Magor · Accepted Answer · 2016-09-11 19:21:11Z

0

The indentation of your print is incorrect which prevents ('Running', '1'). After this is fixed, 1 prints, followed by 2 and 3 intermixed, followed by 4, then 5. 2 and 3 are mixed because they are kicked off together, so operation is as expected.

answered Sep 11, 2016 at 19:21

Wayne Magor

234 bronze badges

Collectives™ on Stack Overflow

Process synchronization in Python

2 Answers 2

4 Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

Comments

Related