Skip to content

Concurrency crash #776

Description

@duijf

Operating system: Ubuntu Xenial (on Travis)
Python version: 3.6.8
Black version: 19.3b0
Does also happen on master: Haven't tested but when through the commits since the release. No commits seem to touch the codepath we're seeing here.

black crashes on certain system configurations due to multiprocessing. Here is the traceback we're seeing.

|Traceback (most recent call last):
|  File "/usr/local/lib/XXX/virtualenv/bin/black", line 11, in <module>
|    sys.exit(patched_main())
|  File "/usr/local/lib/XXX/virtualenv/lib/python3.6/site-packages/black.py", line 3754, in patched_main
|    main()
|  File "/usr/local/lib/XXX/virtualenv/lib/python3.6/site-packages/click/core.py", line 764, in __call__
|    return self.main(*args, **kwargs)
|  File "/usr/local/lib/XXX/virtualenv/lib/python3.6/site-packages/click/core.py", line 717, in main
|    rv = self.invoke(ctx)
|  File "/usr/local/lib/XXX/virtualenv/lib/python3.6/site-packages/click/core.py", line 956, in invoke
|    return ctx.invoke(self.callback, **ctx.params)
|  File "/usr/local/lib/XXX/virtualenv/lib/python3.6/site-packages/click/core.py", line 555, in invoke
|    return callback(*args, **kwargs)
|  File "/usr/local/lib/XXX/virtualenv/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
|    return f(get_current_context(), *args, **kwargs)
|  File "/usr/local/lib/XXX/virtualenv/lib/python3.6/site-packages/black.py", line 435, in main
|    executor = ProcessPoolExecutor(max_workers=os.cpu_count())
|  File "/usr/lib/python3.6/concurrent/futures/process.py", line 402, in __init__
|    EXTRA_QUEUED_CALLS)
|  File "/usr/lib/python3.6/multiprocessing/context.py", line 102, in Queue
|    return Queue(maxsize, ctx=self.get_context())
|  File "/usr/lib/python3.6/multiprocessing/queues.py", line 42, in __init__
|    self._rlock = ctx.Lock()
|  File "/usr/lib/python3.6/multiprocessing/context.py", line 67, in Lock
|    return Lock(ctx=self.get_context())
|  File "/usr/lib/python3.6/multiprocessing/synchronize.py", line 162, in __init__
|    SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)
|  File "/usr/lib/python3.6/multiprocessing/synchronize.py", line 59, in __init__
|    unlink_now)
|OSError: [Errno 30] Read-only file system

This only happens when running black on more than 1 file at a time. There should probably be an option (or codepath which tries to correct this issue) for serial execution.

Metadata

Metadata

Assignees

No one assigned

    Labels

    C: packagingInstallation and packaging of BlackT: bugSomething isn't working

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions