Skip to content

ProcessPoolExecutor executor manager thread crashes if task fails in some circumstances #119592

Description

@batterseapower

Bug report

Bug description:

import time
from concurrent.futures import ProcessPoolExecutor

def fail():
    time.sleep(1)
    raise ValueError()

executor = ProcessPoolExecutor(max_workers=1, max_tasks_per_child=1)
try:
    print(executor.submit(fail).exception() is not None)
finally:
    executor.shutdown(wait=False)

If you run this, then the following is printed to the console:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "python3.11/threading.py", line 1045, in _bootstrap_inner
    self.run()
  File "python3.11/concurrent/futures/process.py", line 364, in run
    executor._adjust_process_count()
  File "python3.11/concurrent/futures/process.py", line 759, in _adjust_process_count
    process_count = len(self._processes)
                    ^^^^^^^^^^^^^^^^^^^^
TypeError: object of type 'NoneType' has no len()

If you change the code to using wait=True, or remove the max_tasks_per_child=1 option then the message is not printed.

Probably the fix is just to check if self.processes is None before accessing it at https://github.com/python/cpython/blob/main/Lib/concurrent/futures/process.py#L762

CPython versions tested on:

3.11

Operating systems tested on:

Linux

Linked PRs

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

3.13bugs and security fixes3.14bugs and security fixesstdlibStandard Library Python modules in the Lib/ directorytopic-multiprocessingtype-bugAn unexpected behavior, bug, or error

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions