Skip to content

Conversation

@dferens
Copy link
Contributor

@dferens dferens commented Nov 20, 2020

Description

Code to replicate issue:

(run and press Ctrl+C)

import asyncio
import time
import aiomultiprocess

async def _main():
    # not reproducible with small number of processes
    async with aiomultiprocess.Pool(20):
        time.sleep(99999)

if __name__ == '__main__':
    try:
        asyncio.run(_main())
    except KeyboardInterrupt:
        pass

You will see a lot of stack traces like:

Traceback (most recent call last):
  File "/Users/user/venv/lib/python3.8/site-packages/aiomultiprocess/core.py", line 143, in run_async
    result: R = loop.run_until_complete(unit.target(*unit.args, **unit.kwargs))
  File "/Users/user/.pyenv/versions/3.8.5/lib/pyaio process 33621 failed
Traceback (most recent call last):
  File "/Users/user/venv/lib/python3.8/site-packages/aiomultiprocess/core.py", line 143, in run_async
    result: R = loop.run_until_complete(unit.target(*unit.args, **unit.kwargs))
  File "/Users/user/.pyenv/versions/3.8.5/lib/python3.8/asyncio/base_events.py", line 603, in run_until_complete
    self.run_forever()
  File "/Users/user/.pyenv/versions/3.8.5/lib/python3.8/asyncio/base_events.py", line 570, in run_forever
    self._run_once()
  File "/Users/user/.pyenv/versions/3.8.5/lib/python3.8/asyncio/base_events.py", line 1823, in _run_once
    event_list = self._selector.select(timeout)
  File "/Users/user/.pyenv/versions/3.8.5/lib/python3.8/selectors.py", line 558, in select
    kev_list = self._selector.control(None, max_ev, timeout)
KeyboardInterrupt
aio process 33627 failed

This PR is supposed to solve this problem.

@amyreese
Copy link
Member

amyreese commented Dec 7, 2020

Keyboard interrupt is an unfortunately complicated and nuanced problem. I think I'd prefer to see this handled using signal handlers to (IMO) ignore keyboard interrupt in the child processes (or at the very least in PoolWorker), and then allow the parent process to catch and gracefully request the child processes be terminated. Eg, for process pools, if the parent process defines a keyboard interrupt handler to gracefully stop their process, the last thing they would want is the child processes to terminate all pending work and abort immediately. As-is, this behavior just causes the parent process to hang forever waiting for results that the child processes will never deliver because they aborted. If you don't have the time to rework this, I have an idea of how I'd like to handle it, and can probably put that together soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants